about summary refs log tree commit homepage
path: root/lib/unicorn/http_response.rb
DateCommit message (Collapse)
2023-06-05chunk unterminated HTTP/1.1 responses
Rack::Chunked will be gone in Rack 3.1, so provide a non-middleware fallback which takes advantage of IO#write supporting multiple arguments in Ruby 2.5+. We still need to support Ruby 2.4, at least, since Rack 3.0 does. So a new (GC-unfriendly) Unicorn::WriteSplat module now exists for Ruby <= 2.4 users.
2023-06-05Support Rack 3 and fix tests on Rack 3
Most changes are to the tests to avoid uppercase characters in header keys, which are no longer allowed in rack 3 (to allow for O(1) access). This also changes a few places where an array of headers was used to switch to a hash, as a hash is requierd in Rack 3. Newer versions of curl use a 000 http_code for invalid http codes, so switch from "42 -eq" to "500 -ne" in the test, as Rack::Lint will always raise a 500 error. There is one test that fails on OpenBSD when opening a fifo. This is unrelated to unicorn as far as I can see, so skip the remaining part of the test in that case on OpenBSD. Tests still pass on Rack 2, and presumably Rack 1 as well, though I didn't test Rack 1. Co-authored-by: Eric Wong <bofh@yhbt.net>
2017-12-16avoid reusing env on hijack
Hijackers may capture and reuse `env' indefinitely, so we must not use it in those cases for future requests. For non-hijack requests, we continue to reuse the `env' object to reduce memory recycling. Reported-and-tested-by: Sam Saffron <sam.saffron@gmail.com>
2016-01-27rack is optional at runtime, required for dev
We do not want to pull in a newer or older version of rack depending on an the application running under it requires. Furthermore, it has always been possible to use unicorn without any middleware at all. Without rack, we'll be missing descriptive status text in the first response line, but any valid HTTP/1.x parser should be able to handle it properly. ref: http://bogomips.org/unicorn-public/20160121201255.GA6186@dcvr.yhbt.net/t/#u Thanks-to: Adam Duke <adam.v.duke@gmail.com> Thanks-to: Aaron Patterson <tenderlove@ruby-lang.org>
2015-11-17http_response: allow nil values in response headers
This blatantly violates Rack SPEC, but we've had this bug since March 2009[1]. Thus, we cannot expect all existing applications and middlewares to fix this bug and will probably have to support it forever. Unfortunately, supporting this bug contributes to application server lock-in, but at least we'll document it as such. [1] commit 1835c9e2e12e6674b52dd80e4598cad9c4ea1e84 ("HttpResponse: speed up non-multivalue headers") Reported-by: Owen Ou <o@heroku.com> Ref: <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
2015-06-30http_response: reduce size of multi-line header path
This should save over 100 bytes of bytecode overhead due to reduced method dispatch points. The performance difference when this is actually hit could go either way depending on how String#<< and realloc(3) interact, but it's uncommon enough that nobody is expected to notice either way.
2015-06-30reduce constants and optimize for Ruby 2.2
Ruby (MRI) 2.1 optimizes allocations away on String#freeze with literal strings. Furthermore, Ruby 2.2 optimizes away literal string allocations when they are used as arguments to Hash#[] and Hash#[]= Thus we can avoid expensive constant lookups and cache overhead by taking advantage of advancements in Ruby. Since Ruby 2.2 has been out for 7 months, now; it ought to be safe to introduce minor performance regressions for folks using older Rubies (1.9.3+ remains supported) to benefit folks on the latest Ruby. This should recover the performance lost in the "reflect changes in Rack::Utils::HTTP_STATUS_CODES" change in synthetic benchmarks.
2015-06-30reflect changes in Rack::Utils::HTTP_STATUS_CODES
Applications may want to alter the message associated with HTTP status codes in Rack::Utils::HTTP_STATUS_CODES. Avoid memoizing status lines ahead-of-time Note: this introduces a minor performance regression, but ought to be unnoticeable unless you're running "Hello world"-type apps.
2015-06-10ensure body is closed during hijack
Middlewares such as Rack::Lock (used by Rails) break badly unless the response body is closed on hijack, so we will close it to follow the lead of other popular Rack servers. While it's unclear if there's anybody using rack.hijack with unicorn, we'll try to emulate the behavior of other servers as much as possible. ref: https://github.com/ngauthier/tubesock/issues/10
2015-06-04http_response: simplify regular expression
It's ugly and less-readable to have redundant \z statements, and according to ObjectSpace.memsize_of, this saves 4 bytes on x86-64.
2015-06-01http_response: avoid special-casing for Rack < 1.5
Rack 1.4 and earlier will soon die out, so avoid having extra, overengineered code and method dispatch to silently drop support for mis-hijacking with old Rack versions. This will cause improperly hijacked responses in all versions of Rack to fail, but allows properly hijacked responses to work regardless of Rack version. Followup-to: commit fdf09e562733f9509d275cb13c1c1a04e579a68a ("http_request: support rack.hijack by default")
2014-08-17http_response: remove Status: header
Whatever compatibility reasons which existed in 2009 likely do not exist now. Other servers (e.g. thin, puma) seem to work alright without it, so there's no reason to waste precious bytes.
2013-01-22support for Rack hijack in request and response
Rack 1.5.0 (protocol version [1,2]) adds support for hijacking the client socket (removing it from the control of unicorn (or any other Rack webserver)). Tested with rack 1.5.0.
2012-12-04fix const error responses for Rainbows!
Rainbows! relies on the ERROR_XXX_RESPONSE constants of unicorn 4.x. Changing the constants in unicorn 4.x will break existing versions of Rainbows!, so remove the dependency on the constants and generate the error response dynamically. Unlike Mongrel, unicorn is unlikely to see malicious traffic and thus unlikely to benefit from making error messages constant. For unicorn 5.x, we will drop these constants entirely. (Rainbows! most likely cannot support check_client_connection consistently across all concurrency models since some of them pessimistically buffer all writes in userspace. However, the extra concurrency of Rainbows! makes it less likely to be overloaded than unicorn, so this feature is likely less useful for Rainbows!)
2012-11-29Begin writing HTTP request headers early to detect disconnected clients
This patch checks incoming connections and avoids calling the application if the connection has been closed. It works by sending the beginning of the HTTP response before calling the application to see if the socket can successfully be written to. By enabling this feature users can avoid wasting application rendering time only to find the connection is closed when attempting to write, and throwing out the result. When a client disconnects while being queued or processed, Nginx will log HTTP response 499 but the application will log a 200. Enabling this feature will minimize the time window during which the problem can arise. The feature is disabled by default and can be enabled by adding 'check_client_connection true' to the unicorn config. [ew: After testing this change, Tom Burns wrote: So we just finished the US Black Friday / Cyber Monday weekend running unicorn forked with the last version of the patch I had sent you. It worked splendidly and helped us handle huge flash sales without increased response time over the weekend. Whereas in previous flash traffic scenarios we would see the number of HTTP 499 responses grow past the number of real HTTP 200 responses, over the weekend we saw no growth in 499s during flash sales. Unexpectedly the patch also helped us ward off a DoS attack where the attackers were disconnecting immediately after making a request. ref: <CAK4qKG3rkfVYLyeqEqQyuNEh_nZ8yw0X_cwTxJfJ+TOU+y8F+w@mail.gmail.com> ] Signed-off-by: Eric Wong <normalperson@yhbt.net>
2011-04-18documentation cleanup/reduction
Don't clutter up our RDoc/website with things that users of Unicorn don't need to see. This should make user-relevant documentation easier to find, especially since Unicorn is NOT intended to be an API.
2011-01-07http_response: do not skip Status header set by app
Rack::Lint already stops apps from using it. If a developer insists on it, then users who inspect their HTTP headers can point and laugh at them for not using Rack::Lint!
2011-01-05close client socket after closing response body
Response bodies may capture the block passed to each and save it for body.close, so don't close the socket before we have a chance to call body.close
2011-01-05http_response: simplify the status == 100 comparison
No need to preserve the response tuplet if we're just going to unpack it eventually.
2011-01-04http_response: implement httpdate in C
This can return a static string and be significantly faster as it reduces object allocations and Ruby method calls for the fastest websites that serve thousands of requests a second. It assumes the Ruby runtime is single-threaded, but that is the case of Ruby 1.8 and 1.9 and also what Unicorn is all about. This change is safe for Rainbows! under 1.8 and 1.9.
2010-12-30http_response: do not account for $, being set
It's a minor garbage reduction, but nobody uses "$,", and if they did, they'd break things in the Ruby standard library as well as Rack, so let anybody who uses "$," shoot themselves in the foot.
2010-12-26http_response: remove TODO item
An unconfigured Rainbows! (e.g. Rainbows! { use :Base }) already does keepalive and supports only a single client per-process.
2010-10-04http_response: avoid singleton method
There's no need for a response class or object since Rack just uses an array as the response. So use a procedural style which allows for easier understanding. We shall also support keepalive/pipelining in the future, too.
2010-06-28http_response: this should be a module, not a class
This affects Rainbows!, but Rainbows! is still using the Unicorn 1.x branch. While we're at it, avoid redeclaring the "Unicorn" module, it makes documentation noisier.
2010-01-05http_response: disallow blank, multi-value headers
The HeaderHash optimizations in Rack 1.1 interact badly with Rails 2.3.5 (and possibly other frameworks/apps) which set multi-value "Set-Cookie" headers without relying on the proper methods provided by Rack::Utils. While this is an issue with Rails not using properly, there may be similar apps that make this mistake and Rack::Lint does not guard against it. Rack-ML-Ref: <20100105235845.GB3377@dcvr.yhbt.net>
2009-09-27http_response: simplify and remove const dependencies
We don't need the Z constant anymore and inlining the header writing gives a small overall performance improvement in microbenchmarks. This also makes this method reentrant and thread-safe for Rainbows as well.
2009-09-16Avoid freezing objects that don't benefit from it
This gives applications more rope to play with in case they have any reasons for changing some values of the default constants. Freezing strings for Hash assignments still speeds up MRI, so we'll keep on doing that for now (and as long as MRI supports frozen strings, I expect them to always be faster for Hashes though I'd be very happy to be proven wrong...)
2009-09-09http_response: don't "rescue nil" for body.close
This can hide bugs in Rack applications/middleware. Most other Rack handlers/servers seem to follow this route as well, so this helps ensure broken things will break loudly and more consistently across all Rack-enabled servers :)
2009-09-08"encoding: binary" comments for all sources (1.9)
This ensures any string literals that pop up in *our* code will just be a bag of bytes. This shouldn't affect/fix/break existing apps in most cases, but most constants will always have the "correct" encoding (none!) to be consistent with HTTP/socket expectations. Since this comment affects things only on a per-source basis, it won't affect existing apps with the exception of strings we pass to the Rack application. This will eventually allow us to get rid of that Unicorn::Z constant, too.
2009-09-03Support HTTP/0.9 entity-body-only responses
HTTP/0.9 GET requests expect responses without headers. Some weird applications/tools still use the ancient HTTP/0.9 protocol for weird reasons, so we'll support them. ref: rfc 1945, section 4.1
2009-08-15http_response: pass through unknown status codes
This lets clients can pass through newly-invented status codes that Rack does not know about.
2009-06-09Avoid duplicating the "Z" constant
Trying not to repeat ourselves. Unfortunately, Ruby 1.9 forces us to actually care about encodings of arbitrary byte sequences.
2009-05-13http_response: allow string status codes
Rack::Lint says they just have to work when to_i is called on the status, so that's what we'll do.
2009-05-13Require Rack for HTTP Status codes
Preventing needless duplication since Rack already has these codes for us. Also, put the status codes in HttpResponse since nothing else needs (or should need) them.
2009-05-03http_response: luserspace buffering is barely faster
Simpler code on our end can be just a tick faster because syscalls are still not as cheap as normal functions and this still manages to play well with our lack of keepalive support as closing the socket will flush it immediately.
2009-04-23http_response: minor performance gains
Avoid creating garbage every time we lookup the status code along with the message. Also, we can use global const arrays for a little extra performance because we only write one-at-a time Looking at MRI 1.8, Array#join with an empty string argument is slightly better because it skips an append for every iteration.
2009-04-23http_response: just barely faster
2009-04-21http_response: small speedup by eliminating loop
This avoids creating yet another binding. socket.syswrite() should really only be called once since we use blocking sockets, but just in case, we emulate a do+while loop with begin+while
2009-04-16Small garbage reduction in HttpResponse
Avoid creating new string objects and then discarding them right away by stuffing non-constant but always-present headers into the initial output.
2009-04-16ensure responses always have the "Status:" header
There are weird (and possibly broken) clients out there that require it despite being present in the first line of the response. So be nice and accomodate them. Keep in mind that the Rack SPEC explicitly forbids this header from being in the headers returned by the Rack-based application; so we have to always inject it ourselves and ignore it if the application sets it.
2009-03-27Always try to send a valid HTTP response back
This reworks error handling throughout the entire stack to be more Ruby-ish. Exceptions are raised instead of forcing the us to check return values. If a client is sending us a bad request, we send a 400. If unicorn or app breaks in an unexpected way, we'll send a 500. Both of these last-resort error responses are sent using IO#write_nonblock to avoid tying Unicorn up longer than necessary and all exceptions raised are ignored. Sending a valid HTTP response back should reduce the chance of us from being marked as down or broken by a load balancer. Previously, some load balancers would mark us as down if we close a socket without sending back a valid response; so make a best effort to send one. If for some reason we cannot write a valid response, we're still susceptible to being marked as down. A successful HttpResponse.write() call will now close the socket immediately (instead of doing it higher up the stack). This ensures the errors will never get written to the socket on a successful response.
2009-03-22HttpResponse: speed up non-multivalue headers
The extra split slows things down a little as as it generates an array with a new string value and adds an extra loop to iterate through.
2009-03-21Handle Rack multivalue headers correctly
Rack uses a single newline character to represent multi-value headers. Thus { 'Set-Cookie' => "foo=bar\nbar=foo" } will get you: Set-Cookie: foo=bar Set-Cookie: bar=foo While RFC2616 says you can combine headers as: Set-Cookie: foo=bar,bar=foo There are probably HTTP clients out there that don't handle things correctly so don't bother... Additionally, don't bother doing duplicate suppression anymore. Just assume Rack or a higher layer knows what it's doing regarding duplicates and we'll get a Hash most of the time anyways.
2009-03-20HttpResponse: close body if it can close
The body could be an IO object that is closeable. So make sure we close it if it can be closed to avoid file descriptor leakage.
2009-02-14HttpResponse: remove crack-addled HTTP_STATUS_HEADERS hash
This also fixes a subtle bug in header generation when the +$,+ ($OFS) variable is defined to something other than nil or "" I'm really wondering what kind of drugs I was on (or _not_ on) when I modified some of this from the Mongrel source.
2009-02-13HttpResponse: use each instead of each_pair to iterate
Rack spec specifies #each must be defined, not #each_pair. Hash#each_pair was marginally faster in Ruby 1.8, but in Ruby 1.9.1, Hash#each and Hash#each_pair are the same function.
2009-02-09Refactor and get exec + FD inheritance working
Along with worker process management. This is nginx-style inplace upgrading (I don't know of another web server that does this). Basically we can preserve our opened listen sockets across entire executable upgrades. Signals: USR2 - Sending USR2 to the master unicorn process will cause it to exec a new master and keep the original workers running. This is useful to validate that the new code changes took place are valid and don't immediately die. Once the changes are validated (manually), you may send QUIT to the original master process to have it gracefully exit. HUP - Sending this to the master will make it immediately exec a new binary and cause the old workers to gracefully exit. Use this if you're certain the latest changes to Unicorn (and your app) are ready and don't need validating. Unlike nginx, re-execing a new binary will pick up any and all configuration changes. However listener sockets cannot be removed when exec-ing; only added (for now). I apologize for making such a big change in one commit, but once I got the ability to replace the entire codebase while preserving connections, it was too tempting to continue working. So I wrote a large chunk of this while hitting the unicorn-hello-world app with the following loop: while curl -vSsfN http://0:8080; do date +%N; done _Zero_ requests lost across multiple restarts.
2009-02-09HttpResponse: use unbuffered I/O for writing, too
Avoid needless userspace copies and craziness. We'll need to handle EINTR since writes to sockets means stupid things like this, but it's a small cost...
2009-02-09Get rid of HeaderOut and simplify HttpResponse
Just stuff what little logic we had for it into HttpResponse since Rack takes care of the rest for us. Put the HTTP_STATUS_HEADERS hash in HttpResponse since we're the only user of it. Also, change HttpResponse.send to HttpResponse.write to avoid overriding the default method.
2009-02-09Simplify HttpResponse since we only handle Rack now
The previous API was very flexible, but I don't think many people really cared for it... We now repeatedly use the same HeaderOut in each process since I completely don't care for multithreading.