about summary refs log tree commit homepage
DateCommit message (Collapse)
2009-09-04unicorn 0.91.0 v0.91.0
2009-09-04Redirect files in binary mode
Avoid potential issues that can arise from logging any weird characters that may not be supported in the current encoding.
2009-09-03Support HTTP/0.9 entity-body-only responses
HTTP/0.9 GET requests expect responses without headers. Some weird applications/tools still use the ancient HTTP/0.9 protocol for weird reasons, so we'll support them. ref: rfc 1945, section 4.1
2009-09-03http: add HttpParser#headers? method
This method determines if there are headers in the request. Simple HTTP/0.9 requests did not have headers in the request (and our responses we make should not have them, either).
2009-09-02http: SERVER_PROTOCOL matches HTTP_VERSION
And it'll default to HTTP/0.9 if HTTP_VERSION is not specified (as version-less HTTP requests imply HTTP/0.9.
2009-09-02test to ensure stderr goes *somewhere* when daemonized
Followup to commit 7f74a16406c92c4362ac20af4ccb8bc821cf978b, we want to ensure error messages do not get swallowed up into /dev/null when daemonizing; so we defer the default redirects to "/dev/null" to as late as possible.
2009-09-02launcher: defer daemonized redirects until config is read
Otherwise errors in the Unicorn-specific config files can get error messages swallowed up when daemonizing.
2009-09-02test_http_parser_ng: fix failing HTTP/0.9 test case
SERVER_PROTOCOL is actually defined as "HTTP/1.1 even though it should not be for HTTP/0.9 responses.
2009-09-01http: support for simple HTTP/0.9 GET requests
HTTP/0.9 only supports GET requests and didn't require a version number in the request line. Additionally, only a single CRLF was required. Note: we don't correctly generate HTTP/0.9 responses, yet.
2009-09-01http: extension-methods allow any tokens
ref: rfc 2616, section 5.1.1 http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1.1 Current version of Rack::Lint agrees with us, too. While I've yet to encounter actual usage of non-upper REQUEST_METHODs, we might as well support what Rack supports.
2009-08-29unicorn_http: "fix" const warning
neither buffer nor p should be const (since we modify buffer in $snake_upcase_char), but this is a much smaller change _for now_
2009-08-20tee_input: fix rdoc
the docs for the TeeInput class was clobbering the Unicorn module documentation instead.
2009-08-18http: support for multi-line HTTP headers
While I still consider pound to be irrelevant, but I still sometimes get hand-crafted HTTP requests that come in with multiline headers. Since these are part of the HTTP specs and not difficult to support, we might as well support them for the sake of completeness.
2009-08-18http: make strings independent before modification
Ruby strings may be copy-on-write (multiple #dup'ed strings can point to the same internal buffers). So since HttpParser#headers modifies its buffer argument, we'll need to make sure the String object we have points to its own private buffer. This doesn't affect Unicorn except in a to-be-written test case that relies on a #dup'ed String.
2009-08-18examples/echo: "Expect:" value is case-insensitive
From RFC 2616, section 14.20: > Comparison of expectation values is case-insensitive for > unquoted tokens (including the 100-continue token), and is > case-sensitive for quoted-string expectation-extensions.
2009-08-17Documentation updates
* Documented Unicorn::HttpParser API methods * Keep GPL2 (COPYING) as-is without RDoc formatting. * The auto-generated index.html is stupid, replace it with README which looks saner.
2009-08-16unicorn 0.90.0 v0.90.0
2009-08-16app/inetd: explicitly close pipe descriptors on CatBody#close
Since Rack permits body objects to have #close called on them, we can safely close our pipe readers immediately instead of waiting on the GC to close them (like we do for TeeInput tempfiles).
2009-08-15Remove explicit requires for Rack things
Rack is autoload-based and so are we.
2009-08-15Make launchers __END__-aware
I've noticed rackup has been __END__-aware as of 7b6046b764eafd332b3b2d9d93b3915c425fae54 in Rack upstream
2009-08-15TODO: remove keep-alive/pipelining
The code I was _about_ to commit to support them was too ugly and the performance benefit in real applications/clients is unproven. Support for these things also had the inevitable side effect of adding overhead to non-persistent requests. Worst of all, interaction with people tweaking TCP_CORK/TCP_NOPUSH suddenly becomes an order of magnitude more complex because of the _need_ to flush the socket out in between requests (for most apps). Aggressive pipelining can actually help a lot with a direct connection to Unicorn, not via proxy. But the applications (and clients) that would benefit from it are very few and moving those applications away from HTTP entirely would be an even bigger benefit. The C/Ragel parser will maintain keepalive support for now since I've always intended for that to be used in more general-purpose HTTP servers than Unicorn. For documentation purposes, the keep-alive/pipelining patch is included below in its entirety. I don't have the TCP_CORK/TCP_NOPUSH parts in there because that's when I finally gave up and decided it would not be worth supporting. diff --git a/lib/unicorn.rb b/lib/unicorn.rb index b185b25..cc58997 100644 --- a/lib/unicorn.rb +++ b/lib/unicorn.rb @@ -439,22 +439,24 @@ module Unicorn # once a client is accepted, it is processed in its entirety here # in 3 easy steps: read request, call app, write app response def process_client(client) + response = nil client.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) - response = app.call(env = REQUEST.read(client)) - - if 100 == response.first.to_i - client.write(Const::EXPECT_100_RESPONSE) - env.delete(Const::HTTP_EXPECT) - response = app.call(env) - end + begin + response = app.call(env = REQUEST.read(client)) - HttpResponse.write(client, response) + if 100 == response.first.to_i + client.write(Const::EXPECT_100_RESPONSE) + env.delete(Const::HTTP_EXPECT) + response = app.call(env) + end + end while HttpResponse.write(client, response, HttpRequest::PARSER.keepalive?) + client.close # if we get any error, try to write something back to the client # assuming we haven't closed the socket, but don't get hung up # if the socket is already closed or broken. We'll always ensure # the socket is closed at the end of this function - rescue EOFError,Errno::ECONNRESET,Errno::EPIPE,Errno::EINVAL,Errno::EBADF - client.write_nonblock(Const::ERROR_500_RESPONSE) rescue nil + rescue EOFError,Errno::ECONNRESET,Errno::EPIPE,Errno::EINVAL, + Errno::EBADF,Errno::ENOTCONN client.close rescue nil rescue HttpParserError # try to tell the client they're bad client.write_nonblock(Const::ERROR_400_RESPONSE) rescue nil diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb index 1358ccc..f73bdd0 100644 --- a/lib/unicorn/http_request.rb +++ b/lib/unicorn/http_request.rb @@ -42,6 +42,7 @@ module Unicorn # to handle any socket errors (e.g. user aborted upload). def read(socket) REQ.clear + pipelined = PARSER.keepalive? && 0 < BUF.size PARSER.reset # From http://www.ietf.org/rfc/rfc3875: @@ -55,7 +56,9 @@ module Unicorn TCPSocket === socket ? socket.peeraddr.last : LOCALHOST # short circuit the common case with small GET requests first - PARSER.headers(REQ, socket.readpartial(Const::CHUNK_SIZE, BUF)) and + PARSER.headers(REQ, + pipelined ? BUF : + socket.readpartial(Const::CHUNK_SIZE, BUF)) and return handle_body(socket) data = BUF.dup # socket.readpartial will clobber data diff --git a/lib/unicorn/http_response.rb b/lib/unicorn/http_response.rb index 5602a43..e22b5f9 100644 --- a/lib/unicorn/http_response.rb +++ b/lib/unicorn/http_response.rb @@ -20,12 +20,15 @@ module Unicorn # to most clients since they can close their connection immediately. class HttpResponse + include Unicorn::SocketHelper # Every standard HTTP code mapped to the appropriate message. CODES = Rack::Utils::HTTP_STATUS_CODES.inject({}) { |hash,(code,msg)| hash[code] = "#{code} #{msg}" hash } + CLOSE = "close".freeze # :nodoc + KEEPALIVE = "keep-alive".freeze # :nodoc # Rack does not set/require a Date: header. We always override the # Connection: and Date: headers no matter what (if anything) our @@ -34,7 +37,7 @@ module Unicorn OUT = [] # :nodoc # writes the rack_response to socket as an HTTP response - def self.write(socket, rack_response) + def self.write(socket, rack_response, keepalive = false) status, headers, body = rack_response status = CODES[status.to_i] || status OUT.clear @@ -50,6 +53,10 @@ module Unicorn end end + if keepalive && (0 == HttpRequest::BUF.size) + keepalive = IO.select([socket], nil, nil, 0.0) + end + # Rack should enforce Content-Length or chunked transfer encoding, # so don't worry or care about them. # Date is required by HTTP/1.1 as long as our clock can be trusted. @@ -57,10 +64,10 @@ module Unicorn socket.write("HTTP/1.1 #{status}\r\n" \ "Date: #{Time.now.httpdate}\r\n" \ "Status: #{status}\r\n" \ - "Connection: close\r\n" \ + "Connection: #{keepalive ? KEEPALIVE : CLOSE}\r\n" \ "#{OUT.join(Z)}\r\n") body.each { |chunk| socket.write(chunk) } - socket.close # flushes and uncorks the socket immediately + keepalive ensure body.respond_to?(:close) and body.close rescue nil end
2009-08-15http: support for "Connection: keep-alive"
ab still sends this with HTTP/1.0 requests, which is unfortunate, but synthetic benchmarks are good for marketing purposes!
2009-08-15update TODO
2009-08-15const: remove unused constants
2009-08-15http_response: pass through unknown status codes
This lets clients can pass through newly-invented status codes that Rack does not know about.
2009-08-15Fix documentation for Util.reopen_logs
2009-08-15GNUmakefile: Fix "install" target
2009-08-15http: fix warning when sizeof(off_t) == sizeof(long long)
We need to declare constants for 64-bit off_t explicitly with the "LL" suffix on 32-bit machines.
2009-08-15Drop the micro benchmarks
It's not worth the effort to keep the internal API consistent between non-bugfix versions.
2009-08-15tee_input: make interface more usable outside of Unicorn
TeeInput being needed is now (once again) an uncommon code path so there's no point in relying on global constants. While we're at it, allow StringIO to be used in the presence of small inputs; too.
2009-08-15http_request: reinstate empty StringIO optimization
This makes a noticeable difference on light GET/HEAD requests. Heck, even the tests run a few seconds faster.
2009-08-15README: everybody loves Ruby DSLs
2009-08-12http: freeze fields when creating them, always
Otherwise we'll be creating an extra garbage string because rb_hash_aset can't tell the string isn't being used elsewhere and creates a new frozen one.
2009-08-11http: add "HttpParser#keepalive?" method
This should be used to detect if a request can really handle keepalives and pipelining. Currently, the rules are: 1. MUST be a GET or HEAD request 2. MUST be HTTP/1.1 3. MUST NOT have "Connection: close" set This also reduces the amount of garbage we create by globalizing constants and using them whenever possible.
2009-08-10http: add CONST_MEM_EQ macro
For comparing a raw memory space against a constant
2009-08-10http: rename read_body to filter_body
This method is strictly a filter, it does no I/O so "read" is not an appropriate name to give it.
2009-08-10test_signals: unlink log files of KILL-ed process
The normal at_exit handlers can't work here
2009-08-10Documentation updates
2009-08-09test_exec: wait for worker readiness
Otherwise Ruby could get confused and not be able to reap the process correctly (and thus wait a long time for timeout).
2009-08-09test_util: explicitly close tempfiles for GC-safety
Otherwise they might be picked up by the GC during the other tests (exposed by Ruby 1.9.1-p243).
2009-08-09http: join repeated headers with a comma
Since Rack requires a Hash object, this is joined in in accordance with rfc2616, section 4.2[1]. Of course, it's up to the framework or application to handle such requests. I could optimize this to avoid creating a single garbage String object, but I don't think it's common enough to worry about... [1] - http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2
2009-08-09http: add test for invalid trailer
Just in case clients decide to get cute.
2009-08-09http: unit tests for overflow and bad lengths
We're bound by the maximum value of off_t when handling input bodies (we need to buffer to disk). Also ensure we stop bad clients that send us unparseable lengths.
2009-08-09Switch to Ragel/C-based chunk/trailer parser
This should be more robust, faster and easier to deal with than the ugly proof-of-concept regexp-based ones.
2009-08-09test_upload: extra CRLF is needed
Our current TrailerParser is liberal and does not require it, but the to-be-activated Ragel one is not.
2009-08-09http: preliminary chunk decoding
2009-08-09http: process Content-Length and Transfer-Encoding
Explicitly track if our request will need Content-Length or chunked body decoding.
2009-08-09http: generic C string vs VALUEs comparison function
The macro version lets us painlessly compare Ruby strings against constant C strings. It wraps a C version of the function which is necessary to avoid double evaluation while preventing the user from having to type sizeof or strlen.
2009-08-09http: prepare http_parser struct for body processing
We'll need to keep track of a few more things to account for content/chunk length and the temporary file descriptor we'll need to use.
2009-08-09http: move non-Ruby-specific macros c_util.h