| Date | Commit message (Collapse) |
|---|
|
This adds support for handling POST/PUT request bodies sent with
chunked transfer encodings ("Transfer-Encoding: chunked").
Attention has been paid to ensure that a client cannot OOM us by
sending an extremely large chunk.
This implementation is pure Ruby as the Ragel-based
implementation in rfuzz didn't offer a streaming interface. It
should be reasonably close to RFC-compliant but please test it
in an attempt to break it.
The more interesting part is the ability to stream data to the
hosted Rack application as it is being transferred to the
server. This can be done regardless if the input is chunked or
not, enabling the streaming of POST/PUT bodies can allow the
hosted Rack application to process input as it receives it. See
examples/echo.ru for an example echo server over HTTP.
Enabling streaming also allows Rack applications to support
upload progress monitoring previously supported by Mongrel
handlers.
Since Rack specifies that the input needs to be rewindable, this
input is written to a temporary file (a la tee(1)) as it is
streamed to the application the first time. Subsequent rewinded
reads will read from the temporary file instead of the socket.
Streaming input to the application is disabled by default since
applications may not necessarily read the entire input body
before returning. Since this is a completely new feature we've
never seen in any Ruby HTTP application server before, we're
taking the safe route by leaving it disabled by default.
Enabling this can only be done globally by changing the
Unicorn HttpRequest::DEFAULTS hash:
Unicorn::HttpRequest::DEFAULTS["unicorn.stream_input"] = true
Similarly, a Rack application can check if streaming input
is enabled by checking the value of the "unicorn.stream_input"
key in the environment hashed passed to it.
All of this code has only been lightly tested and test coverage
is lacking at the moment.
[1] - http://tools.ietf.org/html/rfc2616#section-3.6.1
|
|
|
|
|
|
That method no longer exists, but Ruby would never know until it
tried to run it. Yes, I miss my compiled languages.
|
|
This should be faster/cheaper than using an instance variable
since it's accessed in a critical code path. Unicorn was never
designed to be reentrant or thread-safe at all, either.
|
|
Give this a more palatable name and unfreeze it,
allowing users to modify it more easily.
|
|
This allows alternative I/O implementations to be easier
to use with Unicorn...
|
|
These potentially leaves an open file handle around until the
next request hits the process, but this makes the common case
faster.
|
|
readpartial is actually as low-level as sysread is,
except it's less likely to throw exceptions and
won't change the blocking/non-blocking status of
a file descriptor (we explicitly enable blocking I/O)
|
|
Since the vast majority of web traffic is GET/HEAD
requests without bodies, avoid creating a StringIO
object for every single request that comes in.
|
|
Keep in mind that it's plenty possible to use Unicorn as a
library without using Rack itself. Most of the unit tests
do not depend on Rack, for example.
|
|
This leads to a ~10% improvement in test/benchmark/request.rb
Some of these changes will need to be reworked for
multi-threaded servers (Mongrel); but Unicorn will always be
single-threaded.
|
|
It was just a waste of space and would've caused line wrapping.
This reinstates the "unicorn" prefix when we create tempfiles,
too.
|
|
StringIO.new(partial_body) does not update the offset for new
writes. So instead create the StringIO object and then syswrite
to it and try to follow the same code path used by large uploads
which use Tempfiles.
|
|
This removes the #unicorn_peeraddr methods from TCPSocket and
UNIXSocket core classes. Instead, just move that logic into the
only place it needs to be used in HttpRequest.
|
|
|
|
Otherwise applications can change them behind our back
and affect subsequent requests.
|
|
It's part of the HTTP/1.1 (rfc2616), so we might as well
handle it in there and set PATH_INFO while we're at it.
Also, make "OPTIONS *" test not fail Rack::Lint
|
|
Pass "https" to "rack.url_scheme" if the X-Forwarded-Proto
header matches "https". X-Forwarded-Proto is a semi-standard
header that Ruby frameworks seem to respect; so we use that.
We won't support ENV['HTTPS'] since that can only be set at
start time and some app servers supporting https also support
http.
Currently, "rack.url_scheme" only allows "http" and "https",
so we won't set anything else to avoid breaking Rack::Lint.
|
|
"HTTP_BODY" could conflict with a "Body:" HTTP header if there
ever is one. Also, try to hide this body from the Rack
environment before @app is called since it is only used by
Unicorn internally.
|
|
This reworks error handling throughout the entire stack to be
more Ruby-ish. Exceptions are raised instead of forcing the
us to check return values.
If a client is sending us a bad request, we send a 400.
If unicorn or app breaks in an unexpected way, we'll
send a 500.
Both of these last-resort error responses are sent using
IO#write_nonblock to avoid tying Unicorn up longer than
necessary and all exceptions raised are ignored.
Sending a valid HTTP response back should reduce the chance of
us from being marked as down or broken by a load balancer.
Previously, some load balancers would mark us as down if we close
a socket without sending back a valid response; so make a best
effort to send one. If for some reason we cannot write a valid
response, we're still susceptible to being marked as down.
A successful HttpResponse.write() call will now close the socket
immediately (instead of doing it higher up the stack). This
ensures the errors will never get written to the socket on a
successful response.
|
|
|
|
* commit 'v0.2.3':
unicorn 0.2.3
Ensure Tempfiles are unlinked after every request
Don't bother unlinking UNIX sockets
Conflicts:
lib/unicorn/socket.rb
|
|
Otherwise we bloat TMPDIR and run the host out of space, oops!
|
|
This cuts the HttpParser interface down to #execute and #reset
method. HttpParser#execute will return true if it completes and
false if it is not. http->nread state is kept internally so we
don't have to keep track of it in Ruby; removing one parameter
from #execute.
HttpParser#reset is unchanged.
All errors are handled through exceptions anyways, so the
HttpParser#error? method stopped being useful.
Also added some more unit tests to the HttpParser since I know
some folks are (rightfully) uncomfortable with changing stable C
code. We now have tests for incremental parsing.
In summary, we have:
* more test cases
* less C code
* simpler interfaces
* small performance improvement
=> win \o/
|
|
Most HTTP requests are GET requests and the majority of those
GET requests are complete after one sysread. This is especially
true since we're optimized for fast clients. So short the extra
checks and trust our HTTP parser implementation to do the right
thing (we have decent unit tests for it).
|
|
Ensure constants are used as hash keys and cleanup unused
constants. This gives a 10-15% improvement with
test/benchmark/request.rb
|
|
|
|
Fix the logic in HttpParser up front so we don't have
to mess around with the following convoluted steps:
1. setting the HTTP_CONTENT_{LENGTH,TYPE} headers
2. reading the HTTP_CONTENT_{LENGTH,TYPE} headers again
3. setting the CONTENT_{LENGTH,TYPE} based on the
HTTP_-prefixed one
4. deleting the HTTP_CONTENT_{LENGTH,TYPE} headers
(since Rack doesn't like them)
1, 2, 3 were in the C code, 4 was in Ruby.
Now the logic is:
1. if CONTENT_{LENGTH,TYPE} headers are seen, don't prefix
with "HTTP_".
All the branch logic for the new code is done at init time, too
so there's no additional overhead in the HTTP parsing phase.
There's also no additional overhead of hash lookups in the extra
steps.
|
|
Or lack thereof on POSIX.
|
|
Just in case this stupid Ruby 1.9-ism creeps up on someone; I
haven't been able to reproduce I/O corruption from the test
cases, but better safe than sorry here.
|
|
As opposed to doing this in the shell, this allows the files to
be reopened reliably after rotation.
While we're at it, use $stderr/$stdout instead of STDERR/STDOUT
since they seem to be more favored.
|
|
Avoid conflicting with existing (and future) Mongrel installs in
case either changes. Of course, this also allows us more
freedom to experiment and break the API if needed...
However, I'm only planning on making minor changes to
remove the amount of C code we have to maintain and
possibly some minor performance improvements.
|
|
Tempfile reuse was over-engineered and the problem was not
nearly as big a problem as initially thought.
Additionally, it could lead to a subtle bug in an applications
that link(2)s or rename(2)s the temporary file to a permanent
location _without_ closing it after the request is done.
Applications that suffer from the problem of directory bloat are
still free to modify ENV['TMPDIR'] to influence the creation of
Tempfiles.
|
|
Along with worker process management. This is nginx-style
inplace upgrading (I don't know of another web server that does
this). Basically we can preserve our opened listen sockets
across entire executable upgrades.
Signals:
USR2 - Sending USR2 to the master unicorn process will cause
it to exec a new master and keep the original workers running.
This is useful to validate that the new code changes took place
are valid and don't immediately die. Once the changes are
validated (manually), you may send QUIT to the original
master process to have it gracefully exit.
HUP - Sending this to the master will make it immediately exec
a new binary and cause the old workers to gracefully exit.
Use this if you're certain the latest changes to Unicorn (and
your app) are ready and don't need validating.
Unlike nginx, re-execing a new binary will pick up any and all
configuration changes. However listener sockets cannot be
removed when exec-ing; only added (for now).
I apologize for making such a big change in one commit, but once
I got the ability to replace the entire codebase while preserving
connections, it was too tempting to continue working.
So I wrote a large chunk of this while hitting
the unicorn-hello-world app with the following loop:
while curl -vSsfN http://0:8080; do date +%N; done
_Zero_ requests lost across multiple restarts.
|
|
Since we handle signals, read(2) syscalls can fail on sockets
with EINTR. Restart the call if we hit this.
|
|
Keeping I/O out of unicorn.rb
|
|
I'll be removing signal handling from worker processes...
|
|
This allows us to avoid the overhead of allocating a new buffer
each and every time we call sysread (even when just parsing
headers for GET requests).
|
|
Keep this somewhat consistent with the HttpParser API
which also exposes #reset instead of #reset!
|
|
This will help prevent TMPDIR from becoming bloated when
handling thousands of large uploads a day. This is a problem in
many UNIX filesystems (including ext3): names of entries never
expire even after files are gone and the only way to clear it is
to get rid of the directory itself.
|
|
It's pointless...
|
|
read_body can be a long-running loop; so avoid wasting CPU
cycles by repeatedly performing a hash lookup to get to
a temporary buffer.
|
|
All tests for threading and semaphores have been removed. One
test was changed because it depended on a shared variable.
Tests will be replaced with tests to do process management
instead.
|
|
Avoid conflicting with existing Mongrel libraries since
we'll be incompatible and break things w/o disrupting
Mongrel installations.
|