| Date | Commit message (Collapse) |
|---|
|
We depend on the just-released Unicorn 0.94.0 for the fixed
trailer handling. As with `unicorn', the `rainbows' executable
now sets and respects ENV["RACK_ENV"]. Also small fixes and
cleanups including better FreeBSD 7.2 compatibility and
less likely to over-aggressively kill slow/idle workers
when a very low timeout is set.
Eric Wong (20):
rev: split out heartbeat class
bump Unicorn dependency to (consistently) pass tests
tests: avoid single backquote in echo
event_machine: avoid slurping when proxying
tests: make timeout tests reliable under 1.9
thread_pool: comment for potential SMP issue under 1.9
Allow 'use "model"' as a string as well as symbol
Rev model is the only user of deferred_bodies
ev_core: use Tempfile instead of Unicorn::Util::tmpio
ev_core: ensure quit is triggered on all errors
rainbows: set and use process-wide ENV["RACK_ENV"]
http_server: add one second to any requested timeout
thread_pool: update fchmod heartbeat every second
t0004: tighten up timeout test
ev_core: remove Tempfile usage once again
cleanup: remove unused t????.ru test files
tests: staggered trailer upload test
ensure RACK_ENV is inherited from the parent env
t0100: more precise `expr` usage
|
|
We're simply too uncomfortable with the weird GC issues
associated with Tempfile and having linked temporary files at
all. Instead just depend on the #size-aware TmpIO class that
Unicorn 0.94.0 provides for us.
|
|
Like the rest of the concurrency models. This gives us
more flexibility in case a process-wide blocking operation
started during an "unlucky" period when the join timeout
was about to expire.
|
|
This is because our timeout implementations are less precise
than Unicorn. Since we handle multiple clients with the same
process, we sacrifice precision for performance and instead
implement our fchmod heartbeats at a fixed rate, as doing
fchmod() repeated for short-lived connections would hurt
performance and we have to call fchmod even when connected
clients are idle.
|
|
Just in case something goes wrong with the write
or the logger, make sure we've triggered a quit.
|
|
Since we're geared towards slower clients, we may be able to
make gains from using userspace IO buffering. This allows us to
avoid metadef-ing a #size method for every File we allocate
and save memory.
|
|
We don't use it in EventMachine since EM has its own
built-in ways to handle deferred bodies.
|
|
Since const_get works with a string as well as a symbol,
allow that to be used. It's easier and simpler to just
allow strings as use arguments than to error check and
raise exceptions.
So both of the following should now work:
Rainbows! do
use :Revactor
end
Rainbows! do
use "Revactor"
end
Rainbows! will always use the symbol variant internally,
however, so applications can alway expect env['rainbows.model']
to be a symbol.
|
|
The problem is unconfirmed at the moment, but I've long
anticipated it. I just need to remember the next time I
log into a monster machine.
|
|
Avoid slurping in case we're a fast backend writing to a slow
client. This should prevent our memory usage from exploding
when clients are reading slowly.
|
|
This module will be reused in upcoming Rev-derived concurrency
models.
|
|
Basic single-threaded EventMachine support is now included. It
supports async_synatra[1] via the "async.callback" Rack
environment[2]. For EventMachine, we rely on the updated
attach/watch API in EventMachine 0.12.10.
As Revactor 0.1.5 is now available, our Revactor support now
depends on it as it adds the ability to listen on UNIX domain
sockets.
For developers/QA folks, the integration tests are completely
revamped for easier maintenance when new concurrency models are
introduced and should also produce TAP-compliant output. The
test suite remains highly parallelizable using GNU make.
There are immediate plans to expand support for both Rev and
EventMachine to support use with threaded application dispatch.
Eric Wong (41):
rev: remove Revactor-specific workaround
README: change ordering of concurrency model listing
tests: more correct HTTP/0.9 test
test-lib: avoid stalling due to bad FIFO handling
rev: fix static file responses under HTTP/0.9
add news bodies to site NEWS.atom.xml
tests: avoid needlessly remaking "rainbows"
initial EventMachine support
tests: hopefully fix stalls in input trailer tests
tests: avoid race condition in reopen logs test
tests: prefer "RUBY" to lowercased "ruby"
tests: common setup and wait_start functions
tests: add a TAP producer shell library
tests: port all existing tests to TAP library
tests: remove symlinks and small files, use Make
t9000: bail if run with an unsupported/pointless model
tests: allow "make $model" to run tests for that model
rev: spell ECONNABORTED correctly
rev/evma: move common code for event models into ev_core
ev_core: do not drop deferred bodies on graceful quits
eventmachine: get basic tests working
rev: do not File.expand_path on result of body.to_path
eventmachine 0.12.8 passes all tests
tests: make large file memory tests more reliable
eventmachine: require EM 0.12.10
update gem dependencies in comments/local.mk.sample
rev: enforce Rev::VERSION >= 0.3.0
eventmachine: add async_sinatra support
tests: only load Revactor tests under 1.9.1
tests: gracefully exit if EventMachine is not available
tests: error out if socat + curl aren't reachable
thread*: fix MRI 1.8.6 compatibility
local.mk.sample: cleanups and minor reorg
eventmachine: remove unnecessary ivar assignment
eventmachine: document our support of "async_synatra"
doc: Update TODO and README
tests: generate all dependencies atomically
app_pool: update RDoc
test-lib: DWIM handling of temp UNIX sockets
revactor: require 0.1.5, remove 0.1.4 workarounds
gemspec: bump up Unicorn dep version to 0.93.4
[1] http://github.com/raggi/async_sinatra
[2] this is not 100% Rack::Lint compatible, but we'll let it
slide since there are already folks depending on
the async_sinatra gem
|
|
Also new are added basic HTTP tests for UNIX domain socket
handling (for all models, now, of course).
|
|
Now that we have EM support (which is basically like
Rev), update our docs for that. Also, expand on why
Revactor isn't supported just yet...
|
|
ref: http://github.com/raggi/async_sinatra
|
|
It shouldn't be needed, really, favor simpler code
here until proven otherwise.
|
|
Array#count is not available until MRI 1.8.7
|
|
This is should be compatible with how the Thin webserver
provides async callback support.
See http://github.com/raggi/async_sinatra for the details
|
|
We depend on the ability to create listeners from existing IO
objects here.
|
|
The EM::attach/EM::watch API changed incompatibly
in 0.12.10
|
|
This means Rainbows::DevFdBody async responses and large
file streaming without slurping.
This is only with eventmachine 0.12.8, it looks like 0.12.10
changes the attach/watch API...
|
|
Rack::Lint requires File::exist? to be true on this, so
there's no need to expand the path name to resolve it
here.
|
|
log reopens, graceful shutdown, HTTP error responses
should all be working now.
|
|
Graceful quit means we finish sending everything we have before
exiting. Additionally, only signal quits after we've queued
the error response up.
|
|
EventMachine and Rev models seem to be able to share a lot of
common code, so lets share. We may support Packet in the
future, too, and end up with a similar programming model there
as well.
|
|
Odd that Ruby didn't barf on this until now...
|
|
Working for simple GET requests, completely untested otherwise
and most definitely not able to handle async/Comet-style
requests yet, either. No tests for this are enabled.
This could share a lot of code with the existing Rev model.
|
|
Since HTTP/0.9 responses have no headers to write, the
on_write_complete handler we rely on never got triggered,
leading to additional reads to never get queued up.
Additionally, we need to explicitly detect and close client
sockets if we've written the last response body since HTTP/0.9
clients never know when it's time to close a connection.
|
|
This was blindly copied and pasted from the Revactor code.
|
|
The major feature of this release is the new DeferredResponse
middleware for the Rev-based concurrency model. It should be
transparently compatible with non-Rev models, as well. As a
pleasant side effect, this change also allows large files to be
streamed to the client with Rev as the socket becomes writable
instead of slurping the entire file into an IO::Buffer first.
Bugfixes to graceful shutdowns support for all concurrency
models. The Rev-based model also gets a working heartbeat
mechanism (oops!) and fixed HTTP/1.1 pipelining support.
Eric Wong (37):
app_pool: note it being currently broken with Revactor
Revactor tests can sleep more easily
tests: sleep.ru handles "Expect: 100-continue"
Fix graceful shutdown handling of Thread* models harder
DRY setting of rack.multithread
test-lib: dbgcat adds headers with key name
use timeout correctly to join threads on SIGQUIT
Rev: simplification to error handling
tests: sleep.ru slurps rack.input stream
refactor graceful shutdowns again, harder
tests: introduce require_for_model function
tests: add unbuffered tee(1)-like helper
tests: rack.input trailer tests for all models
tests: fix issues with non-portable shell constructs
tests: fix random_blob dependency
tests: factor out a common parser error "library"
tests: DRY setting of the "model" environment var
tests: DRY Ruby requires based on model
test-lib: quiet down pipefail error message
tests: DRY require tests for Rev/Revactor
rev: handle fully-buffered, pipelined requests
rev: avoid stack overflow through pipelining
tests: common basic HTTP tests for all models
tests: rack.input hammer concurrency testing
tests: for log reopening for all concurrency models
http_response: filter out X-Rainbows-* headers
rev: fix heartbeat timeouts
revactor: switch to a 1 second heartbeat
rev: async response bodies with DevFdResponse middleware
tests: more reliable error checking
tests: DWIM FIFO creation
tests: predictable and simpler tempfile management
rev: AsyncResponse => DeferredResponse API cleanup
rev: update documentation for this model
TUNING: update documentation notes
TODO: update with new items
local.mk.sample: sync with BDFL's version
|
|
|
|
The "async" moniker wasn't appropriate since this API also
handles static files without slurping, so "deferred" is a more
appropriate term (even if I have trouble speling words with
double conssonants in them).
The DeferredResponse.write method now emulates the
HttpResponse.write method for consistency.
|
|
This new middleware should be a no-op for non-Rev concurrency
models (or by explicitly setting env['rainbows.autochunk'] to
false).
Setting env['rainbows.autochunk'] to true (the default when Rev
is used) allows (e)poll-able IO objects (sockets, pipes) to be
sent asynchronously after app.call(env) returns.
This also has a fortunate side effect of introducing a code path
which allows large, static files to be sent without slurping
them into a Rev IO::Buffer, too. This new change works even
without the DevFdResponse middleware, so you won't have to
reconfigure your app.
This lets us epoll on response bodies that come in from a pipe
or even a socket and send them either straight through or with
chunked encoding.
|
|
Like everybody else... Closing the listener sockets doesn't seem
to wakeup the actors reliably and since it's easier to use a 1
second heartbeat than correct signal/messaging for all the rest
of the other clients, we'll just do that instead of relying on
one-off signal handlers.
|
|
Oops, looks like they were never implemented at all.
|
|
We'll be using some custom headers to craft responses
|
|
Handling HTTP pipelining through recursion is not good since
several hundred kilobytes worth of GET/HEAD requests can be a
LOT of GET/HEAD requests...
|
|
This leaves us vulnerable to stack overflows through excessive
pipelining. The next patch will fix things hopefully.
|
|
We use the "G" global constant from the Rev model everywhere
to simplify things a little.
Test cases are more consistent now, too.
|
|
on_write_complete has no chance of being called
there so remove the unnecessary ensure statement
|
|
Subtraction is a difficult concept for some folks (like
myself) to grasp and implement.
|
|
It's more fool-proof this way and prevents us from using
idiotic/non-obvious concurrency model names.
|
|
I need better tests for graceful shutdown...
|
|
Revactor does not use threads and blocking on a stock Queue
class does not work. Eventually this should be made to work
with the Actor model, but until then, we'll at least document
it...
|
|
This release adds preliminary Rev support for network
concurrency under Ruby 1.8 and Ruby 1.9. There are caveats to
this model and reading the RDoc for Rainbows::Rev is
recommended.
Rainbows::AppPool Rack middleware is now available to limit
application concurrency on a per-process basis independently of
network concurrency. See the RDoc for this class for further
details.
Per-client timeouts have been removed, see
http://mid.gmane.org/20091013062602.GA13128@dcvr.yhbt.net
for the reasoning.
Rack environment changes:
* "rack.multithread" is now only true for models with "Thread"
in their name. Enabling thread-safe (but not reentrant) code
may actually be harmful for Revactor.
* "rainbows.model" is now exposed so the application can easily
figure out which network concurrency model is in use.
Bugfixes include better shutdown and error handling for all
existing models, OpenBSD compatibility for the per-process
heartbeat (same as found in unicorn v0.93.3).
Eric Wong (53):
add SIGNALS doc to RDoc
SIGNALS: add Rainbows!-specific notes
doc: better "Rainbows!" RDoc examples and linkage
tests: generate random_blob once for all tests
tests: move trash files to their own trash/ directory
t0000: basic test includes keepalive + pipelining
tests: simplify temporary file management
tests: add dbgcat() utility method
fchmod heartbeat flips between 0/1
tests: add revactor pipelining/keepalive test
thread_spawn: trap EAGAIN on accept_nonblock
thread_spawn: more robust loop
thread_spawn: non-blocking accept() shouldn't EINTR
tests: enable pipefail shell option if possible
README for test suite
tests: TEST_OPTS => SH_TEST_OPTS
tests: update TRACER examples in makefile
tests: create a bad exit code by default
thread_spawn: clean up nuking of timed-out threads
factor out common listen loop error handling
graceful exit on trap TypeError from IO.select
expand and share init_worker_process
revactor: break on EBADF in the accepting actors
revactor: cleanups and remove redundancy
No need to be halving timeout, already done for us
revactor: graceful death of keepalive clients
revactor: continue fchmod beat in graceful exit
cleanup thread models, threads no longer time out
revactor: fix graceful shutdown timeouts
Fix graceful shutdowns for threaded models
SIGINT/SIGTERM shuts down instantly in workers
tests: check for common exceptions with "Error"
DEPLOY: update with notes on DoS potential
tests: add reopen logs test for revactor
vs Unicorn: use diagrams for concurrency models
vs Unicorn: fix wording to be consistent with diagrams
vs Unicorn: fix copy+paste errors and grammar fail
README: alter reply conventions for the mailing list
preliminary Rev support
local.mk.sample: use ksh93 as default $(SHELL)
rack.multithread is only true for Thread* models
Rev: general module documentation + caveats
Rev: fix error handling for parser errors
t3003: set executable bit
documentation updates (mostly on network models)
rack: expose "rainbows.model" in Rack environment
tests: enforce rack.multithread and rainbows.model
README: update URLs
README: update with Rev model caveats
Add Rainbows::AppPool Rack middleware
t4003: chmod +x
local.mk.sample: use rev 0.3.1 instead
README: link to AppPool and extra note about Rev model
|
|
|
|
This allows applications to determine which concurrency model
they're running under and possibly make adjustments accordingly.
The standard "rack.multithread" isn't enough for some
applications to determine what to do, especially when reentrancy
is required/recommended.
|
|
|
|
We should try to send 400s back to the client if possible.
|
|
|