| Date | Commit message (Collapse) |
|---|
|
It works exactly like Actor.sleep and similar to Kernel.sleep
(no way to sleep indefinitely), but is compatible with the
IO.select-based Fiber scheduler we run. This method only works
within the context of a Rainbows! application dispatch.
|
|
|
|
This is another Fiber-based concurrency model that can exploit
a streaming "rack.input" for clients. Spawning Fibers seems
pretty fast, but maybe there are apps that will benefit from
this.
|
|
This one seems a easy to get working and supports everything we
need to support from the server perspective. Apps will need
modified drivers, but it doesn't seem too hard to add
more/better support for wrapping IO objects with Fiber::IO.
|
|
Due to the addition of keepalive_timeouts, it's safer to
pay a performance penalty and use a hash here instead.
|
|
|
|
Exposing a synchronous interface is too complicated for too
little gain. Given the following factors:
* basic ThreadSpawn performs admirably under REE 1.8
* both ThreadSpawn and Revactor work well under 1.9
* few applications/requests actually need a streaming "rack.input"
We've decided its not worth the effort to attempt to support
streaming rack.input at the moment. Instead, the new
RevThreadSpawn model performs much better for most applications
under Ruby 1.9
|
|
No point in rewinding the NULL_IO especially when most requests
use them instead of bodies that actually have something.
|
|
And change the default to 2 seconds, most clients can
render the page and load all URLs within 2 seconds.
|
|
Fortunately it's easy here.
|
|
This is a bit trickier than the rest since we have to ensure
deferred (proxied) responses aren't nuked.
|
|
If the Revactor implementation using lightweight Actors/Fibers
needs it, then thread implementations do, too.
|
|
We'll be getting a keepalive_timeout setting soon,
clients with 300 second idle keepalives are ridiculous
and Ruby objects are still not that cheap in 1.9
|
|
Client shutdowns/errors when streaming "rack.input" into the
Rack application are quieter now. Rev and EventMachine workers
now shutdown correctly when the master dies. Worker processes
now fail gracefully if log reopening fails. ThreadSpawn and
ThreadPool models now load Unicorn classes in a thread-safe way.
There's also an experimental RevThreadSpawn concurrency
model which may be heavily reworked in the future...
Eric Wong (30):
Threaded models have trouble with late loading under 1.9
cleanup worker heartbeat and master deathwatch
tests: allow use of alternative sha1 implementations
rev/event_machine: simplify keepalive checking a bit
tests: sha1.ru now handles empty bodies
rev: split out further into separate files for reuse
rev: DeferredResponse is independent of parser state
remove unnecessary class variable
ev_core: cleanup handling of APP constant
rev: DeferredResponse: always attach to main loop
initial cut of the RevThreadSpawn model
rev_thread_spawn/revactor: fix TeeInput for short reads
rev_thread_spawn: make 1.9 TeeInput performance tolerable
tests: add executable permissions to t0102
tests: extra check to avoid race in reopen logs test
rev_thread_spawn: 16K chunked reads work better
tests: ensure proper accounting of worker_connections
tests: heartbeat-timeout: simplify and avoid possible race
tests: ensure we process "START" from FIFO when starting
http_response: don't "rescue nil" for body.close
cleanup error handling pieces
tests: more stringent tests for error handling
revactor/tee_input: unnecessary error handling
gracefully exit workers if reopening logs fails
revactor/tee_input: raise ClientDisconnect on EOFError
bump versions since we depend on Unicorn::ClientShutdown
revactor/tee_input: share error handling with superclass
RevThreadSpawn is still experimental
Revert "Threaded models have trouble with late loading under 1.9"
Rakefile: add raa_update task
|
|
This reverts commit e1dcadef6ca242e36e99aab19e3e040bf01070f9.
This is fixed separately in Unicorn 0.95.0 (commit
560216c2fecfc5cf3489f749dc7a0221fd78eb26)
|
|
|
|
Less stuff to maintain is good.
|
|
|
|
Based on unicorn.git commit e4256da292f9626d7dfca60e08f65651a0a9139a
raise Unicorn::ClientShutdown if client aborts in TeeInput
Leaving the EOFError exception as-is bad because most
applications/frameworks run an application-wide exception
handler to pretty-print and/or log the exception with a huge
backtrace.
Since there's absolutely nothing we can do in the server-side
app to deal with clients prematurely shutting down, having a
backtrace does not make sense. Having a backtrace can even be
harmful since it creates unnecessary noise for application
engineers monitoring or tracking down real bugs.
|
|
Permissions for the logs could've been badly set by the master.
So we we'll let the master reopen them and refork children to
get around this problem. We have to be more careful when
reopening logs because we can reopen them in the middle of
client requests (we have to) whereas Unicorn has the luxury
of _knowing_ it has no active clients when it does the reopen.
|
|
We're doomed if the client socket EOFs on us while we're reading
it. So don't hide it and let the exception bubble all the way
up the stack.
|
|
Unicorn 0.94.0 got a more generic handle_error function
that's useful in the Thread* models. The Revactor one
is a little different but similar to be worth refactoring
to match our standard pieces.
|
|
This can hide bugs in Rack applications/middleware. Most other
Rack handlers/servers seem to follow this route as well, so
this helps ensure broken things will break loudly and more
consistently across all Rack-enabled servers.
|
|
When reading 4K chunks, performance is dismal under 1.8
|
|
Somehow 1.8 performance blows with shorter reads in the Rack
application. This may be because the Rev framework uses
a default 16K IO size and our test applications may request
less.
|
|
Explicitly requested short reads may cause too much data to be
returned, which would be bad and potentially break the
application. We need to ensure proper IO#readpartial-like
semantics in both of these models.
|
|
Seems to pass all tests, but that may only mean our
test cases are lacking...
|
|
It's too complicated to deal with multiple Rev loops
so only use the main one for now under 1.9.
|
|
It'll make development of future ev_core-derived things
easier, hopefully.
|
|
It's already global...
|
|
In the upcoming RevThread* models, the parser may be parsing
other requests already by the time DeferredResponse is called.
|
|
This will make things easier to manage with more
Rev-based concurrency models.
|
|
Since the HTTP parser is frozen during app dispatch, there's
no point in checking for HTTP keepalive sooner. Of course we
check G.alive as late as possible since we could've received a
:QUIT signal while app.call was running.
|
|
It turns out neither the EventMachine and Rev classes
checked for master death in its heartbeat mechanism.
Since we managed to forget the same thing twice, we
now have a test case for it and also centralized the
code to remove duplication.
|
|
Loading TeeInput or HttpResponse late does not always work well
in multithreaded situations and have been causing random test
failures on heavily loaded multicore boxes.
|
|
We depend on the just-released Unicorn 0.94.0 for the fixed
trailer handling. As with `unicorn', the `rainbows' executable
now sets and respects ENV["RACK_ENV"]. Also small fixes and
cleanups including better FreeBSD 7.2 compatibility and
less likely to over-aggressively kill slow/idle workers
when a very low timeout is set.
Eric Wong (20):
rev: split out heartbeat class
bump Unicorn dependency to (consistently) pass tests
tests: avoid single backquote in echo
event_machine: avoid slurping when proxying
tests: make timeout tests reliable under 1.9
thread_pool: comment for potential SMP issue under 1.9
Allow 'use "model"' as a string as well as symbol
Rev model is the only user of deferred_bodies
ev_core: use Tempfile instead of Unicorn::Util::tmpio
ev_core: ensure quit is triggered on all errors
rainbows: set and use process-wide ENV["RACK_ENV"]
http_server: add one second to any requested timeout
thread_pool: update fchmod heartbeat every second
t0004: tighten up timeout test
ev_core: remove Tempfile usage once again
cleanup: remove unused t????.ru test files
tests: staggered trailer upload test
ensure RACK_ENV is inherited from the parent env
t0100: more precise `expr` usage
|
|
We're simply too uncomfortable with the weird GC issues
associated with Tempfile and having linked temporary files at
all. Instead just depend on the #size-aware TmpIO class that
Unicorn 0.94.0 provides for us.
|
|
Like the rest of the concurrency models. This gives us
more flexibility in case a process-wide blocking operation
started during an "unlucky" period when the join timeout
was about to expire.
|
|
This is because our timeout implementations are less precise
than Unicorn. Since we handle multiple clients with the same
process, we sacrifice precision for performance and instead
implement our fchmod heartbeats at a fixed rate, as doing
fchmod() repeated for short-lived connections would hurt
performance and we have to call fchmod even when connected
clients are idle.
|
|
Just in case something goes wrong with the write
or the logger, make sure we've triggered a quit.
|
|
Since we're geared towards slower clients, we may be able to
make gains from using userspace IO buffering. This allows us to
avoid metadef-ing a #size method for every File we allocate
and save memory.
|
|
We don't use it in EventMachine since EM has its own
built-in ways to handle deferred bodies.
|
|
Since const_get works with a string as well as a symbol,
allow that to be used. It's easier and simpler to just
allow strings as use arguments than to error check and
raise exceptions.
So both of the following should now work:
Rainbows! do
use :Revactor
end
Rainbows! do
use "Revactor"
end
Rainbows! will always use the symbol variant internally,
however, so applications can alway expect env['rainbows.model']
to be a symbol.
|
|
The problem is unconfirmed at the moment, but I've long
anticipated it. I just need to remember the next time I
log into a monster machine.
|
|
Avoid slurping in case we're a fast backend writing to a slow
client. This should prevent our memory usage from exploding
when clients are reading slowly.
|
|
This module will be reused in upcoming Rev-derived concurrency
models.
|
|
Basic single-threaded EventMachine support is now included. It
supports async_synatra[1] via the "async.callback" Rack
environment[2]. For EventMachine, we rely on the updated
attach/watch API in EventMachine 0.12.10.
As Revactor 0.1.5 is now available, our Revactor support now
depends on it as it adds the ability to listen on UNIX domain
sockets.
For developers/QA folks, the integration tests are completely
revamped for easier maintenance when new concurrency models are
introduced and should also produce TAP-compliant output. The
test suite remains highly parallelizable using GNU make.
There are immediate plans to expand support for both Rev and
EventMachine to support use with threaded application dispatch.
Eric Wong (41):
rev: remove Revactor-specific workaround
README: change ordering of concurrency model listing
tests: more correct HTTP/0.9 test
test-lib: avoid stalling due to bad FIFO handling
rev: fix static file responses under HTTP/0.9
add news bodies to site NEWS.atom.xml
tests: avoid needlessly remaking "rainbows"
initial EventMachine support
tests: hopefully fix stalls in input trailer tests
tests: avoid race condition in reopen logs test
tests: prefer "RUBY" to lowercased "ruby"
tests: common setup and wait_start functions
tests: add a TAP producer shell library
tests: port all existing tests to TAP library
tests: remove symlinks and small files, use Make
t9000: bail if run with an unsupported/pointless model
tests: allow "make $model" to run tests for that model
rev: spell ECONNABORTED correctly
rev/evma: move common code for event models into ev_core
ev_core: do not drop deferred bodies on graceful quits
eventmachine: get basic tests working
rev: do not File.expand_path on result of body.to_path
eventmachine 0.12.8 passes all tests
tests: make large file memory tests more reliable
eventmachine: require EM 0.12.10
update gem dependencies in comments/local.mk.sample
rev: enforce Rev::VERSION >= 0.3.0
eventmachine: add async_sinatra support
tests: only load Revactor tests under 1.9.1
tests: gracefully exit if EventMachine is not available
tests: error out if socat + curl aren't reachable
thread*: fix MRI 1.8.6 compatibility
local.mk.sample: cleanups and minor reorg
eventmachine: remove unnecessary ivar assignment
eventmachine: document our support of "async_synatra"
doc: Update TODO and README
tests: generate all dependencies atomically
app_pool: update RDoc
test-lib: DWIM handling of temp UNIX sockets
revactor: require 0.1.5, remove 0.1.4 workarounds
gemspec: bump up Unicorn dep version to 0.93.4
[1] http://github.com/raggi/async_sinatra
[2] this is not 100% Rack::Lint compatible, but we'll let it
slide since there are already folks depending on
the async_sinatra gem
|
|
Also new are added basic HTTP tests for UNIX domain socket
handling (for all models, now, of course).
|
|
Now that we have EM support (which is basically like
Rev), update our docs for that. Also, expand on why
Revactor isn't supported just yet...
|
|
ref: http://github.com/raggi/async_sinatra
|