Date | Commit message (Collapse) |
|
Just a bunch of improvements found while running tests.
It's still incomplete and missing a bunch of features from
existing servers, but maybe it works...
Clone git://yhbt.net/yahns.git for full details
Eric Wong (18):
test/helper: fix __covmerge race for forked processes
wire up client_max_body_size limits
fix and test Rack hijack support
SIGUSR2 handling uses Process.spawn + tests
fix USR1 log reopening when using worker_processes
test_bin: add additional tests for SIGHUP
test_server: skip test_mp_balance for now
test/server_helper: dump entire log on errors
ensure we stop all threads at exit
GNUmakefile: avoid calling exit in test-mt
test_bin: set close-on-exec for Ruby 1.9.3 compatibility
test_server: bigger delays for graceful shutdown test
tests: disable $-w on 1.9.3 to quiet down warnings
test/helper: prevent minitest at_exit from running in children
recheck IO#closed? on thread pools after a short delay
test_config: isolate directories with logs
test_reopen_logs: workaround timing problem with worker_processes
set close-on-exec on all long-lived descriptors
|
|
This means ruby 1.9.3 should be supported, as well as Ruby
implementations which do not set the close-on-exec flag by default.
Note: this is only best-effort outside of modern Linux with threads,
since a multithreaded process may create and inadvertantly share
descriptors. This is why Linux supports O_CLOEXEC, SOCK_CLOEXEC and
friends, as kernel support is the only way to sanely fix this.
|
|
worker_processes only open the new log file after the master does,
so the workers processing the request may not be writing to the
new one, yet, so we repeat the HTTP request.
|
|
Otherwise, this is racy when we have the reopen logs tests running
in the same process. We can't workaround this in the process
reopening logs (by ignoring ENOENT) since recreating the file can
also race with the FileUtils.rm_rf.
|
|
The closed IO may not immediately register to all threads due
to ordering problems
|
|
Since we fork in tests, at_exit handlers may run in children
and cause strange test failures. This seems necessary in
minitest 5.0.8, but not 4.3.2. We also cannot use exit! in
forked children because apps hosted on yahns will probably expect
at_exit handlers they register to run.
|
|
openssl warns about an unused variable
|
|
This should hopefully increase test reliability as timing of
signal delivery is not guaranteed...
|
|
|
|
This can cause error exit codes to be lost depending on which test
suite (and version) we're using.
|
|
Leaving running threads at exit seems to lead to occasional bugs at
finalization on Ruby 2.0.0. This could be a bug with sleepy_penguin
or kgio, too, so I'll have to investigate further. For now, we'll
just destroy the IOs associated with each queue and let the threads
die on their own.
This changes the QueueEgg internals a bit and I've removed the unit
test for QueueEgg now since the rest of the server already works
well (and QueueEgg internals may change even more).
Queues/worker threads no longer have their own logger, it seems like
excessive configurability/complexity since acceptors do not have
their own logger, either. This logger only exists to log bugs in
yahns, not the application, so using the server logger is sufficient.
|
|
Merely showing the error lines could discard useful information
when tracking down bugs or problems with the tests.
|
|
This should probably made into statistical test to allow for scheduler
unfairness causing imperfect balance between new connections. For now,
it's easier to skip it since it fails a small percentage of the time.
In real world use, a small amount of imbalance does not matter on
a busy system as long as connections are reasonably well distributed.
|
|
We can just add it into the SIGUSR2 test for now.
|
|
Just forgot to trap USR1 in workers. While we're at it, add tests.
|
|
We no longer have to worry about 1.8 compatibility, so use
Process.spawn and shorten our code. Also, add tests for this
functionality.
|
|
Rack hijacking may close the socket before it yields control back to our
epoll event loop. So it's not safe to attempt to use the socket (or
even get the .fileno) with :delete/EPOLL_CTL_DEL.
So change :delete to :ignore, and only decrement the FD counter to allow
yahns to do graceful shutdown. This means we'll potentially waste ~200
bytes of kernel memory due to epoll overhead until the FD is closed by
the app hijacking.
I'm not sure how Rack servers handle graceful shutdown when
hijacking, but maybe they just retrap SIGQUIT...
|
|
This is mostly code imported from Rainbows! (so GPLv2+). This should
implement everything necessary to prevent clients from DoS-ing us with
overly large bodies. The default is 1M (same as Rainbows! and nginx).
Yahns::MaxBody may become part of the public API (as the equivalent is
in Rainbows!), since it makes more sense in the rackup (config.ru) file
(since it's endpoint-specific). However, that's confusing as
Yahns::MaxBody only works when input_buffering is :lazy or false, and
not when it is true (preread).
|
|
flock works on file handles (which are shared across processes).
So our use of flock on a global constant was not fork safe as the
same file handle would share locks between the client/parent
process. Since we have to create/truncate the coverage.dump file in
the Makefile before any tests run anyways, we can just lazily open
the file handle when we need to use it, and never before.
|
|
Installing prrerelease versions are strange on RubyGems.
Anyways, new project, no risk of breakage, so maybe this
works better...
|
|
Shorter URLs should be easier-to-type, since we already have
the project name in the path component of the URL.
|
|
Follow-up to commit 4b7ffee7c2436b1839146fd124e731a63f20d03e
(remove tiny_input)
|
|
Hmm... greylisting is annoying on Rubyforge.
|
|
Apparently "#{name}" does not expand properly in gemspecs :x
|
|
It runs, maybe!
|
|
We want to shutdown gracefully, but still relatively quickly (to
make way for the new one). So we must disable persistent
connections to prevent clients from keeping to-be-dead server alive
indefinitely.
|
|
Some users may wish to disable persistent connections for testing
or whatever reason, let them. We'll also be using this feature
to force SIGQUIT to expire clients.
|
|
We'll probably drop the unicorn dependency in the future and use
kcar once it gains the ability to parse server requests (and not
just responses). kcar is descended from the unicorn parser
anyways and supports the same enhancements (chunked
encoding/trailers).
|
|
Since we are smarter about closing @input, we can just call
.close on it as closing inputs is forbidden in Rack apps,
anyways. This reduces our class count and should make
things easier to follow.
|
|
That is the common case, so we should not be calling
the discard method on NULL_IO. This makes NULL_IO a StringIO,
again.
|
|
This should help prevent us from running out of FDs prematurely
as our test suite becomes more multi-threaded. We can also use
plain TCPSocket in tests where we inherit the file descriptor
from the parent (w/o accept).
|
|
Unfortunately, this will need further work because tests
are MT and FD counts vary, so reproducing reliable FD
counts will need to rely on exec + FD_CLOEXEC.
|
|
This is rarely-called code, but we need to be sure we can expire
clients correctly when thresholds are reached. We also correctly
handle negative values of the client_expire_threshold directive.
|
|
Rainbows!/unicorn (and nginx) were designed when migrating clients
between threads/processes was not feasible. yahns expires based on
FD pressure, so it should not need an arbitrary limit (but the Rack
app may specify "Connection: close" to kill a connection.
|
|
|