about summary refs log tree commit homepage
DateCommit message (Collapse)
2013-10-20yahns 0.0.1 - many small fixes and test coverage v0.0.1
Just a bunch of improvements found while running tests. It's still incomplete and missing a bunch of features from existing servers, but maybe it works... Clone git://yhbt.net/yahns.git for full details Eric Wong (18): test/helper: fix __covmerge race for forked processes wire up client_max_body_size limits fix and test Rack hijack support SIGUSR2 handling uses Process.spawn + tests fix USR1 log reopening when using worker_processes test_bin: add additional tests for SIGHUP test_server: skip test_mp_balance for now test/server_helper: dump entire log on errors ensure we stop all threads at exit GNUmakefile: avoid calling exit in test-mt test_bin: set close-on-exec for Ruby 1.9.3 compatibility test_server: bigger delays for graceful shutdown test tests: disable $-w on 1.9.3 to quiet down warnings test/helper: prevent minitest at_exit from running in children recheck IO#closed? on thread pools after a short delay test_config: isolate directories with logs test_reopen_logs: workaround timing problem with worker_processes set close-on-exec on all long-lived descriptors
2013-10-20set close-on-exec on all long-lived descriptors
This means ruby 1.9.3 should be supported, as well as Ruby implementations which do not set the close-on-exec flag by default. Note: this is only best-effort outside of modern Linux with threads, since a multithreaded process may create and inadvertantly share descriptors. This is why Linux supports O_CLOEXEC, SOCK_CLOEXEC and friends, as kernel support is the only way to sanely fix this.
2013-10-20test_reopen_logs: workaround timing problem with worker_processes
worker_processes only open the new log file after the master does, so the workers processing the request may not be writing to the new one, yet, so we repeat the HTTP request.
2013-10-20test_config: isolate directories with logs
Otherwise, this is racy when we have the reopen logs tests running in the same process. We can't workaround this in the process reopening logs (by ignoring ENOENT) since recreating the file can also race with the FileUtils.rm_rf.
2013-10-20recheck IO#closed? on thread pools after a short delay
The closed IO may not immediately register to all threads due to ordering problems
2013-10-20test/helper: prevent minitest at_exit from running in children
Since we fork in tests, at_exit handlers may run in children and cause strange test failures. This seems necessary in minitest 5.0.8, but not 4.3.2. We also cannot use exit! in forked children because apps hosted on yahns will probably expect at_exit handlers they register to run.
2013-10-20tests: disable $-w on 1.9.3 to quiet down warnings
openssl warns about an unused variable
2013-10-20test_server: bigger delays for graceful shutdown test
This should hopefully increase test reliability as timing of signal delivery is not guaranteed...
2013-10-20test_bin: set close-on-exec for Ruby 1.9.3 compatibility
2013-10-20GNUmakefile: avoid calling exit in test-mt
This can cause error exit codes to be lost depending on which test suite (and version) we're using.
2013-10-20ensure we stop all threads at exit
Leaving running threads at exit seems to lead to occasional bugs at finalization on Ruby 2.0.0. This could be a bug with sleepy_penguin or kgio, too, so I'll have to investigate further. For now, we'll just destroy the IOs associated with each queue and let the threads die on their own. This changes the QueueEgg internals a bit and I've removed the unit test for QueueEgg now since the rest of the server already works well (and QueueEgg internals may change even more). Queues/worker threads no longer have their own logger, it seems like excessive configurability/complexity since acceptors do not have their own logger, either. This logger only exists to log bugs in yahns, not the application, so using the server logger is sufficient.
2013-10-20test/server_helper: dump entire log on errors
Merely showing the error lines could discard useful information when tracking down bugs or problems with the tests.
2013-10-20test_server: skip test_mp_balance for now
This should probably made into statistical test to allow for scheduler unfairness causing imperfect balance between new connections. For now, it's easier to skip it since it fails a small percentage of the time. In real world use, a small amount of imbalance does not matter on a busy system as long as connections are reasonably well distributed.
2013-10-19test_bin: add additional tests for SIGHUP
We can just add it into the SIGUSR2 test for now.
2013-10-19fix USR1 log reopening when using worker_processes
Just forgot to trap USR1 in workers. While we're at it, add tests.
2013-10-19SIGUSR2 handling uses Process.spawn + tests
We no longer have to worry about 1.8 compatibility, so use Process.spawn and shorten our code. Also, add tests for this functionality.
2013-10-19fix and test Rack hijack support
Rack hijacking may close the socket before it yields control back to our epoll event loop. So it's not safe to attempt to use the socket (or even get the .fileno) with :delete/EPOLL_CTL_DEL. So change :delete to :ignore, and only decrement the FD counter to allow yahns to do graceful shutdown. This means we'll potentially waste ~200 bytes of kernel memory due to epoll overhead until the FD is closed by the app hijacking. I'm not sure how Rack servers handle graceful shutdown when hijacking, but maybe they just retrap SIGQUIT...
2013-10-19wire up client_max_body_size limits
This is mostly code imported from Rainbows! (so GPLv2+). This should implement everything necessary to prevent clients from DoS-ing us with overly large bodies. The default is 1M (same as Rainbows! and nginx). Yahns::MaxBody may become part of the public API (as the equivalent is in Rainbows!), since it makes more sense in the rackup (config.ru) file (since it's endpoint-specific). However, that's confusing as Yahns::MaxBody only works when input_buffering is :lazy or false, and not when it is true (preread).
2013-10-19test/helper: fix __covmerge race for forked processes
flock works on file handles (which are shared across processes). So our use of flock on a global constant was not fork safe as the same file handle would share locks between the client/parent process. Since we have to create/truncate the coverage.dump file in the Makefile before any tests run anyways, we can just lazily open the file handle when we need to use it, and never before.
2013-10-18yahns 0.0.0 - hopefully fix installation problems v0.0.0
Installing prrerelease versions are strange on RubyGems. Anyways, new project, no risk of breakage, so maybe this works better...
2013-10-18README: fix URL for git repository
Shorter URLs should be easier-to-type, since we already have the project name in the path component of the URL.
2013-10-18tmpio: remove unused discard alias
Follow-up to commit 4b7ffee7c2436b1839146fd124e731a63f20d03e (remove tiny_input)
2013-10-18README: update with mailing list information
Hmm... greylisting is annoying on Rubyforge.
2013-10-18gemspec: fix authors
Apparently "#{name}" does not expand properly in gemspecs :x
2013-10-18yahns 0.0.0TP1
It runs, maybe!
2013-10-18disable persistent connections on SIGQUIT
We want to shutdown gracefully, but still relatively quickly (to make way for the new one). So we must disable persistent connections to prevent clients from keeping to-be-dead server alive indefinitely.
2013-10-18http_response: respect setups with non-persistent connections
Some users may wish to disable persistent connections for testing or whatever reason, let them. We'll also be using this feature to force SIGQUIT to expire clients.
2013-10-18remove unused unicorn constants to hopefully trim memory
We'll probably drop the unicorn dependency in the future and use kcar once it gains the ability to parse server requests (and not just responses). kcar is descended from the unicorn parser anyways and supports the same enhancements (chunked encoding/trailers).
2013-10-18remove tiny_input
Since we are smarter about closing @input, we can just call .close on it as closing inputs is forbidden in Rack apps, anyways. This reduces our class count and should make things easier to follow.
2013-10-18http_client: do not set @input if it is NULL_IO
That is the common case, so we should not be calling the discard method on NULL_IO. This makes NULL_IO a StringIO, again.
2013-10-18test: cleanup socket usage and lifetimes
This should help prevent us from running out of FDs prematurely as our test suite becomes more multi-threaded. We can also use plain TCPSocket in tests where we inherit the file descriptor from the parent (w/o accept).
2013-10-18test_client_expire: WIP test for EMFILE expiry
Unfortunately, this will need further work because tests are MT and FD counts vary, so reproducing reliable FD counts will need to rely on exec + FD_CLOEXEC.
2013-10-18test and fix client expiry
This is rarely-called code, but we need to be sure we can expire clients correctly when thresholds are reached. We also correctly handle negative values of the client_expire_threshold directive.
2013-10-18http_client: bump up keepalive_requests
Rainbows!/unicorn (and nginx) were designed when migrating clients between threads/processes was not feasible. yahns expires based on FD pressure, so it should not need an arbitrary limit (but the Rack app may specify "Connection: close" to kill a connection.
2013-10-18initial commit