about summary refs log tree commit homepage
DateCommit message (Collapse)
2012-11-29unicorn 4.5.0pre1 - check_client_connection option v4.5.0pre1
The new check_client_connection option allows unicorn to detect most disconnected clients before potentially expensive application processing begins. This feature is useful for applications experiencing spikes of traffic leading to undesirable queue times, as clients will disconnect (and perhaps even retry, compounding the problem) before unicorn can even start processing the request. To enable this feature, add the following line to a unicorn config file: check_client_connection true A huge thanks to Tom Burns for implementing and testing this change in production with real traffic (including mitigating an unexpected DoS attack).
2012-11-29check_client_connection: document local-only requirement
In my testing, only dropped clients over Unix domain sockets or loopback TCP were detected with this option. Since many nginx+unicorn combinations run on the same host, this is not a problem. Furthermore, tcp_nodelay:true appears to work over loopback, so remove the requirement for tcp_nodelay:false.
2012-11-29Begin writing HTTP request headers early to detect disconnected clients
This patch checks incoming connections and avoids calling the application if the connection has been closed. It works by sending the beginning of the HTTP response before calling the application to see if the socket can successfully be written to. By enabling this feature users can avoid wasting application rendering time only to find the connection is closed when attempting to write, and throwing out the result. When a client disconnects while being queued or processed, Nginx will log HTTP response 499 but the application will log a 200. Enabling this feature will minimize the time window during which the problem can arise. The feature is disabled by default and can be enabled by adding 'check_client_connection true' to the unicorn config. [ew: After testing this change, Tom Burns wrote: So we just finished the US Black Friday / Cyber Monday weekend running unicorn forked with the last version of the patch I had sent you. It worked splendidly and helped us handle huge flash sales without increased response time over the weekend. Whereas in previous flash traffic scenarios we would see the number of HTTP 499 responses grow past the number of real HTTP 200 responses, over the weekend we saw no growth in 499s during flash sales. Unexpectedly the patch also helped us ward off a DoS attack where the attackers were disconnecting immediately after making a request. ref: <CAK4qKG3rkfVYLyeqEqQyuNEh_nZ8yw0X_cwTxJfJ+TOU+y8F+w@mail.gmail.com> ] Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-11-13tests: remove assert_nothing_raised (part 2)
assert_nothing_raised ends up hiding errors and backtraces, making things harder to debug. Since Test::Unit already fails on uncaught exceptions, there is no need to assert on the lack of exceptions for a successful test run. This is a followup to commit 5acf5522295c947d3118926d1a1077007f615de9
2012-10-11Rakefile: fm_update task updated for HTTPS
Freecode.com now requires HTTPS.
2012-10-11unicorn 4.4.0 - minor updates v4.4.0
Non-regular files are no longer reopened on SIGUSR1. This allows users to specify FIFOs as log destinations. TCP_NOPUSH/TCP_CORK is no longer set/unset by default. Use :tcp_nopush explicitly with the "listen" directive if you wish to enable TCP_NOPUSH/TCP_CORK. Listen sockets are now bound _after_ loading the application for preload_app(true) users. This prevents load balancers from sending traffic to an application server while the application is still loading. There are also minor test suite cleanups.
2012-10-01util: only consider regular files as logs
If a user specifies a non-regular file for stderr_path or stdout_path, we should not attempt to reopen or chown it. This should also allow users to specify FIFOs as log destinations.
2012-08-06avoid assert_nothing_raised in unit tests
It's better to show errors and backtraces when stuff breaks
2012-08-06do not touch TCP_NOPUSH/TCP_CORK at all by default
On a certain FreeBSD 8.1 installation, explicitly setting TCP_NOPUSH to zero (off) can cause EADDRNOTAVAIL errors and also resets the listen backlog to 5. Enabling TCP_NOPUSH explicitly did not exhibit this issue for the user who (privately) reported this issue. To be on the safe side, we won't set/unset TCP_NOPUSH/TCP_CORK at all, which will leave it off on all current systems.
2012-08-02bind listeners after loading for preload_app users
In the case where preload_app is true, delay binding new listeners until after loading the application. Some applications have very long load times (especially Rails apps with Ruby 1.9.2). Binding listeners early may cause a load balancer to incorrectly believe the unicorn workers are ready to serve traffic even while the app is being loaded. Once a listener is bound, connect() requests from the load balancer succeed until the listen backlog is filled. This allows requests to pile up for a bit (depending on backlog size) before getting rejected by the kernel. By the time the application is loaded and ready-to-run, requests in the listen backlog are likely stale and not useful to process. Processes inheriting listeners do not suffer this effect, as the old process should still be capable of serving new requests. This change does not improve the situation for the preload_app=false (default) use case. There may not be a solution for preload_app=false users using large applications. Fortunately Ruby 1.9.3+ improves load times of large applications significantly over 1.9.2 so this should be less of a problem in the future. Reported via private email sent on 2012-06-29T22:59:10Z
2012-07-28remove Rails-oriented integration tests
It's too much overhead to keep Rails-specific tests working, especially when it's hauling in an ancient version of SQLite3. Since Rails 3 has settled down with Rack and unicorn_rails is unlikely to need changing in the future, we can drop these tests.
2012-04-29unicorn 4.3.1 - shutdown() fixes v4.3.1
* Call shutdown(2) if a client EOFs on us during upload. We can avoid holding a socket open if the Rack app forked a process during uploads. * ignore potential Errno::ENOTCONN errors (from shutdown(2)). Even on LANs, connections can occasionally be accept()-ed but be unusable afterwards. Thanks to Joel Nimety <jnimety@continuity.net>, Matt Smith <matt@nearapogee.com> and George <lists@southernohio.net> on the mongrel-unicorn@rubyforge.org mailing list for their feedback and testing for this release.
2012-04-29isolate_for_tests: upgrade to kgio-monkey 0.4.0
Seems to work well enough...
2012-04-27stream_input: call shutdown(2) if a client EOFs on us
In case the Rack app forks before a client upload is complete, shutdown(2) the socket to ensure the client isn't attempting to read from us (even if it explicitly stopped writes).
2012-04-27http_server: ignore ENOTCONN (mostly from shutdown(2))
Since there's nothing unicorn can do to avoid this error on unconnected/halfway-connected clients, ignoring ENOTCONN is a safe bet. Rainbows! has long had this rescue as it called getpeername(2) on untrusted sockets
2012-04-17unicorn 4.3.0 - minor fixes and updates v4.3.0
* PATH_INFO (aka REQUEST_PATH) increased to 4096 (from 1024). This allows requests with longer path components and matches the system PATH_MAX value common to GNU/Linux systems for serving filesystem components with long names. * Apps that fork() (but do not exec()) internally for background tasks now indicate the end-of-request immediately after writing the Rack response. Thanks to Hongli Lai, Lawrence Pit, Patrick Wenger and Nuo Yan for their valuable feedback for this release.
2012-04-17tests: set executable bit on integration shell scripts
These should be made executable for ease-of-understanding and consistency, regardless of whether we actually execute them.
2012-04-17http: increase REQUEST_PATH maximum length to 4K
The previous REQUEST_PATH limit of 1024 is relatively small and some users encounter problems with long URLs. 4K is a common limit for PATH_MAX on modern GNU/Linux systems and REQUEST_PATH is likely to translate to a filesystem path name. Thanks to Nuo Yan <yan.nuo@gmail.com> and Lawrence Pit <lawrence.pit@gmail.com> for their feedback on this issue. ref: http://mid.gmane.org/CB935F19-72B8-4EC2-8A1D-5084B37C09F2@gmail.com
2012-04-12shutdown client socket for apps which fork in background
Previously we relied on implicit socket shutdown() from the close() syscall. However, some Rack applications fork() (without calling exec()), creating a potentially long-lived reference to the underlying socket in a child process. This ends up causing nginx to wait on the socket shutdown when the child process exits. Calling shutdown() explicitly signals nginx (or whatever client) that the unicorn worker is done with the socket, regardless of the number of FD references to the underlying socket in existence. This was not an issue for applications which exec() since FD_CLOEXEC is always set on the client socket. Thanks to Patrick Wenger for discovering this. Thanks to Hongli Lai for the tip on using shutdown() as is done in Passenger. ref: http://mid.gmane.org/CAOG6bOTseAPbjU5LYchODqjdF3-Ez4+M8jo-D_D2Wq0jkdc4Rw@mail.gmail.com
2012-04-12t/sslgen.sh: use larger keys for tests
This seems required for TLSv1.2 under OpenSSL 1.0.1
2012-04-11misc documentation spelling fixes
Found via rdoc-spellcheck
2012-03-26unicorn 4.2.1 - minor fix and doc updates v4.2.1
* Stale pid files are detected if a pid is recycled by processes belonging to another user, thanks to Graham Bleach. * nginx example config updates thanks to to Eike Herzbach. * KNOWN_ISSUES now documents issues with apps/libs that install conflicting signal handlers.
2012-03-24tests: depend on kgio 2.7.4
This latest version of kgio improves portability to FreeBSD-based systems.
2012-03-20log EPERM errors from invalid pid files
In some cases, EPERM may indicate a real configuration problem, but it can also just mean the pid file is stale.
2012-03-20KNOWN_ISSUES: document signal conflicts in libs/apps
Jeffrey Yeung confirmed this issue on the mailing list. ref: <E8D9E7CCC2621343A0A3BB45E8DEDFA91C682DD23D@CRPMBOXPRD04.polycom.com>
2012-03-20examples/nginx.conf: use $scheme instead of hard-coded "https"
This adds a little more flexibility to the nginx config, especially as protocols (e.g. SPDY) become more prevalent. Suggested-by: Eike Herzbach <eike@herzbach.net>
2012-03-20examples/nginx.conf: remove redundant word
From: Eike Herzbach <eike@herzbach.net>
2012-02-29Start the server if another user has a PID matching our stale pidfile.
If unicorn doesn't get terminated cleanly (for example if the machine has its power interrupted) and the pid in the pidfile gets used by another process, the current unicorn code will exit and not start a server. This tiny patch fixes that behaviour. Acked-by: Eric Wong <normalperson@yhbt.net>
2012-01-28unicorn 4.2.0 v4.2.0
The GPLv3 is now an option to the Unicorn license. The existing GPLv2 and Ruby-only terms will always remain options, but the GPLv3 is preferred. Daemonization is correctly detected on all terminals for development use (Brian P O'Rourke). Unicorn::OobGC respects applications that disable GC entirely during application dispatch (Yuichi Tateno). Many test fixes for OpenBSD, which may help other *BSDs, too. (Jeremy Evans). There is now _optional_ SSL support (via the "kgio-monkey" RubyGem). On fast, secure LANs, SSL is only intended for detecting data corruption that weak TCP checksums cannot detect. Our SSL support is remains unaudited by security experts. There are also some minor bugfixes and documentation improvements. Ruby 2.0.0dev also has a copy-on-write friendly GC which can save memory when combined with "preload_app true", so if you're in the mood, start testing Unicorn with the latest Ruby!
2012-01-28doc: update doc for Ruby 2.0.0dev CoW-friendliness
Ruby 2.0.0dev is the future and includes a CoW-friendly GC, so we shall encourage folks to give Ruby 2.0.0dev a spin.
2012-01-27script/isolate_for_tests: disable sqlite3-ruby for Ruby 2.0.0dev
We don't need it because we don't test old Rails with bleeding edge Ruby.
2012-01-27disable old Rails tests for Ruby 2.0.0
I doubt anybody would attempt to run ancient, unsupported versions of Rails on the latest (unreleased, even) versions of Ruby...
2012-01-27script/isolate_for_tests: update to kgio 2.7.2
Again, we test with the latest version.
2012-01-24update tests for Rack 1.4.1
Trying to ensure things always work with the latest version.
2012-01-08Rakefile: swap freshmeat.net URL for freecode.com
:<
2011-12-28update tests for rack 1.4.0
It's the latest and greatest version, so ensure everything works with it.
2011-12-17http: test case for "Connection: TE"
We need to be sure we don't barf on this header.
2011-12-13cleanup exception handling on SIGUSR1
No need to duplicate logic here
2011-12-13quiet possible IOError from SIGUSR1 (reopen logs)
It's possible for a SIGUSR1 signal to be received in the worker immediately before calling IO.select. In that case, do not clutter logging with IOError and just process the reopen log request.
2011-12-05socket_helper: fix grammerr fail
Oops :x
2011-12-05socket_helper: set SO_KEEPALIVE on TCP sockets
Even LANs can break or be unreliable sometimes and socket disconnect messages get lost, which means we fall back to the global (kill -9) timeout in Unicorn. While the default global timeout is much shorter (60s) than typical TCP timeouts, some HTTP application dispatches take much I/O or computational time (streaming many gigabytes), so the global timeout becomes ineffective. Under Linux, sysadmins are encouraged to lower the default net.ipv4.tcp_keepalive_* knobs in sysctl. There should be similar knobs in other operating systems (the default keepalive intervals are usually ridiculously high, too high for anything). When the listen socket has SO_KEEPALIVE set, the flag should be inherited by accept()-ed sockets.
2011-12-05socket_helper: remove out-of-date comment for TCP_NODELAY
We favor low latency and consistency with the Unix socket behavior even with TCP.
2011-12-05bump dependencies
We should always be testing with the newest available versions to watch for incompatibilities, even if we don't /require/ the latest ones to run.
2011-11-15tests: try to set a shorter path for Unix domain sockets
We're only allowed 108 bytes for Unix domain sockets. mktemp(1) usually generates path names of reasonable length and we rely on it anyways.
2011-11-15tests: just use the sha1sum implemented in Ruby
The output of SHA1 command-line tools is too unstable and I'm more comfortable with Ruby 1.9 encoding support than I was in 2009. Jeremy Evans noted the output of "openssl sha1" has changed since I last used it.
2011-11-15test_helper: ensure test client connects to valid address
You can listen on 0.0.0.0, but trying to connect to it doesn't work well on OpenBSD. Acked-by: Eric Wong <normalperson@yhbt.net>
2011-11-15t0011: fix test under OpenBSD
expr on OpenBSD uses a basic regular expression (according to re_format(7)), which doesn't support +, only *. Acked-by: Eric Wong <normalperson@yhbt.net>
2011-11-15configurator: limit timeout to 30 days
There's no practical difference between a timeout of 30 days and 68 years from an HTTP server standpoint. POSIX limits us to 31 days, actually, but there could be rounding error with floats used in Ruby time calculations and there's no real difference between 30 and 31 days, either... Thanks to Jeremy Evans for pointing out large values will throw EINVAL (on select(2) under OpenBSD with Ruby 1.9.3 and RangeError on older Rubies.
2011-11-15t: ensure SSL certificates exist on fresh test
We throw up some fake SSL certs for testing
2011-10-04OobGC: force GC.start
[ew: we need to explicitly enable GC if it is disabled and respect applications that disable GC] Acked-by: Eric Wong <normalperson@yhbt.net>