about summary refs log tree commit homepage
path: root/test
DateCommit message (Collapse)
2017-03-23Merge branch 'ccc-tcp-v3'
* ccc-tcp-v3: test_ccc: use a pipe to synchronize test http_request: support proposed Raindrops::TCP states on non-Linux
2017-03-22test_ccc: use a pipe to synchronize test ccc-tcp-v3
Sleeping 1 second to test 100 requests is too long for some systems; and not long enough for others. We need to also finish reading the sleeper response to ensure the server actually got the second request in, before sending SIGQUIT to terminate it; as it's possible for the test client to connect and abort 100 clients before the server even increments the request counter for the 2nd request.
2017-03-15Merge remote-tracking branch 'origin/ccc-tcp-v3'
* origin/ccc-tcp-v3: http_request: reduce insn size for check_client_connection support "struct tcp_info" on non-Linux and Ruby 2.2+ revert signature change to HttpServer#process_client new test for check_client_connection check_client_connection: use tcp state on linux
2017-03-14test_exec: SO_KEEPALIVE value only needs to be true
On FreeBSD 10.3, the value of SO_KEEPALIVE returned by getsockopt is 8, even when set to '1' via setsockopt. Relax the test to only ensure the boolean value is interpreted as "true". Verified independently of Ruby using the following: --------8<--------- #include <sys/types.h> #include <sys/socket.h> #include <stdio.h> static int err(const char *msg) { perror(msg); return 1; } int main(void) { int sv[2]; int set = 1; int got; socklen_t len = (socklen_t)sizeof(int); int rc; rc = socketpair(PF_LOCAL, SOCK_STREAM, 0, sv); if (rc) return err("socketpair failed"); rc = setsockopt(sv[0], SOL_SOCKET, SO_KEEPALIVE, &set, len); if (rc) return err("setsockopt failed"); rc = getsockopt(sv[0], SOL_SOCKET, SO_KEEPALIVE, &got, &len); if (rc) return err("getsockopt failed"); printf("got: %d\n", got); return 0; }
2017-03-14tests: keep disabled tests defined
Some versions of test-unit will fail if an unspecified test is attempted via "-n", so we need to define an empty test. We cannot use "skip", either, as that seems exclusive to minitest; and we won't use minitest since it has more incompatible changes than test-unit over the last 8 years. The memory leak test is gone since we're more versed in the Ruby C API nowadays, modern GCs + mallocs may be less predictable about releasing memory back to the OS.
2017-03-08revert signature change to HttpServer#process_client
We can force kgio_tryaccept to return an internal class for TCP objects by subclassing Kgio::TCPServer. This avoids breakage in any unfortunate projects which depend on our undocumented internal APIs, such as gctools <https://github.com/tmm1/gctools>
2017-03-08new test for check_client_connection
This was a bit tricky to test, but it's probably more reliable now that we're relying on TCP_INFO. Based on test by Simon Eskildsen <simon.eskildsen@shopify.com>: https://bogomips.org/unicorn-public/CAO3HKM49+aLD=KLij3zhJqkWnR7bCWVan0mOvxD85xfrW8RXOw@mail.gmail.com/
2017-03-08check_client_connection: use tcp state on linux ccc-tcp-v2
* Use a frozen empty array and a class variable for TCP_Info to avoid garbage. As far as I can tell, this shouldn't result in any garbage on any requests (other than on the first request). * Pass listener socket to #read to only check the client connection on a TCP server. * Short circuit CLOSE_WAIT after ESTABLISHED since in my testing it's the most common state after ESTABLISHED, it makes the numbers un-ordered, though. But comment should make it OK. * Definition of of `check_client_connection` based on whether Raindrops::TCP_Info is defined, instead of the class variable approach. * Changed the unit tests to pass a `nil` listener. Tested on our staging environment, and still works like a dream. I should note that I got the idea between this patch into Puma as well! https://github.com/puma/puma/pull/1227 [ew: squashed in temporary change for oob_gc.rb, but we'll come up with a different change to avoid breaking gctools <https://github.com/tmm1/gctools>] Acked-by: Eric Wong <e@80x24.org>
2017-02-15Revert "remove response_start_sent"
Oops, this was a half-baked change I was considering but forgot about. This reverts commit 69fd4f9bbff3708166fbf70163fa6e192dde1497.
2017-02-13remove response_start_sent
2016-10-20Add some tolerance (RFC2616 sec. 19.3)
Hi all. We're implementing client certificate authentication with nginx and unicorn.  Nginx configured in the following way: proxy_set_header X-SSL-Client-Cert $ssl_client_cert; When client submits certificate and nginx passes it to the unicorn, unicorn responds with 400 (Bad Request). This caused because nginx doesn't use "\r\n" they using just "\n" and multilne headers is failed to parse (I've added test). Accorording to RFC2616 section 19.3: https://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.3 "The line terminator for message-header fields is the sequence CRLF. However, we recommend that applications, when parsing such headers, recognize a single LF as a line terminator and ignore the leading CR." CRLF changed to ("\r\n" | "\n") Github commit https://github.com/uno4ki/unicorn/commit/ed127b66e162aaf176de05720f6be758f8b41b1f PS: Googling "nginx unicorn ssl_client_cert" shows the problem. 
2015-12-13http: TypedData C-API conversion
This provides some extra type safety if combined with other C extensions, as well as allowing us to account for memory usage of the HTTP parser in ObjectSpace. This requires Ruby 1.9.3+ and has remained a stable API since then. This will become officially supported when Ruby 2.3.0 is released later this month. This API has only been documented in doc/extension.rdoc (formerly README.EXT) in the Ruby source tree since April 2015, r50318
2015-11-17http_response: allow nil values in response headers
This blatantly violates Rack SPEC, but we've had this bug since March 2009[1]. Thus, we cannot expect all existing applications and middlewares to fix this bug and will probably have to support it forever. Unfortunately, supporting this bug contributes to application server lock-in, but at least we'll document it as such. [1] commit 1835c9e2e12e6674b52dd80e4598cad9c4ea1e84 ("HttpResponse: speed up non-multivalue headers") Reported-by: Owen Ou <o@heroku.com> Ref: <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
2015-10-27inheriting sockets from UNICORN_FD does not close them
For some reason, I thought invalid descriptors passed to UNICORN_FD would be automatically closed by the master process; but apparently this hasn't been the case. On the other hand, this bug has been around for over 6 years now and nobody noticed or cared enough to tell us, so fixing it might break existing setups. Since there may be users relying on this behavior, we cannot change the behavior anymore; so update the documentation and write at test to ensure we can never "fix" this bug at the expense of breaking any working setups which may be out there. Keep in mind that a before_exec hook may always be used to modify the UNICORN_FD environment by setting the close_on_exec flag and removing the appropriate descriptor from the environment. I originally intended to add the ability to inherit new listeners without a config file specification so systemd users can avoid repeating themselves in the systemd and unicorn config files, but apparently there is nothing to change in our code.
2015-10-27sd_listen_fds emulation cleanup
Re-enable and expand on the test case while we're at it for new Rubies. The bug is now fixed in Ruby 2.3.0dev as of r51576. We shall assume anybody running a pre-release 2.3.0 at this point is running a fairly recent snapshot, so we won't bother doing a finer-grained check in the test for an exact revision number.
2015-07-08test_exec: disable systemd inheritance test
Turns out ruby does have trouble emulating systemd, for now: [ruby-core:69895] https://bugs.ruby-lang.org/issues/11336 When we re-enable this test, we'll only enable it for fixed Rubies. The actual socket inheritance functionality works in any version of Ruby, of course, it's just that emulating systemd won't work until ruby-core fixes mainline Ruby.
2015-07-05test/unit/test_response.rb: compatibility with older test-unit
assert_predicate really isn't that useful even if it seems preferred in another project I work on. Avoid having folks download the latest test-unit if they're on an old version of Ruby (e.g. 1.9.3) which bundled it.
2015-07-05emulate sd_listen_fds for systemd support
systemd socket emulation shares FDs across execve, just like the built-in SIGUSR2 upgrade process in unicorn. Thus it is easy to support inheriting sockets from systemd. Tested-by: Christos Trochalakis <yatiohi@ideopolis.gr>
2015-06-30reflect changes in Rack::Utils::HTTP_STATUS_CODES
Applications may want to alter the message associated with HTTP status codes in Rack::Utils::HTTP_STATUS_CODES. Avoid memoizing status lines ahead-of-time Note: this introduces a minor performance regression, but ought to be unnoticeable unless you're running "Hello world"-type apps.
2015-06-10ensure body is closed during hijack
Middlewares such as Rack::Lock (used by Rails) break badly unless the response body is closed on hijack, so we will close it to follow the lead of other popular Rack servers. While it's unclear if there's anybody using rack.hijack with unicorn, we'll try to emulate the behavior of other servers as much as possible. ref: https://github.com/ngauthier/tubesock/issues/10
2015-06-06http: move response_start_sent into the C ext
Combined with the previous commit to eliminate the `@socket' instance variable, this eliminates the last instance variable in the Unicorn::HttpRequest class. Eliminating the last instance variable avoids the creation of a internal hash table used for implementing the "generic" instance variables found in non-pure-Ruby classes. Method entry overhead remains the same. While this change doesn't do a whole lot for unicorn memory usage where the HttpRequest is a singleton, it helps other HTTP servers which rely on this code where thousands of clients may be connected.
2015-06-04http_server: remove a few more accessors and constants
Unnecessarily exposed accessors and constants take up unnecessary memory in constant/method tables as well as using extra space in instruction sequences. Preforking servers like unicorn are a bloated pigs anyways, but saving a few hundred bytes here and there can add up and make them marginally less bad.
2015-04-24tee_input: support for Rack::TempfileReaper middleware
Rack::TempfileReaper was added in rack 1.6 to cleanup temporary files. Make Unicorn::TmpIO ducktype-compatible so Rack::TempfileReaper may be used to free up space used by temporary buffer files. Ref: <CY1PR0301MB078011EB5A22B733EB222A45A4EE0@CY1PR0301MB0780.namprd03.prod.outlook.com> Reported-by: Mike Mulvaney <MMulvaney@bna.com>
2015-03-02http: remove experimental dechunk! method
It was never used anywhere AFAIK and wastes precious bytes.
2015-02-06test_socket_helper: do not depend on SO_REUSEPORT
Older Rubies (2.0) may not define SO_REUSEPORT even if the kernel and libc support it
2015-02-06fix uninstalled testing and reduce require paths
This fixes a bug introduced in commit fe83ead4eae6f011fa15f506cd80cb4256813a92 (GNUmakefile: fix clean gem build + reduce build cruft) which broke clean Ruby installations without an existing unicorn gem installed :x
2015-02-06doc: update support status for Ruby versions
unicorn 5 will not support Ruby 1.8 anymore. Drop mentions of Rubinius, too, it's too difficult to support due to the proprietary and registration-required nature of its bug tracker. The smaller memory footprint and CoW-friendly memory allocator in mainline Ruby is a better fit for unicorn, anyways. Since Ruby 1.9+ bundles RubyGems and gem startup is faster nowadays, we'll just depend on that instead of not loading RubyGems. Drop the local.mk.sample file, too, since it's way out-of-date and probably isn't useful (I have not used it in a while).
2015-02-05socket_helper: reduce constant lookups and caching
In Ruby 1.9.2+, socket options may be specified using symbols instead of constants to avoid the need to import Socket::Constants into the namespace. This also has a nice side-effect of reducing the size of the bytecode by trading 3 instructions (getinlinecache, getconstant, setinlinecache) for one "putobject" instruction. Nowadays, we may also avoid defining OS-specific constants ourselves since 1.9+ versions of Ruby already provide them to further reduce bytecode size. getsockopt also returns Socket::Option objects in 1.9.2+, allowing us to avoid the larger "unpack('i')" method dispatch for an operand-free "int" method call. Finally, favor Object#nil? calls rather than "== nil" comparisons to reduce bytecode size even more. Since this code is only called at startup time, it does not benefit from inline caching of constant lookups in current mainline Ruby. Combined, these changes reduce YARV bytecode size by around 2K on a 64-bit system.
2014-12-21remove SSL support
We implemented barely-advertised support for SSL for two reasons: 1) to detect corruption on LANs beyond what TCP offers 2) to support other servers based on unicorn (never happened) Since this feature is largely not useful for unicorn itself, there's no reason to penalize unicorn 5.x users with bloat. In our defense, SSL support appeared in version 4.2.0 :)
2014-09-17http: reduce parser from 72 to 56 bytes on 64-bit
This allows the parser struct to fit in one cache line on x86-64 systems where cache lines are 64 bytes. Using 32-bit integer lengths is safe here because these are only for tracking offsets within the HTTP header buffer. We can safely limit HTTP headers and in-memory buffers to be less than 4GB without anybody complaining. HTTP bodies continue to use off_t (usually 64-bit, even on 32-bit systems) sizes and support as much as the OS/hardware can handle.
2014-08-18http: remove the keepalive requests limit
This was a hack for some event loops such as those found in nginx and some Rainbows! concurrency models. Using epoll/kqueue with one-shot notification (which yahns does) avoids all fairness problems.
2014-08-18remove mongrel.rubyforge.org references
mongrel.rubyforge.org has been dead longer than rubyforge.org!
2014-08-17http_response: remove Status: header
Whatever compatibility reasons which existed in 2009 likely do not exist now. Other servers (e.g. thin, puma) seem to work alright without it, so there's no reason to waste precious bytes.
2014-05-29http: remove xftrust options
This has long been considered a mistake and not documented for very long. I considered removing X-Forwarded-Proto and X-Forwarded-SSL handling, too, so rack.url_scheme is always "http", but that might lead to compatibility issues in rare apps if Rack::Request#scheme is not used.
2013-11-04tests: fix SO_REUSEPORT tests for old Linux and non-Linux
On BSD-derived platforms the getsockopt true value may be any (>= 0) value, not just one as it is on Linux. Additionally, SO_REUSEPORT is only supported since Linux 3.9, so folks on older kernels may not have it available. We still define it for Linux since kernel upgrades are usually more common than glibc upgrades. Note: we will still raise an exception at runtime if a user explicitly requests :reuseport in their config and runs an older Linux kernel. Reported-by: Andrew Hobson <ahobson@gmail.com>
2013-10-26license: allow all future versions of the GNU GPL
There is currently no GPLv4, so this change has no effect at the moment. In case the GPLv4 arrives and I am not alive to approve/review it, the lesser of evils is have give blanket approval of all future GPL versions (as published by the FSF). The worse evil is to be stuck with a license which cannot guarantee the Free-ness of this project in the future. This unfortunately means the FSF can theoretically come out with license terms I do not agree with, but the GPLv2 and GPLv3 will always be an option to all users.
2013-10-25support SO_REUSEPORT on new listeners (:reuseport)
This allows users to start an independent instance of unicorn on a the same port as a running unicorn (as long as both instances use :reuseport). ref: https://lwn.net/Articles/542629/
2013-08-17test_util: fix encoding test for Ruby trunk (2.1.0dev)
As of r40610 in ruby trunk, internal encoding is ignored if external coding is ASCII-8BIT (binary) ref: r40610 http://svn.ruby-lang.org/repos/ruby/trunk
2013-05-08HttpParser#next? becomes response_start_sent-aware
This could allow servers with persistent connection support[1] to support our check_client_connection in the future. [1] - Rainbows!/zbatery, possibly others
2013-05-04test_signals: increase delay between Process.kill
Otherwise, the signalled process may take too long to react to and process all the signals on machines with few CPUs.
2013-01-29test_exec: do not count '\n' as column width
This off-by-one error was incorrectly rejecting a line which would've been readable without wrapping on an 80-column terminal.
2012-11-29Begin writing HTTP request headers early to detect disconnected clients
This patch checks incoming connections and avoids calling the application if the connection has been closed. It works by sending the beginning of the HTTP response before calling the application to see if the socket can successfully be written to. By enabling this feature users can avoid wasting application rendering time only to find the connection is closed when attempting to write, and throwing out the result. When a client disconnects while being queued or processed, Nginx will log HTTP response 499 but the application will log a 200. Enabling this feature will minimize the time window during which the problem can arise. The feature is disabled by default and can be enabled by adding 'check_client_connection true' to the unicorn config. [ew: After testing this change, Tom Burns wrote: So we just finished the US Black Friday / Cyber Monday weekend running unicorn forked with the last version of the patch I had sent you. It worked splendidly and helped us handle huge flash sales without increased response time over the weekend. Whereas in previous flash traffic scenarios we would see the number of HTTP 499 responses grow past the number of real HTTP 200 responses, over the weekend we saw no growth in 499s during flash sales. Unexpectedly the patch also helped us ward off a DoS attack where the attackers were disconnecting immediately after making a request. ref: <CAK4qKG3rkfVYLyeqEqQyuNEh_nZ8yw0X_cwTxJfJ+TOU+y8F+w@mail.gmail.com> ] Signed-off-by: Eric Wong <normalperson@yhbt.net>
2012-11-13tests: remove assert_nothing_raised (part 2)
assert_nothing_raised ends up hiding errors and backtraces, making things harder to debug. Since Test::Unit already fails on uncaught exceptions, there is no need to assert on the lack of exceptions for a successful test run. This is a followup to commit 5acf5522295c947d3118926d1a1077007f615de9
2012-08-06avoid assert_nothing_raised in unit tests
It's better to show errors and backtraces when stuff breaks
2012-07-28remove Rails-oriented integration tests
It's too much overhead to keep Rails-specific tests working, especially when it's hauling in an ancient version of SQLite3. Since Rails 3 has settled down with Rack and unicorn_rails is unlikely to need changing in the future, we can drop these tests.
2012-01-27disable old Rails tests for Ruby 2.0.0
I doubt anybody would attempt to run ancient, unsupported versions of Rails on the latest (unreleased, even) versions of Ruby...
2011-12-17http: test case for "Connection: TE"
We need to be sure we don't barf on this header.
2011-11-15test_helper: ensure test client connects to valid address
You can listen on 0.0.0.0, but trying to connect to it doesn't work well on OpenBSD. Acked-by: Eric Wong <normalperson@yhbt.net>
2011-09-15add preliminary SSL support
This will also be the foundation of SSL support in Rainbows! and Zbatery. Some users may also want to use this in Unicorn on LANs to meet certain security/auditing requirements. Of course, Nightmare! (in whatever form) should also be able to use it.
2011-08-29add GPLv3 option to the license
Existing license terms (Ruby-specific) and GPLv2 remain in place, but GPLv3 is preferred as it helps with distribution of AGPLv3 code and is explicitly compatible with Apache License (v2.0). Many more reasons are documented by the FSF: https://www.gnu.org/licenses/quick-guide-gplv3.html http://gplv3.fsf.org/rms-why.html ref: http://thread.gmane.org/gmane.comp.lang.ruby.unicorn.general/933