| Date | Commit message (Collapse) |
|---|
|
IO#sysread may only capture the 103 response and return before
the server can send the 200. Since we don't support persistent
connections, we can just use IO#read to rely on the server
giving us an EOF after the 200 is sent.
Cc: Jean Boussier <jean.boussier@gmail.com>
|
|
While not part of the rack spec, this API is exposed
by both puma and falcon, and Rails use it when available.
The 103 Early Hints response code is specified in RFC 8297.
|
|
We need to favor "Transfer-Encoding: chunked" over
"Content-Length" in the request header if they both exist.
Furthermore, we now reject redundant chunking and cases where
"chunked" is not the final encoding.
We currently do not and have no plans to decode "gzip",
"deflate", or "compress" encoding as described by RFC 7230.
That's a job more appropriate for middleware, anyways.
cf. https://tools.ietf.org/html/rfc7230
https://www.rfc-editor.org/errata_search.php?rfc=7230
|
|
We can start using Ruby 1.9 APIs, nowadays
|
|
It was added nearly 11 years ago in commit 6945342a1f0a4caa
("Transfer-Encoding: chunked streaming input support") but
never used.
|
|
My hardware gets worse and worse every year :<
|
|
Ruby 2.7.0dev warns on them
|
|
In preparation for kgio removal, I want to ensure we can
maintain existing performance when swapping kgio_tryaccept
for accept_nonblock on Ruby 2.3+
There's plenty of TCP benchmarking tools, but TCP port reuse
delays hurt predictability since unicorn doesn't do persistent
connections.
So this is exclusively for Unix sockets and uses Perl instead
of Ruby since I don't want to be bothered with GC
unpredictability on the client side.
|
|
This is intended to demonstrate how badly we suck at dealing
with slow clients making uploads. It can help users evaluate
alternative fully-buffering reverse proxies, because nginx
should not be the only option.
|
|
This is intended to demonstrate how badly we suck at dealing
with slow clients. It can help users evaluate alternative
fully-buffering reverse proxies, because nginx should not
be the only option.
Update the benchmark README while we're at it
|
|
String#-@ deduplicates strings starting with Ruby 2.5.0
Hash#[]= deduplicates strings starting in Ruby 2.6.0-rc1
This allows us to save a small amount of memory by sharing
objects with other parts of the stack (e.g. Rack).
|
|
We have never had any need for pipes with the default 64K
capacity on Linux. Our pipes are only used for tiny writes
in signal handlers and to perform parent shutdown detection.
With the current /proc/sys/fs/pipe-user-pages-soft
default, only 1024 pipes can be created by an unprivileged
user before Linux clamps down the pipe size to 4K (a single page)
for newly-created pipes[1].
So avoid penalizing OTHER pipe users who could benefit from the
increased capacity and use only a single page for ourselves.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/pipe.c?h=v4.18#n642
|
|
Ruby trunk started warning about more mismatched indentations
starting around r62836.
|
|
IO#wait_readable is introduced since 2.0
I confirmed we can pass tests for all versions of Ruby with this patch.
https://github.com/mtsmfm/unicorn/pull/2
|
|
Add a new "check-warnings" target to the GNUmakefile to make
checking for this easier. Warnings aren't fatal, and newer
versions of Ruby tend to increase warnings.
|
|
We need to ensure the portability of the sd_listen_fds emulation
test, too, which didn't get tested on my FreeBSD 10.3 install
due to it being on Ruby 2.2
Followup-to: 4ce6b00f75f1 ("test_exec: SO_KEEPALIVE value only needs to be true")
|
|
* ccc-tcp-v3:
test_ccc: use a pipe to synchronize test
http_request: support proposed Raindrops::TCP states on non-Linux
|
|
Sleeping 1 second to test 100 requests is too long for some
systems; and not long enough for others.
We need to also finish reading the sleeper response to ensure
the server actually got the second request in, before sending
SIGQUIT to terminate it; as it's possible for the test client
to connect and abort 100 clients before the server even
increments the request counter for the 2nd request.
|
|
* origin/ccc-tcp-v3:
http_request: reduce insn size for check_client_connection
support "struct tcp_info" on non-Linux and Ruby 2.2+
revert signature change to HttpServer#process_client
new test for check_client_connection
check_client_connection: use tcp state on linux
|
|
On FreeBSD 10.3, the value of SO_KEEPALIVE returned by
getsockopt is 8, even when set to '1' via setsockopt.
Relax the test to only ensure the boolean value is
interpreted as "true".
Verified independently of Ruby using the following:
--------8<---------
#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
static int err(const char *msg)
{
perror(msg);
return 1;
}
int main(void)
{
int sv[2];
int set = 1;
int got;
socklen_t len = (socklen_t)sizeof(int);
int rc;
rc = socketpair(PF_LOCAL, SOCK_STREAM, 0, sv);
if (rc) return err("socketpair failed");
rc = setsockopt(sv[0], SOL_SOCKET, SO_KEEPALIVE, &set, len);
if (rc) return err("setsockopt failed");
rc = getsockopt(sv[0], SOL_SOCKET, SO_KEEPALIVE, &got, &len);
if (rc) return err("getsockopt failed");
printf("got: %d\n", got);
return 0;
}
|
|
Some versions of test-unit will fail if an unspecified test is
attempted via "-n", so we need to define an empty test.
We cannot use "skip", either, as that seems exclusive to
minitest; and we won't use minitest since it has more
incompatible changes than test-unit over the last 8 years.
The memory leak test is gone since we're more versed in the
Ruby C API nowadays, modern GCs + mallocs may be less
predictable about releasing memory back to the OS.
|
|
We can force kgio_tryaccept to return an internal class
for TCP objects by subclassing Kgio::TCPServer.
This avoids breakage in any unfortunate projects which depend on
our undocumented internal APIs, such as gctools
<https://github.com/tmm1/gctools>
|
|
This was a bit tricky to test, but it's probably more reliable
now that we're relying on TCP_INFO.
Based on test by Simon Eskildsen <simon.eskildsen@shopify.com>:
https://bogomips.org/unicorn-public/CAO3HKM49+aLD=KLij3zhJqkWnR7bCWVan0mOvxD85xfrW8RXOw@mail.gmail.com/
|
|
* Use a frozen empty array and a class variable for TCP_Info to avoid
garbage. As far as I can tell, this shouldn't result in any garbage on
any requests (other than on the first request).
* Pass listener socket to #read to only check the client connection on
a TCP server.
* Short circuit CLOSE_WAIT after ESTABLISHED since in my testing it's
the most common state after ESTABLISHED, it makes the numbers
un-ordered, though. But comment should make it OK.
* Definition of of `check_client_connection` based on whether
Raindrops::TCP_Info is defined, instead of the class variable
approach.
* Changed the unit tests to pass a `nil` listener.
Tested on our staging environment, and still works like a dream.
I should note that I got the idea between this patch into Puma as well!
https://github.com/puma/puma/pull/1227
[ew: squashed in temporary change for oob_gc.rb, but we'll come
up with a different change to avoid breaking gctools
<https://github.com/tmm1/gctools>]
Acked-by: Eric Wong <e@80x24.org>
|
|
Oops, this was a half-baked change I was considering
but forgot about.
This reverts commit 69fd4f9bbff3708166fbf70163fa6e192dde1497.
|
|
|
|
Hi all.
We're implementing client certificate authentication with nginx and
unicorn.
Nginx configured in the following way:
proxy_set_header X-SSL-Client-Cert $ssl_client_cert;
When client submits certificate and nginx passes it to the unicorn,
unicorn responds with 400 (Bad Request). This caused because nginx
doesn't use "\r\n" they using just "\n" and multilne headers is failed
to parse (I've added test).
Accorording to RFC2616 section 19.3:
https://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.3
"The line terminator for message-header fields is the sequence CRLF.
However, we recommend that applications, when parsing such headers,
recognize a single LF as a line terminator and ignore the leading CR."
CRLF changed to ("\r\n" | "\n")
Github commit
https://github.com/uno4ki/unicorn/commit/ed127b66e162aaf176de05720f6be758f8b41b1f
PS: Googling "nginx unicorn ssl_client_cert" shows the problem.
|
|
This provides some extra type safety if combined with other
C extensions, as well as allowing us to account for memory usage of
the HTTP parser in ObjectSpace.
This requires Ruby 1.9.3+ and has remained a stable API since
then. This will become officially supported when Ruby 2.3.0 is
released later this month.
This API has only been documented in doc/extension.rdoc (formerly
README.EXT) in the Ruby source tree since April 2015, r50318
|
|
This blatantly violates Rack SPEC, but we've had this bug since
March 2009[1]. Thus, we cannot expect all existing applications
and middlewares to fix this bug and will probably have to
support it forever.
Unfortunately, supporting this bug contributes to application
server lock-in, but at least we'll document it as such.
[1] commit 1835c9e2e12e6674b52dd80e4598cad9c4ea1e84
("HttpResponse: speed up non-multivalue headers")
Reported-by: Owen Ou <o@heroku.com>
Ref: <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
|
|
For some reason, I thought invalid descriptors passed to UNICORN_FD
would be automatically closed by the master process; but apparently
this hasn't been the case. On the other hand, this bug has been
around for over 6 years now and nobody noticed or cared enough to
tell us, so fixing it might break existing setups.
Since there may be users relying on this behavior, we cannot change
the behavior anymore; so update the documentation and write at test
to ensure we can never "fix" this bug at the expense of breaking
any working setups which may be out there.
Keep in mind that a before_exec hook may always be used to modify
the UNICORN_FD environment by setting the close_on_exec flag and
removing the appropriate descriptor from the environment.
I originally intended to add the ability to inherit new listeners
without a config file specification so systemd users can avoid
repeating themselves in the systemd and unicorn config files,
but apparently there is nothing to change in our code.
|
|
Re-enable and expand on the test case while we're at it for new
Rubies. The bug is now fixed in Ruby 2.3.0dev as of r51576. We
shall assume anybody running a pre-release 2.3.0 at this point is
running a fairly recent snapshot, so we won't bother doing a
finer-grained check in the test for an exact revision number.
|
|
Turns out ruby does have trouble emulating systemd, for now:
[ruby-core:69895] https://bugs.ruby-lang.org/issues/11336
When we re-enable this test, we'll only enable it for fixed Rubies.
The actual socket inheritance functionality works in any version of
Ruby, of course, it's just that emulating systemd won't work until
ruby-core fixes mainline Ruby.
|
|
assert_predicate really isn't that useful even if it seems
preferred in another project I work on. Avoid having folks
download the latest test-unit if they're on an old version of
Ruby (e.g. 1.9.3) which bundled it.
|
|
systemd socket emulation shares FDs across execve, just like
the built-in SIGUSR2 upgrade process in unicorn. Thus it is
easy to support inheriting sockets from systemd.
Tested-by: Christos Trochalakis <yatiohi@ideopolis.gr>
|
|
Applications may want to alter the message associated with HTTP
status codes in Rack::Utils::HTTP_STATUS_CODES. Avoid memoizing
status lines ahead-of-time
Note: this introduces a minor performance regression, but ought to
be unnoticeable unless you're running "Hello world"-type apps.
|
|
Middlewares such as Rack::Lock (used by Rails) break badly unless
the response body is closed on hijack, so we will close it to follow
the lead of other popular Rack servers.
While it's unclear if there's anybody using rack.hijack with unicorn,
we'll try to emulate the behavior of other servers as much as
possible.
ref: https://github.com/ngauthier/tubesock/issues/10
|
|
Combined with the previous commit to eliminate the `@socket'
instance variable, this eliminates the last instance variable
in the Unicorn::HttpRequest class.
Eliminating the last instance variable avoids the creation of a
internal hash table used for implementing the "generic" instance
variables found in non-pure-Ruby classes. Method entry overhead
remains the same.
While this change doesn't do a whole lot for unicorn memory usage
where the HttpRequest is a singleton, it helps other HTTP servers
which rely on this code where thousands of clients may be connected.
|
|
Unnecessarily exposed accessors and constants take up unnecessary
memory in constant/method tables as well as using extra space in
instruction sequences.
Preforking servers like unicorn are a bloated pigs anyways,
but saving a few hundred bytes here and there can add up and
make them marginally less bad.
|
|
Rack::TempfileReaper was added in rack 1.6 to cleanup temporary
files. Make Unicorn::TmpIO ducktype-compatible so
Rack::TempfileReaper may be used to free up space used by temporary
buffer files.
Ref: <CY1PR0301MB078011EB5A22B733EB222A45A4EE0@CY1PR0301MB0780.namprd03.prod.outlook.com>
Reported-by: Mike Mulvaney <MMulvaney@bna.com>
|
|
It was never used anywhere AFAIK and wastes precious bytes.
|
|
Older Rubies (2.0) may not define SO_REUSEPORT even if the
kernel and libc support it
|
|
This fixes a bug introduced in
commit fe83ead4eae6f011fa15f506cd80cb4256813a92
(GNUmakefile: fix clean gem build + reduce build cruft)
which broke clean Ruby installations without an existing
unicorn gem installed :x
|
|
unicorn 5 will not support Ruby 1.8 anymore.
Drop mentions of Rubinius, too, it's too difficult to support due to
the proprietary and registration-required nature of its bug tracker.
The smaller memory footprint and CoW-friendly memory allocator in
mainline Ruby is a better fit for unicorn, anyways.
Since Ruby 1.9+ bundles RubyGems and gem startup is faster nowadays,
we'll just depend on that instead of not loading RubyGems.
Drop the local.mk.sample file, too, since it's way out-of-date
and probably isn't useful (I have not used it in a while).
|
|
In Ruby 1.9.2+, socket options may be specified using symbols
instead of constants to avoid the need to import Socket::Constants
into the namespace. This also has a nice side-effect of reducing
the size of the bytecode by trading 3 instructions (getinlinecache,
getconstant, setinlinecache) for one "putobject" instruction.
Nowadays, we may also avoid defining OS-specific constants ourselves
since 1.9+ versions of Ruby already provide them to further reduce
bytecode size.
getsockopt also returns Socket::Option objects in 1.9.2+,
allowing us to avoid the larger "unpack('i')" method dispatch
for an operand-free "int" method call.
Finally, favor Object#nil? calls rather than "== nil" comparisons
to reduce bytecode size even more.
Since this code is only called at startup time, it does not benefit
from inline caching of constant lookups in current mainline Ruby.
Combined, these changes reduce YARV bytecode size by around 2K on a
64-bit system.
|
|
We implemented barely-advertised support for SSL for two reasons:
1) to detect corruption on LANs beyond what TCP offers
2) to support other servers based on unicorn (never happened)
Since this feature is largely not useful for unicorn itself,
there's no reason to penalize unicorn 5.x users with bloat.
In our defense, SSL support appeared in version 4.2.0 :)
|
|
This allows the parser struct to fit in one cache line on
x86-64 systems where cache lines are 64 bytes.
Using 32-bit integer lengths is safe here because these are only for
tracking offsets within the HTTP header buffer. We can safely limit
HTTP headers and in-memory buffers to be less than 4GB without
anybody complaining.
HTTP bodies continue to use off_t (usually 64-bit, even on 32-bit
systems) sizes and support as much as the OS/hardware can handle.
|
|
This was a hack for some event loops such as those found in nginx
and some Rainbows! concurrency models. Using epoll/kqueue with
one-shot notification (which yahns does) avoids all fairness
problems.
|
|
mongrel.rubyforge.org has been dead longer than rubyforge.org!
|
|
Whatever compatibility reasons which existed in 2009 likely do not exist
now. Other servers (e.g. thin, puma) seem to work alright without it,
so there's no reason to waste precious bytes.
|
|
This has long been considered a mistake and not documented for very
long.
I considered removing X-Forwarded-Proto and X-Forwarded-SSL
handling, too, so rack.url_scheme is always "http", but that might
lead to compatibility issues in rare apps if Rack::Request#scheme is
not used.
|