| Date | Commit message (Collapse) |
|---|
|
This release now depends on "kgio-sendfile", a (hopefully temporary)
fork of the original sendfile gem for mainline ruby trunk
compatibility and a (probably correct) FreeBSD-related bugfix.
kqueue/FreeBSD support is considered highly experimental. Of course;
you should never rely on anything in production unless you can get bugs
fixed in every part of your stack; even the kernel. yahns (ab)uses
kqueue and epoll in uncommon ways, so you may encounter subtle kernel
bugs along the way.
Because yahns has been self-hosting its own website for months without
crashes or major problems (BORING! :P), I've decided to start hosting the
yahns website <http://yahns.YHBT.net/README> with ruby trunk (currently
r45341).
yahns - dangerous by design (and sleepy!)
|
|
We favor foo(&:bar) when possible for conciseness.
|
|
This likely makes no difference in 99% of real world situations with
fast response generation. The only case where lack of output buffer
bypass makes a difference is when the following sequence happens:
1) a giant response cannot fit into socket buffers
2) temporary file created for buffering
3) client consumes output so we get more space in socket buffers
4) app response generation continues (streaming response)
5) we successfully write _all_ previously buffered data to socket
6) we may bypass buffering for data generated in step 4
So right now, we cannot do step 6 outside of Linux. Instead, once an
output buffer is created; we must always write to the output buffers.
|
|
There's a feature request for better coverage support in ruby-trunk
https://bugs.ruby-lang.org/issues/9508
At minimum, we need to preserve compatibility; but we should set
aside time to take advantage of the extra coverage support.
|
|
* origin/freebsd:
implement kqueue and sendfile compatibility support
test: log skipped tests on non-Linux systems
test_server: check_client_connection uses Unix sockets
test_client_expire: add delays for non-Linux OSes
wbuf_common: avoid trysendfile on empty sf_count
sigevent_pipe: kgio_wait_writable takes variadic args
queue_quitter_pipe: do not prematurely close reader
|
|
Hopefully kgio-sendfile is only a temporary fork until
Toby gets back to maintaining the normal sendfile gem.
|
|
Tested on Debian GNU/kFreeBSD (sid):
make check SENDFILE_BROKEN=1
|
|
|
|
TCP socket timing is too varied over different OSes, Unix sockets
seem to test more reliably than TCP ones on my Debian GNU/kFreeBSD
system.
|
|
Not all TCP stacks behave the same with timing-sensitive tests.
Fortunately these timing differences aren't very critical over real
networks.
|
|
We may unnecessarily drop persistent connections from this bug,
and we had an incorrect assertion in our unit test, even.
|
|
We may pass a timeout to kgio_wait_writable during shutdown.
|
|
This will lead to ENOENT when placing writer into kqueue
changelist via kevent(2)
|
|
On responses with a known, non-zero Content-Length, yahns will now
automatically use the MSG_MORE flag when sending HTTP headers. This
reduces TCP packet transmits and allows clients to wake up and read
one less time.
This is similar to the TCP_NOPUSH/TCP_CORK functionality of other
servers, but requires no additional syscalls. It is only supported
on the Linux kernel, however (but yahns is currently epoll-only
with kqueue support on the horizon).
Eric Wong (5):
quiet down EHOSTUNREACH errors
http_response: use kgio_syssend with MSG_MORE
load yahns/version file
socket_helper: remove SO_REUSEPORT define for untested arches
response: do not use MSG_MORE on empty bodies
|
|
We must not cork response headers when the response body is empty,
otherwise those headers will be delayed by 200ms.
|
|
We'll let systems with updated glibc headers define those
(or interested users, if any, on those architectures test
and report back to us). No point in causing unnecessary
breakage if we screw things up.
|
|
We need this for "yahns --version" on the command line
|
|
If Content-Length is known, try to save some bandwidth by
corking the headers until the body is sendable. This allows
us to avoid sending an extra packet for small HTTP responses.
This allows high-performance websites like YHBT.net to be served
faster!
|
|
We can't do anything about EHOSTUNREACH, so quiet the backtrace.
I've noticed this happening very rarely (once every few weeks?)
while yahns was exposed to untrusted traffic.
|
|
Mostly fixes for the extras/* stuff running on yahns.YHBT.net and
documentation/packaging fixes. Anyways, this is hosting the
yahns website for a while now (running rack.git) without any
major issues.
Also, the mailing list will need to move/change because Rubyforge is
going away. Everything I said for the unicorn ML applies to this list,
too: http://mid.gmane.org/20131217015614.GB1125@dcvr.yhbt.net
|
|
Unicorn::HttpResponse#httpdate is currently not thread-safe without GVL.
|
|
Proper POSIX filesystems are encoding-agnostic.
|
|
ab(1) is less common than our other dependencies, so allow tests
to run without it.
|
|
unicorn uses semantic versioning, and we are compatible
with 4.7.0, so use it.
|
|
Having two ~> dependencies doesn't OR the dependency as intended.
In either case, assume 6.x is not compatible out-of-the box and
do not enable it. While we're at it, document our dependency
policy for the test library.
|
|
We will override it internally regardless of middlewares.
|
|
This was causing zombies on the bogomips.org cgit instance.
|
|
We do not use these strings in string assignment, and we're not
in danger of modifying these strings in place, so do not freeze
them.
|
|
The body may contain extra repeated newlines, of course
|
|
This allows yahns to continue using sendfile when clients
request gzipped/bzipped tarballs on my server.
|
|
Clients may be requesting gzipped files through Rack::Deflater,
which will attempt to further compress files.
|
|
This is mainly needed for the regexp in extras/try_gzip_static.rb
(POSIX filesystem paths have no encoding, it's just a bag of bytes).
Since we host apps of all types and for all (human)
languages/encodings, all of our internals must be encoding-agnostic.
|
|
Leave that up to Rack::Chunked/Rack::ContentLength.
Chunking ourselves interacts badly with Rack::Deflater, since
Deflater will blindly deflate already-chunked portions.
|
|
In case we have bugs, this can help us find bugs in our code.
|
|
Some attackers may try /path/to/file/foo where /path/to/file
is actually a valid path to a regular file. Of course, requests
like this work on dynamic websites, but not static file mappings
because Unix directories and files cannot be the same thing.
|
|
There are several important bugfixes around graceful shutdown.
The shutdown_timeout directive should work more consistently
and work even if an app is streaming/trickling slowly to
a client.
This also plugs an odd FD leak in response buffering.
There's also some internal comments/documentation for folks
reviewing the strange internals of yahns.
Otherwise, it seems to be capable of serving its own website,
http://yahns.yhbt.net/README quite well.
Rack application authors merely need to write code as if they have a
gun to their head if they expect to keep code running on yahns.
Again, yahns is extremely intolerant of bugs in the applications
it hosts. Otherwise I'm comfortable in the stability of yahns
itself.
Eric Wong (17):
http_client: do not dump backtrace on ETIMEDOUT
ensure we close response body if buffering caught up
http_response: reorder wbuf_maybe on successful early flush
wbuf: document reasoning for the design of these clases
build: improve NEWS generation, add atom feed
exec_cgi: release pipe/pid sooner
SIGWINCH works after SIGUSR2 upgrades
shutdown_timeout works around long-running response/apps
doc: recommend worker_processes if the app uses SIGCHLD
fdmap: simplify IO expiry interface
fdmap: document + fix for level-trigger
queue_epoll: document epoll concurrency caveats
doc: caution users against disabling buffering
queue: eliminate :wait_readwrite
test: exec_cgi test uses worker_processes
test_server: remove unneccessary IO#wait call
test: workaround sockets not being binary on rbx
|
|
ref: https://github.com/rubinius/rubinius/issues/2772
|
|
This was failing under Rubinius and not necessary after all
ref: https://github.com/rubysl/rubysl-io-wait/issues/2
|
|
Following our own advice in
commit a79a6d8775171ad5cceda9bb3a77946ba60e26ce
(doc: recommend worker_processes if the app uses SIGCHLD)
|
|
Simultaneously waiting for both readability and writability is
likely unnecessary. Sockets become writable much more commonly,
and if we need to read, we'll issue a read the next time it enters
yahns_step (after it becomes writable).
Also, this makes kqueue easier-to-implement as I don't believe it
implements a way to combine EVFILT_READ with EVFILT_WRITE
(especially not safely while relying on EV_ONESHOT behavior).
|
|
Users may be tempted to disable buffering because it improves
synthetic benchmark performance. Synthetic benchmarks are
just that and not indicative of what evil (or really crippled)
clients can do to a server.
|
|
Multithreaded epoll is a tricky interface to use. Fortunately it
should be hidden from users and yahns is no different from any other
multithreaded server from a user application programming
perspective (which requires a lot of careful programming in the
first place).
|
|
Our QueueQuitter may race with epoll_wait and GC, potentially
triggering memory corruption if a level-trigger object was
placed into the ready list and no longer referenced.
We'll work around this problem by making the QueueQuitter object
a thread-local (on thread failure) to prevent concurency issues
where the epoll pointer no longer points to a Ruby object.
|
|
The first argument of the old __expire_for method was completely
unused in all but one invocation of it, so it is not worth it. This
also simplifies the code a bit too.
|
|
If worker_processes are not enabled, our SIGCHLD handler may
conflict with one installed by the application. Fortunately it is
uncommon in Ruby web apps to rely on SIGCHLD, but it happens and
that is bad.
|
|
apps/responses may trickle infinitely, so our QueueQuitter may
not properly kill epoll worker threads, unfortunately. So we'll
start the timer and stop waiting (risking segfaults/ugliness) by
closing epoll descriptors
|
|
We must know we're daemonized after a SIGUSR2 upgrade, even if we
don't use the pipe to signal to the controlling terminal we've
started up.
|
|
We don't want to leave extra resources lying around when slow
clients read from us and yahns is forced to buffer.
yahns delays the close of the request body until the request is
fully written so response loggers can have an accurate time of
how long it takes.
|
|
This should hopefully make it easier to share info about
new releases.
|
|
The choice to jump directly to temporary files on EAGAIN may
seem odd to some, document our reasoning for choosing this approach.
|