Date | Commit message (Collapse) |
|
Using the 'update-copyright' script from gnulib[1]:
git ls-files | UPDATE_COPYRIGHT_HOLDER='all contributors' \
UPDATE_COPYRIGHT_USE_INTERVALS=2 \
xargs /path/to/gnulib/build-aux/update-copyright
We're also switching to 'GPL-3.0+' as recommended by SPDX
to be consistent with our gemspec and other metadata
(as opposed to the longer but equivalent "GPLv3 or later").
[1] git://git.savannah.gnu.org/gnulib.git
|
|
There are likely yet-to-be-discovered bugs in here.
Also, keeping explicit #freeze calls for 2.2 users, since most
users have not migrated to 2.3, yet.
|
|
We do not expose any sort of API beyond what's in the config file
manpage to our users. Do not mislead them into thinking we
currently have a stable API (though I'm considering one).
This avoids wasting disk space and installation time for users who
do not have a: "gem: --no-ri --no-rdoc" line in their ~/.gemrc
|
|
Ruby 2.3 will support this feature to reduce allocations
for common errors.
|
|
This typo would've only be triggered if the sendfile gem itself
turns out to be buggy and returns an undocumented return code.
|
|
Remove unnecessary anonymous procs and conditionals to save some
bytecode memory Also, no valid PID can be zero, as kill(2)
treats zero specially.
|
|
Reduce unnecessary arguments to "exit" and "exit!". Additionally,
rely on a "putnil" instruction rather than a "putstring" argument
with an unnecessary string operand for an uncommon code path.
|
|
This can reduce the amount of garbage we have by a small
amount. Once Ruby 2.3 comes out, we can rely on the
"frozen_string_literal: true" directive
|
|
Clarify APP_CONFIG is only public for modules maintained within
yahns, not external users.
While we're at it, avoid the http:// -> https:// redirect for
bugs.ruby-lang.org in the comment.
|
|
Future updates may use the update-copyright script in gnulib:
git ls-files | UPDATE_COPYRIGHT_HOLDER='all contributors' \
UPDATE_COPYRIGHT_USE_INTERVALS=2 \
xargs /path/to/gnulib/build-aux/update-copyright
|
|
Clients may disconnect on us while we're writing the response
here, so we should shut the backtrace up to avoid polluting
logs.
|
|
Unfortunately we cannot rely on the RubyVM for optimizing hash
deletions, yet, so manually freeze for this string since it
(unlike every line after it) is a hot path if handling request
bodies.
Update a comment while we're at it.
|
|
systemd socket emulation shares FDs across execve, just like
the built-in SIGUSR2 upgrade process in unicorn. Thus it is
easy to support inheriting sockets from systemd.
|
|
TCP socket options are now set when inheriting existing sockets from
a parent process. I'm fairly certain all the TCP setsockopt knobs
we use are idempotent and harmless to change.
If anything, the only directive I'd be uncomfortable changing is
shortening the listen(2) (aka :backlog) size, but we've always
changed that anyways since it also applies to UNIX sockets.
Note: removing a configuration knob in a yahns config file can not
reset the value to the OS-provided default setting. Inherited
sockets must use a new setting to override existing ones.
(or the socket needs to be closed and re-created in the process
launcher before yahns inherits it).
Based on unicorn commit 1db9a8243d42cc86d5ca4901bceb305061d0d212
Noticed-by: Christos Trochalakis <yatiohi@ideopolis.gr>
<20150626114129.GA25883@luke.ws.skroutz.gr>
|
|
This saves around 200 bytes on x86-64 and potentially improves
CPU cache performance. This does not reduce inline method
cache overhead as String#<< already has optimized dispatch
support (opt_ltlt in insns.def in Ruby 1.9+)
|
|
Ruby (MRI) 2.1 optimizes allocations away on String#freeze with
literal strings.
Furthermore, Ruby 2.2 optimizes away literal string allocations
when they are used as arguments to Hash#[] and Hash#[]=
Thus we can avoid expensive constant lookups and cache overhead
by taking advantage of advancements in Ruby.
Since Ruby 2.2 has been out for 7 months, now; it ought to be safe
to introduce minor performance regressions for folks using older
Rubies (1.9.3+ remains supported) to benefit folks on the latest
Ruby.
|
|
Rack::Utils::HTTP_STATUS_CODES may be altered by the underlying
application, allow changes to that to be reflected in our responses
and do not rely on the Unicorn::HttpResponse::CODES hash which
will probably go away soon.
|
|
Middlewares such as Rack::Lock (used by Rails) break badly unless
the response body is closed on hijack, so we will close it to follow
the lead of other popular Rack servers.
While it's unclear if there's anybody using rack.hijack besides
yahns/proxy_pass we'll try to emulate the behavior of other servers
as much as possible.
ref: https://github.com/ngauthier/tubesock/issues/10
While we're at it, use DieIfUsed correctly in test_ssl.rb :x
|
|
We can use symbols in the SleepyPenguin APIs to trade speed for
space in uncommon code paths.
|
|
We don't need to waste a valuable ivar slot on each socket
when we know unicorn already maintains this flag for us.
|
|
This has no effect for the (default) single process case with
no master/worker relationship as that does not support SIGWINCH.
Some process managers such as foreman and daemontools rely on
yahnsnot daemonizing, but we still want to be able to process
SIGWINCH in that case.
stdout and stderr may be redirected to a pipe (for cronolog or
similar process), so those are less likely to be attached to a TTY
than stdin. This also allows users to process SIGWINCH when running
inside a regular terminal if they redirect stdin to /dev/null.
This follows unicorn commit a6077391bb62d0b13016084b0eea36b987afe8f0
Thanks to Dan Moore for suggesting it on the unicorn list.
|
|
We shouldn't blindly pass the "Server" tag through, since we may
be proxying Apache instances and we don't want to misadvertise,
either.
IMHO, it is best to say nothing at all to save bandwidth and
reduce the potential for attackers in case a vulnerability is
discovered in yahns.
|
|
It does not release memory immediately and GC can handle it
reliably, so don't waste a constant lookup + cache entry on it.
While we're at it, explain why we can't do a simpler
respond_to? check instead.
|
|
Some applications may lazily load code during app dispatch,
triggering LoadError or SyntaxError exceptions. Log the error and
backtrace so application maintainers can more easily notice and
diagnose problems.
Keep in mind users are likely to have performance and race condition
problems with lazy loading, and the process may still be in a bad
state due to partially-loaded code. This commit is only intended to
give application authors a chance to notice and fix or avoid
problems in the future.
Note: logging fatal exceptions by default in all threads was
proposed in ruby-core, but currently not implemented in any released
version:
https://bugs.ruby-lang.org/issues/6647
Reported-by: Lin Jen-Shin (godfat) <godfat@godfat.org>
<CAA2_N1umJO12XH9r+JHnA6r=z=Mwp_PqOrdnW65oqW2K2-iAoQ@mail.gmail.com>
|
|
Rack::TempfileReaper was added in rack 1.6 to cleanup temporary
files. Make Yahns::TmpIO ducktype-compatible and put it into
env['rack.tempfiles'] array so Rack::TempfileReaper may be used to
free up space used by temporary buffer files.
ref: commit 3bdf5481e49d76b4502c51e5bdd93f68bfd1f0b4 in unicorn
|
|
Arrays are less verbose, but they have more bytecode overhead
which actually matters at runtime.
|
|
We do not want rack.hijack users relying on kgio_* methods since
kgio is trying to make itself obsolete (as Ruby itself adopts
kgio features). This is a bit wonky since our common case tries
to minimize object allocation by only using the Kgio::Socket
derived class.
|
|
OpenSSL::SSL::SSLSocket does not actually respond to a shutdown
method, and it would not be safe to call anyways. Merely
shutdown at the OS level and let any handling thread clean it
up.
|
|
Proc object allocation is not cheap, so avoid doing it and
allow Yahns::HttpClient to be assigned as the
env["rack.hijack"] callback for Rack applications.
|
|
No point in wasting space and reducing code clarity with this
method to remove references to live objects.
|
|
When calling proxy_busy_mod_blocked to re-enable a descriptor via
epoll, the ensure block is dangerous because the "ensure" clause
modifies the object after the ReqRes is injected into epoll.
This is extremely dangerous as we give up exclusive access to
the object once we call epoll_ctl.
This simplifies the code a bit while we're at it.
|
|
Reactivating a client socket after the proxied response is
complete requires the object remain visible to the Ruby GC while
no thread is accessing it. So we must place the object back
into the fdmap to prevent the GC from eating it (and having
epoll return an invalid pointer).
|
|
Bad connections or dead upstreams cannot be solved looking at a
backtrace, so avoid polluting logs with them and making other
problems less visible.
|
|
We don't need optimized dispatch methods in cold code, so use
the more space-efficient "nil?" method dispatch to save us one
word per-call site for a rough total of 24 bytes saving.
|
|
We may set binary mode upon open by passing "b" in the mode string,
so avoid the extra method dispatch and bytecode/cache overhead that
entails.
|
|
The entire idea of a one-shot-based design is all the mutual
exclusion is handled by the event dispatch mechanism (epoll or
kqueue) without burdening the user with extra locking. However, the
way the hijack works means we check the Rack env for the
'rack.hijack_io' key which is shared across requests and may
be cleared.
Ideally, this would not be a problem if the Rack dispatch allowed
returning a special value (e.g. ":ignore") instead of the normal
status-headers-body array, much like what the non-standard
"async.callback" API Thin started.
We could also avoid this problem by disallowing our "unhijack-ing"
of the socket but at a significant cost of crippling code
reusability, including that of existing middleware.
Thus, we allocate a new, empty request object here to avoid a TOCTTOU
in the following timeline:
original thread: | another thread
HttpClient#yahns_step |
r = k.app.call(env = @hs.env) # socket hijacked into epoll queue
<thread is scheduled away> | epoll_wait readiness
| ReqRes#yahns_step
| proxy dispatch ...
| proxy_busy_mod_done
************************** DANGER BELOW ********************************
| HttpClient#yahns_step
| # clears env
# sees empty env: |
return :ignore if env.include?('rack.hijack_io') |
In other words, we cannot ever touch the original env seen by the
original thread since it must see the 'rack.hijack_io' value because
both are operating in the same Yahns::HttpClient object. This will
happen regardless of GVL existence.
Avoiding errors like this is absolutely critical to every one-shot
based design.
|
|
Upstreams may shut us down while we're writing a request body,
attempt to forward any responses from the upstream back to the
client which may explain the rejection reason for giant uploads.
|
|
This giant method needs to spew an error response in case uploads
to the upstream fail, ensure the local variable is defined early.
|
|
Not all backends are capable of generating chunked responses
(especially not to HTTP/1.0 clients) nor can they generate
the Content-Length (especially when gzipping), so they'll
close the socket to signal EOF instead.
|
|
kgio_writev returns nil on success instead of the number of bytes
written, so we must manually calculate the number of bytes written
intead :x
This is triggerable when sending giant chunked responses.
|
|
A dumb string comparison will do here, so there's no point
in paying the memory and CPU cost of a regexp match when we
already extracted the suffix from a header key.
|
|
hijack seems incompatible with many middlewares, so return a
wonky response tuplet just in case...
|
|
Instance variable attr methods are cheaper and we can shove the
complexity down to tmpio by allowing it to accept a nil argument
for the temporary directory.
This adds 4 bytes to tmpio but removes over 1K in http_context
on a 32-bit system.
|
|
Literal regexps costs over 400 bytes of memory on x86-64 per-site,
so there's no point in using them to cause bloat at cold call sites
where runtime performance does not matter.
|
|
We cannot pass trailers from upstreams to HTTP/1.0 clients without
fully-buffering the response body AND trailers before forwarding the
response header.
Of course, one of the reasons yahns exists is to support lazy
buffering, so fully-buffering up front is wasteful and hurts
latency. So instead, degrade to 1.0 requests to upstreams for
HTTP/1.0 clients, this should prevent upstreams from sending
trailers in the first place.
HTTP/1.0 clients on Rails apps may suffer, but there probably
are not too many HTTP/1.0 clients out there.
|
|
Rack apps may (through a round-about way) send HTTP trailers
to HTTP/1.1 clients, and we need a way to forward those responses
through without losing the trailers.
|
|
We need to ensure more uncommon cases such as gigantic upstream
headers and truncated upstream responses are handled properly
and predictably.
|
|
We were incorrectly stashing the return value of detach_rbuf!
into the inter-thread buffer buffer which is bound to the client.
|
|
This allows our reverse proxy to avoid having an innefficient 1:1
relationship between threads and upstream connections, reducing
memory usage when there are many upstream connections (possibly to
multiple backend machines).
|
|
This allows us to write chunked response bodies without extra
copying to clients which support streaming.
|