| Date | Commit message (Collapse) |
|---|
|
Once again Ruby seems ready to introduce more incompatibilities
and force busywork upon maintainers[1]. In order to avoid
incompatibilities in the future, I used a Perl script[2] to
prepend `frozen_string_literal: false' to every Ruby file.
Somebody interested will have to go through every Ruby source
file and enable frozen_string_literal once they've thoroughly
verified it's safe to do so.
[1] https://bugs.ruby-lang.org/issues/20205
[2] https://yhbt.net/add-fsl.git/74d7689/s/?b=add-fsl.perl
|
|
kgio is an extra download and shared object which costs users
bandwidth, disk space, startup time and memory. Ruby 2.3+
provides `Socket#accept_nonblock(exception: false)' support
in addition to `exception: false' support in IO#*_nonblock
methods from Ruby 2.1.
We no longer distinguish between TCPServer and UNIXServer as
separate classes internally; instead favoring the `Socket' class
of Ruby for both. This allows us to use `Socket#accept_nonblock'
and get a populated `Addrinfo' object off accept4(2)/accept(2)
without resorting to a getpeername(2) syscall (kgio avoided
getpeername(2) in the same way).
The downside is there's more Ruby-level argument passing and
stack usage on our end with HttpRequest#read_headers (formerly
HttpRequest#read). I chose this tradeoff since advancements in
Ruby itself can theoretically mitigate the cost of argument
passing, while syscalls are a high fixed cost given modern CPU
vulnerability mitigations.
Note: no benchmarks have been run since I don't have a suitable
system.
|
|
Rack::Chunked will be gone in Rack 3.1, so provide a
non-middleware fallback which takes advantage of IO#write
supporting multiple arguments in Ruby 2.5+.
We still need to support Ruby 2.4, at least, since Rack 3.0
does. So a new (GC-unfriendly) Unicorn::WriteSplat module now
exists for Ruby <= 2.4 users.
|
|
Ruby 2.0+ sets FD_CLOEXEC by default on all FDs.
|
|
bogomips.org is due to expire, soon, and I'm not willing to pay
extortionist fees to Ethos Capital/PIR/ICANN to keep a .org. So
it's at yhbt.net, for now, but it will change again to
whatever's affordable... Identity is overrated.
Tor users can use .onions and kick ICANN to the curb:
torsocks w3m http://unicorn.ou63pmih66umazou.onion/
torsocks git clone http://ou63pmih66umazou.onion/unicorn.git/
torsocks w3m http://ou63pmih66umazou.onion/unicorn-public/
While we're at it, `s/news.gmane.org/news.gmane.io/g', too.
(but I suspect that'll need to be resynched since our mail
"List-Id:" header is changing).
|
|
When running: ```
require 'kgio'
require 'raindrops'
F_SETPIPE_SZ = 1031 if RUBY_PLATFORM =~ /linux/
Kgio::Pipe.new.each do |io|
io.close_on_exec = true
if defined?(F_SETPIPE_SZ)
begin
puts "setting"
io.fcntl(F_SETPIPE_SZ, Raindrops::PAGE_SIZE)
rescue Errno::EINVAL
puts "rescued"
rescue => e
puts ["FAILED HARD", e].inspect
end
end
end
```
on a few servers to test some Unicorn boot failures I saw:
```
["FAILED HARD", #<Errno::EPERM: Operation not permitted>]
```
The `EPERM` error gets raised by the Linux kernel if:
```
(too_many_pipe_buffers_hard(pipe->user) ||
too_many_pipe_buffers_soft(pipe->user)) &&
!capable(CAP_SYS_RESOURCE) && !capable(CAP_SYS_ADMIN)
```
Given that the resize is not strictly necessary Unicorn should
rescue the error and continue booting.
|
|
This allows for the equivalent of the
-N/--no-default_middleware command line option to be
specified in the configuration file so it doesn't
need to be specified on the command line every time
unicorn is executed.
It explicitly excludes the use of -N/--no-default_middleware
as an embedded configuration option in the rackup file, by
ignoring the options after ARGV is parsed.
In order to allow the configuration method to work, have
the lambda that Unicorn.builder returns accept two arguments.
Technically, only one argument is needed for the HttpServer
instance, but I'm guessing if the lambda accepts a single
argument, we expect that to be a rack application instead
of a lambda that returns a rack application.
The command line option option to disable default middleware
will take precedence over the unicorn configuration file option
if both are present.
For backwards compatibility, if the lambda passed to
HttpServer accepts 0 arguments, then call it without
arguments.
[ew: fix precedence for arity checking in build_app!
configurator: ensure -N is respected when set in command-line]
|
|
We have never had any need for pipes with the default 64K
capacity on Linux. Our pipes are only used for tiny writes
in signal handlers and to perform parent shutdown detection.
With the current /proc/sys/fs/pipe-user-pages-soft
default, only 1024 pipes can be created by an unprivileged
user before Linux clamps down the pipe size to 4K (a single page)
for newly-created pipes[1].
So avoid penalizing OTHER pipe users who could benefit from the
increased capacity and use only a single page for ourselves.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/pipe.c?h=v4.18#n642
|
|
Slowly reducing dependencies on kgio. 'io/wait' is required by
'socket' these days, so it's no extra relocations for .so
loading, either.
|
|
While "unicorn -d" requires 'pp' when setting $DEBUG, we did not
account for (rare) Rack applications setting $DEBUG at load time.
Thanks-to: James P (Jim) Robinson Jr <James.Robinson3@Cigna.com>
|
|
bogomips.org is dropping prefixes to reduce subjectAltName bloat
in TLS certificates.
|
|
State explicitly applications should not rely on it, and instead
rescue the generic EOFError exception. This class will stick
around because there may inevitably be things which rely on it,
but we should not encourage it, either.
|
|
We do not want to pull in a newer or older version of rack depending
on an the application running under it requires. Furthermore, it
has always been possible to use unicorn without any middleware at
all.
Without rack, we'll be missing descriptive status text in the first
response line, but any valid HTTP/1.x parser should be able to
handle it properly.
ref:
http://bogomips.org/unicorn-public/20160121201255.GA6186@dcvr.yhbt.net/t/#u
Thanks-to: Adam Duke <adam.v.duke@gmail.com>
Thanks-to: Aaron Patterson <tenderlove@ruby-lang.org>
|
|
They'll continue to be maintained, but we're no longer advertising
them. Also, favor lowercase "unicorn" while we're at it since that
matches the executable and gem name to avoid unnecessary escaping
for RDoc.
|
|
rack 1.6 added a TempfileReaper middleware to cleanup temporary
files. Enable it by default for users running rack 1.6 or later
to avoid leaving temporary files around.
|
|
Empty classes do not need a heavy class definition scope.
|
|
require_relative appeared in Ruby 1.9.2 to speed up load times by
avoiding needless open() syscalls. This has no effect if you're using
RUBYLIB or the '-I' option when running ruby(1), but avoids searching
paths in other gems.
This does not affect unicorn greatly as unicorn does not activate many
gems, but still leads to reducing ~45 syscalls during startup.
|
|
IO#close_on_exec* methods are available since Ruby 1.9.1. It
allows us to use less bytecode as it requires fewer operands and
avoids constant lookups.
|
|
Signaling using normal kill(2) is preserved, but the master now
prefers to signal workers using a pipe rather than kill(2).
Non-graceful signals (:TERM/:KILL) are still sent using kill(2),
as they ask for immediate shutdown.
This change is necessary to avoid triggering the ubf (unblocking
function) for rb_thread_call_without_gvl (and similar) functions
extensions. Most notably, this fixes compatibility with newer
versions of the 'pg' gem which will cancel a running DB query if
signaled[1].
This also has the nice side-effect of allowing a premature
master death (assuming preload_app didn't cause the master to
spawn off rogue child daemons).
Note: users should also refrain from using "killall" if using the
'pg' gem or something like it.
Unfortunately, this increases FD usage in the master as the writable
end of the pipe is preserved in the master. This limit the number
of worker processes the master may run to the open file limit of the
master process. Increasing the open file limit of the master
process may be needed. However, the FD use on the workers is
reduced by one as the internal self-pipe is no longer used. Thus,
overall pipe allocation for the kernel remains unchanged.
[1] - pg is correct to cancel a query, as it cannot know if
the signal was for a) graceful unicorn shutdown or
b) oh-noes-I-started-a-bad-query-ABORT-ABORT-ABORT!!
|
|
This fixes the -N (a.k.a. --no-defaut-middleware) option, which
was not working. The problem was that Unicorn::Configurator::RACKUP
is cleared before the lambda returned by Unicorn.builder is run,
which means that checking whether the :no_default_middleware option
was set from the lambda could not detect anything. This patch copies
it to a local variable that won't get clobbered, restoring the feature.
[ew: squashed test commit into the fix, whitespace fixes]
Signed-off-by: Eric Wong <normalperson@yhbt.net>
|
|
This would prevent Unicorn from adding default middleware,
as if RACK_ENV were always none. (not development nor deployment)
This should also be applied to `rainbows' and `zbatery' as well.
One of the reasons to add this is to avoid conflicting
RAILS_ENV and RACK_ENV. It would be helpful in the case
where a Rails application and Rack application are composed
together, while we want Rails app runs under development
and Rack app runs under none (if we don't want those default
middleware), and we don't really want to make RAILS_ENV
set to development and RACK_ENV to none because it might be
confusing. Note that Rails would also look into RACK_ENV.
Another reason for this is that only `rackup' would be
inserting those default middleware. Both `thin' and `puma'
would not do this, nor does Rack::Handler.get.run which is
used in Sinatra.
So using this option would make it work differently from
`rackup' but somehow more similar to `thin' or `puma'.
Discussion thread on the mailing list:
http://rubyforge.org/pipermail/mongrel-unicorn/2013-January/001675.html
Signed-off-by: Eric Wong <normalperson@yhbt.net>
|
|
In the case where preload_app is true, delay binding new
listeners until after loading the application.
Some applications have very long load times (especially Rails
apps with Ruby 1.9.2). Binding listeners early may cause a load
balancer to incorrectly believe the unicorn workers are ready to
serve traffic even while the app is being loaded.
Once a listener is bound, connect() requests from the load
balancer succeed until the listen backlog is filled. This
allows requests to pile up for a bit (depending on backlog size)
before getting rejected by the kernel. By the time the
application is loaded and ready-to-run, requests in the
listen backlog are likely stale and not useful to process.
Processes inheriting listeners do not suffer this effect, as the
old process should still be capable of serving new requests.
This change does not improve the situation for the
preload_app=false (default) use case. There may not be a
solution for preload_app=false users using large applications.
Fortunately Ruby 1.9.3+ improves load times of large
applications significantly over 1.9.2 so this should be less of
a problem in the future.
Reported via private email sent on 2012-06-29T22:59:10Z
|
|
This is needed to match the behavior of Rack::Server for
RACK_ENV=(deployment|development), actually. This won't
affect users of other RACK_ENV values.
This change has minor performance consequences, so users
negatively affected should set RACK_ENV to "none" instead for
full control of their middleware stack.
This mainly affects Rainbows!/Zbatery users since they have
persistent connections and /need/ Content-Length or
Transfer-Encoding:chunked headers.
|
|
We do not want to affect terminals of users who view our log
files.
|
|
This matches the latest Rack behavior.
We can't just use Rack::Builder.parse_file because our option
parser logic is slightly different and incompatible.
ref: rack commit d31cf2b7c0c77c04510c08d95776315ceb24ba54
|
|
Backtraces are now formatted properly (with timestamps) and
exceptions will be logged more consistently and similar to
Logger defaults:
"#{exc.message} (#{e.class})"
backtrace.each { |line| ... }
This may break some existing monitoring scripts, but errors
will be more standardized and easier to check moving forward.
|
|
There's absolutely no need to keep the OptionParser around in
worker processes.
|
|
This reduces the size of `caller` by 5 frames,
which should make backtraces easier-to-read, raising
exceptions less expensive, and reduce GC runtime.
|
|
Don't clutter up our RDoc/website with things that users
of Unicorn don't need to see. This should make user-relevant
documentation easier to find, especially since Unicorn is
NOT intended to be an API.
|
|
We will eventually expose a Unicorn::StreamInput object as
"rack.input" for Rack 2.x applications. StreamInput allows
applications to avoid buffering input to disk, removing the
(potentially expensive) rewindability requirement of Rack 1.x.
TeeInput is also rewritten to build off StreamInput for
simplicity. The only regression is that TeeInput#rewind forces
us to consume an unconsumed stream before returning, a
negligible price to pay for decreased complexity.
|
|
This should be easier for Rainbows! to use
|
|
This also affects some constant scoping rules, but hopefully
makes things easier to follow. Accessing ivars (not via
accessor methods) are also slightly faster, so use them in
the criticial process_client code path.
|
|
Thanks to kgio, we no longer use accept_nonblock.
|
|
This is slightly shorter and hopefully easier to find.
|
|
This should hopefully make the non-blocking accept()
situation more tolerable under Ruby 1.9.2.
|
|
Rainbows! will be able to reuse this.
|
|
This hopefully makes things easier to read, follow, and find
since it's mostly documentation...
|
|
There's no need for a response class or object since Rack just
uses an array as the response. So use a procedural style which
allows for easier understanding.
We shall also support keepalive/pipelining in the future, too.
|
|
These are minor changes to remove unnecessary loop nesting and
begin usage to reduce our code size and hopefully simplify
flow for readers.
|
|
Something is wrong if workers exit with a non-zero status,
so we'll increase the log level to help prevent people
from missing it.
|
|
In addition to SIGHUP, it should be possible to gradually bring
workers back up (to avoid overloading the machine) when rolling
back upgrades after SIGWINCH.
Noticed-by: Lawrence Pit
ref: http://mid.gmane.org/4C3F8C9F.2090903@gmail.com
|
|
As described in our SIGNALS documentation, sending SIGHUP to the
old master (to respawn SIGWINCH-ed children) while the new
master (spawned from SIGUSR2) is active is useful for backing
out of an upgrade before sending SIGQUIT to the new master.
Unfortunately, the SIGHUP signal to the old master will cause
the ".oldbin" pid file to be reset to the non-".oldbin" version
and thus attempt to clobber the pid file in use by the
to-be-terminated new master process.
Thanks to the previous commit to prevent redaemonization in the
new master, the old master can reliably detect if the new master
is active while it is reloading the config file.
Thanks to Lawrence Pit for discovering this bug.
ref: http://mid.gmane.org/4C3BEACF.7040301@gmail.com
|
|
"stringio" is part of the Ruby distro and we use it in multiple
places, so avoid re-requiring it.
|
|
"[]" is slightly faster under Ruby 1.9 (but slightly
slower under 1.8).
|
|
This is fixed upstream in Rubinius by commit
b630ad9ddb4544a62e8e2282ba7dc59c4269bad7
|
|
While log reopening worked reliably for newly-created File
objects in the unit tests, the $stderr and $stdout handles that
get redirected did not get reopened reliably under Rubinius.
We work around this by relying on Rubinius internals and
directly setting the @path instance variable. This is harmless
for MRI and should be harmless for other any other Ruby
implementations we'll eventually support.
ref: http://github.com/evanphx/rubinius/issues/360
|
|
Since we added support for the "working_directory" parameter, it
often became unclear where/when certain paths would be bound.
There are some extremely nasty dependencies and ordering issues
when doing this. It's all pretty fragile, but works for now
and we even have a full integration test to keep it working.
I plan on cleaning this up 2.x.x to be less offensive to look
at (Rainbows! and Zbatery are a bit tied to this at the moment).
Thanks to Pierre Baillet for reporting this.
ref: http://mid.gmane.org/AANLkTimKb7JARr_69nfVrJLvMZH3Gvs1o_KwZFLKfuxy@mail.gmail.com
|
|
It can be useful to track down problems with
|
|
No need for extra checks when we're just doing read_nonblock.
We'll also avoid reallocating a 16K buffer every time we sleep,
we can just reuse the buffer the workers normally use to process
request.
|
|
Modern version of Unicorn have working_directory available and
should use that instead.
|