| Date | Commit message (Collapse) |
|---|
|
Non-regular files are no longer reopened on SIGUSR1. This
allows users to specify FIFOs as log destinations.
TCP_NOPUSH/TCP_CORK is no longer set/unset by default. Use
:tcp_nopush explicitly with the "listen" directive if you wish
to enable TCP_NOPUSH/TCP_CORK.
Listen sockets are now bound _after_ loading the application for
preload_app(true) users. This prevents load balancers from
sending traffic to an application server while the application
is still loading.
There are also minor test suite cleanups.
|
|
If a user specifies a non-regular file for stderr_path or
stdout_path, we should not attempt to reopen or chown
it. This should also allow users to specify FIFOs as log
destinations.
|
|
On a certain FreeBSD 8.1 installation, explicitly setting
TCP_NOPUSH to zero (off) can cause EADDRNOTAVAIL errors and also
resets the listen backlog to 5. Enabling TCP_NOPUSH explicitly
did not exhibit this issue for the user who (privately) reported
this issue.
To be on the safe side, we won't set/unset TCP_NOPUSH/TCP_CORK
at all, which will leave it off on all current systems.
|
|
In the case where preload_app is true, delay binding new
listeners until after loading the application.
Some applications have very long load times (especially Rails
apps with Ruby 1.9.2). Binding listeners early may cause a load
balancer to incorrectly believe the unicorn workers are ready to
serve traffic even while the app is being loaded.
Once a listener is bound, connect() requests from the load
balancer succeed until the listen backlog is filled. This
allows requests to pile up for a bit (depending on backlog size)
before getting rejected by the kernel. By the time the
application is loaded and ready-to-run, requests in the
listen backlog are likely stale and not useful to process.
Processes inheriting listeners do not suffer this effect, as the
old process should still be capable of serving new requests.
This change does not improve the situation for the
preload_app=false (default) use case. There may not be a
solution for preload_app=false users using large applications.
Fortunately Ruby 1.9.3+ improves load times of large
applications significantly over 1.9.2 so this should be less of
a problem in the future.
Reported via private email sent on 2012-06-29T22:59:10Z
|
|
* Call shutdown(2) if a client EOFs on us during upload.
We can avoid holding a socket open if the Rack app forked a
process during uploads.
* ignore potential Errno::ENOTCONN errors (from shutdown(2)).
Even on LANs, connections can occasionally be accept()-ed but
be unusable afterwards.
Thanks to Joel Nimety <jnimety@continuity.net>,
Matt Smith <matt@nearapogee.com> and George <lists@southernohio.net>
on the mongrel-unicorn@rubyforge.org mailing list for their
feedback and testing for this release.
|
|
In case the Rack app forks before a client upload is complete,
shutdown(2) the socket to ensure the client isn't attempting to
read from us (even if it explicitly stopped writes).
|
|
Since there's nothing unicorn can do to avoid this error
on unconnected/halfway-connected clients, ignoring ENOTCONN
is a safe bet.
Rainbows! has long had this rescue as it called getpeername(2)
on untrusted sockets
|
|
* PATH_INFO (aka REQUEST_PATH) increased to 4096 (from 1024).
This allows requests with longer path components and matches
the system PATH_MAX value common to GNU/Linux systems for
serving filesystem components with long names.
* Apps that fork() (but do not exec()) internally for background
tasks now indicate the end-of-request immediately after
writing the Rack response.
Thanks to Hongli Lai, Lawrence Pit, Patrick Wenger and Nuo Yan
for their valuable feedback for this release.
|
|
Previously we relied on implicit socket shutdown() from the
close() syscall. However, some Rack applications fork()
(without calling exec()), creating a potentially long-lived
reference to the underlying socket in a child process. This
ends up causing nginx to wait on the socket shutdown when the
child process exits.
Calling shutdown() explicitly signals nginx (or whatever client)
that the unicorn worker is done with the socket, regardless of
the number of FD references to the underlying socket in
existence.
This was not an issue for applications which exec() since
FD_CLOEXEC is always set on the client socket.
Thanks to Patrick Wenger for discovering this. Thanks to
Hongli Lai for the tip on using shutdown() as is done in
Passenger.
ref: http://mid.gmane.org/CAOG6bOTseAPbjU5LYchODqjdF3-Ez4+M8jo-D_D2Wq0jkdc4Rw@mail.gmail.com
|
|
Found via rdoc-spellcheck
|
|
* Stale pid files are detected if a pid is recycled by processes
belonging to another user, thanks to Graham Bleach.
* nginx example config updates thanks to to Eike Herzbach.
* KNOWN_ISSUES now documents issues with apps/libs that install
conflicting signal handlers.
|
|
In some cases, EPERM may indicate a real configuration problem,
but it can also just mean the pid file is stale.
|
|
If unicorn doesn't get terminated cleanly (for example if the machine
has its power interrupted) and the pid in the pidfile gets used by
another process, the current unicorn code will exit and not start a
server. This tiny patch fixes that behaviour.
Acked-by: Eric Wong <normalperson@yhbt.net>
|
|
The GPLv3 is now an option to the Unicorn license. The existing GPLv2
and Ruby-only terms will always remain options, but the GPLv3 is
preferred.
Daemonization is correctly detected on all terminals for development
use (Brian P O'Rourke).
Unicorn::OobGC respects applications that disable GC entirely
during application dispatch (Yuichi Tateno).
Many test fixes for OpenBSD, which may help other *BSDs, too.
(Jeremy Evans).
There is now _optional_ SSL support (via the "kgio-monkey"
RubyGem). On fast, secure LANs, SSL is only intended for
detecting data corruption that weak TCP checksums cannot detect.
Our SSL support is remains unaudited by security experts.
There are also some minor bugfixes and documentation
improvements.
Ruby 2.0.0dev also has a copy-on-write friendly GC which can save memory
when combined with "preload_app true", so if you're in the mood, start
testing Unicorn with the latest Ruby!
|
|
No need to duplicate logic here
|
|
It's possible for a SIGUSR1 signal to be received in the
worker immediately before calling IO.select. In that case,
do not clutter logging with IOError and just process the
reopen log request.
|
|
Oops :x
|
|
Even LANs can break or be unreliable sometimes and socket
disconnect messages get lost, which means we fall back to
the global (kill -9) timeout in Unicorn.
While the default global timeout is much shorter (60s) than
typical TCP timeouts, some HTTP application dispatches take much
I/O or computational time (streaming many gigabytes), so the
global timeout becomes ineffective.
Under Linux, sysadmins are encouraged to lower the default
net.ipv4.tcp_keepalive_* knobs in sysctl. There should be
similar knobs in other operating systems (the default keepalive
intervals are usually ridiculously high, too high for anything).
When the listen socket has SO_KEEPALIVE set, the flag should be
inherited by accept()-ed sockets.
|
|
We favor low latency and consistency with the Unix socket
behavior even with TCP.
|
|
There's no practical difference between a timeout of 30 days and
68 years from an HTTP server standpoint.
POSIX limits us to 31 days, actually, but there could be
rounding error with floats used in Ruby time calculations and
there's no real difference between 30 and 31 days, either...
Thanks to Jeremy Evans for pointing out large values will throw
EINVAL (on select(2) under OpenBSD with Ruby 1.9.3 and
RangeError on older Rubies.
|
|
[ew: we need to explicitly enable GC if it is disabled
and respect applications that disable GC]
Acked-by: Eric Wong <normalperson@yhbt.net>
|
|
This will also be the foundation of SSL support in Rainbows!
and Zbatery. Some users may also want to use this in
Unicorn on LANs to meet certain security/auditing requirements.
Of course, Nightmare! (in whatever form) should also be able to
use it.
|
|
This prevents the stopping of all workers by SIGWINCH if you're
using a windowing system that will 'exec' unicorn from a process
that's already in a process group.
Acked-by: Eric Wong <normalperson@yhbt.net>
|
|
The old comment was confusing. We only zero the tick counter
when forking because application loading can take a long time.
Otherwise, it's always updated.
ref: http://mid.gmane.org/20110908191352.GA25251@dcvr.yhbt.net
|
|
There is no need to keep extra hashes or Proc objects around in
the heap.
|
|
Existing license terms (Ruby-specific) and GPLv2 remain
in place, but GPLv3 is preferred as it helps with
distribution of AGPLv3 code and is explicitly compatible
with Apache License (v2.0).
Many more reasons are documented by the FSF:
https://www.gnu.org/licenses/quick-guide-gplv3.html
http://gplv3.fsf.org/rms-why.html
ref: http://thread.gmane.org/gmane.comp.lang.ruby.unicorn.general/933
|
|
The last-resort timeout mechanism was inaccurate and often
delayed in activation since the 2.0.0 release. It is now fixed
and remains power-efficient in idle situations, especially with
the wakeup reduction in MRI 1.9.3+.
There is also a new document on application timeouts
intended to discourage the reliance on this last-resort
mechanism. It is visible on the web at:
http://unicorn.bogomips.org/Application_Timeouts.html
|
|
I've noticed in stderr logs from some folks that (last resort)
timeouts from the master process are taking too long to activate
due to the workarounds for suspend/hibernation.
|
|
* Rack::Chunked and Rack::ContentLength middlewares are loaded
by default for RACK_ENV=(development|deployment) users to match
Rack::Server behavior. As before, use RACK_ENV=none if you want
fine-grained control of your middleware. This should also
help users of Rainbows! and Zbatery.
* CTL characters are now rejected from HTTP header values
* Exception messages are now filtered for [:cntrl:] characters
since application/middleware authors may forget to do so
* Workers will now terminate properly if a SIGQUIT/SIGTERM/SIGINT
is received while during worker process initialization.
* close-on-exec is explicitly disabled to future-proof against
Ruby 2.0 changes [ruby-core:38140]
|
|
The signal handler from the master is still active and will
push the pending signal to SIG_QUEUE if a worker receives
a signal immediately after forking.
|
|
This is needed to match the behavior of Rack::Server for
RACK_ENV=(deployment|development), actually. This won't
affect users of other RACK_ENV values.
This change has minor performance consequences, so users
negatively affected should set RACK_ENV to "none" instead for
full control of their middleware stack.
This mainly affects Rainbows!/Zbatery users since they have
persistent connections and /need/ Content-Length or
Transfer-Encoding:chunked headers.
|
|
We do not want to affect terminals of users who view our log
files.
|
|
We only need the fileno in the key which we use
to generate the UNICORN_FD env. Otherwise the IO
object is accepted and understood by Ruby.
|
|
Setting the close-on-exec flag by default and closing
non-standard descriptors is proposed for Ruby 1.9.4/2.0.0.
Since Unicorn is one of the few apps to rely on FD inheritance
across exec(), we need to workaround this by redirecting each
listener FD to itself for Kernel#exec.
Ruby supports a hash as the final argument to Kernel#exec since
at least 1.9.1 (nobody cares for 1.9.0 anymore). This allows
users to backport close-on-exec by default patches to older
1.9.x installs without breaking anything.
ref: http://redmine.ruby-lang.org/issues/5041
|
|
This helps close a race condition preventing shutdown if
loading the application (preload_app=false) takes a long
time and the user decides to kil workers instead.
|
|
Future versions of Ruby may change this from the default *nix
behavior, so we need to explicitly allow FD passing via exec().
ref: http://redmine.ruby-lang.org/issues/5041
|
|
I corrupted a Ruby build and SOL_TCP didn't get defined :x
|
|
This release fixes things for users of per-worker "listen"
directives in the after_fork hook. Thanks to ghazel@gmail.com
for reporting the bug.
The "timeout" configurator directive is now truncated to
0x7ffffffe seconds to prevent overflow when calling
IO.select.
|
|
Nobody will miss one second if they specify an "infinite"
timeout of ~68 years. This prevents duplicating this logic
in Rainbows!
|
|
The testcase for this was broken, too, so we didn't notice
this :<
Reported-by: ghazel@gmail.com on the Rainbows! mailing list,
http://mid.gmane.org/BANLkTi=oQXK5Casq9SuGD3edeUrDPvRm3A@mail.gmail.com
|
|
IO.select in Ruby can't wait longer than this. This
means Unicorn can't support applications that take
longer than 68 years to respond :(
|
|
A single Unicorn instance may manage more than 1024 workers
without needing privileges to modify resource limits. As a
result of this, the "raindrops"[1] gem/library is now a required
dependency.
TCP socket defaults now favor low latency to mimic UNIX domain
socket behavior (tcp_nodelay: true, tcp_nopush: false). This
hurts throughput, users who want to favor throughput should
specify "tcp_nodelay: false, tcp_nopush: true" in the listen
directive.
Error logging is more consistent and all lines should be
formatted correctly in backtraces. This may break the
behavior of some log parsers.
The call stack is smaller and thus easier to examine backtraces
when debugging Rack applications.
There are some internal API changes and cleanups, but none that
affect applications designed for Rack. See "git log v3.7.0.."
for details.
For users who cannot install kgio[2] or raindrops, Unicorn 1.1.x
remains supported indefinitely. Unicorn 3.x will remain
supported if there is demand. We expect raindrops to introduce
fewer portability problems than kgio did, however.
[1] http://raindrops.bogomips.org/
[2] http://bogomips.org/kgio/
|
|
It's still O(n) since we don't maintain a reverse mapping of
spawned processes, but at least we avoid the extra overhead of
creating an array every time.
|
|
Some applications/libraries may launch background threads which
can lock up the process. So we can't disable heartbeat checking
just because the main thread is sleeping. This also has the
side effect of reducing master process wakeups when all workers
are idle.
|
|
This matches the latest Rack behavior.
We can't just use Rack::Builder.parse_file because our option
parser logic is slightly different and incompatible.
ref: rack commit d31cf2b7c0c77c04510c08d95776315ceb24ba54
|
|
We don't want the Worker#tick= assignment to trigger after we
accept a client, since we'd drop that request when we raise the
exception that breaks us out of the worker loop.
Also, we don't want to enter IO.select with an empty LISTENERS
array so we can fail with IOError or Errno::EBADF.
|
|
A leftover from the fchmod() days
|
|
Backtraces are now formatted properly (with timestamps) and
exceptions will be logged more consistently and similar to
Logger defaults:
"#{exc.message} (#{e.class})"
backtrace.each { |line| ... }
This may break some existing monitoring scripts, but errors
will be more standardized and easier to check moving forward.
|
|
"app error" is more correct, and consistent with Rainbows!
|
|
rescuing from SystemExit and exit()-ing again is ugly, but
changes made to lower stack depth positively affect _everyone_
so we'll tolerate some ugliness here.
We'll need to disable graceful exit for some tests, too...
|