Date | Commit message (Collapse) |
|
This renables the ability for Ruby 1.8.6 environments to perform reexecs
[ew: clarified this is for 1.8.6,
favor literal {} over Hash.new,
tweaked LISTENERS.map => LISTENERS.each, thanks to Hleb Valoshka
]
Signed-off-by: Eric Wong <normalperson@yhbt.net>
|
|
In case we (and Linux) supports other values in the future,
we can update it then. Until now, ensure users only set
true or false for this option.
|
|
There is currently no GPLv4, so this change has no effect at the
moment.
In case the GPLv4 arrives and I am not alive to approve/review it,
the lesser of evils is have give blanket approval of all future GPL
versions (as published by the FSF). The worse evil is to be stuck
with a license which cannot guarantee the Free-ness of this project
in the future.
This unfortunately means the FSF can theoretically come out with
license terms I do not agree with, but the GPLv2 and GPLv3 will
always be an option to all users.
|
|
Thanks to Hongli Lai for noticing my typo. While we're at it, finish up
a halfway-written comment for the EXDEV case
|
|
Ruby 1.9 and later includes IO#autoclose=, so we can use it
and prevent some dead IO objects from hanging around.
|
|
This allows users to start an independent instance of unicorn on
a the same port as a running unicorn (as long as both instances
use :reuseport).
ref: https://lwn.net/Articles/542629/
|
|
This will preserve mtime on successful renames for comparisions.
While we're at it, avoid writing the new PID until the listeners are
inherited successfully. This can be useful to avoid accidentally
clobbering a good PID if binding the listener or building the app
(preload_app==true) fails
|
|
In multithreaded apps, we must use dup2/dup3 with a temporary
descriptor to reopen log files atomically. This is the only way
to protect all concurrent userspace access to a file when reopening.
ref: http://bugs.ruby-lang.org/issues/9036
ref: yahns commit bcb10abe53cfb1d6a8ef7daef59eb10ced397c8a
|
|
We do not attempt to write HTTP responses for socket errors if
clients disconnect from us unexpectedly.
Additionally, we do not hide backtraces EINVAL/EBADF errors, since
they are indicative of real bugs which must be fixed.
We do continue to hide hide EOF, ECONNRESET, ENOTCONN, and EPIPE
because clients (even "friendly") ones will break connections due to
client crashes or network failure (which is common for me :P), and
the backtraces from those will cause excessive logging and even
become a DoS vector.
|
|
This fixes the -N (a.k.a. --no-defaut-middleware) option, which
was not working. The problem was that Unicorn::Configurator::RACKUP
is cleared before the lambda returned by Unicorn.builder is run,
which means that checking whether the :no_default_middleware option
was set from the lambda could not detect anything. This patch copies
it to a local variable that won't get clobbered, restoring the feature.
[ew: squashed test commit into the fix, whitespace fixes]
Signed-off-by: Eric Wong <normalperson@yhbt.net>
|
|
The PrivateTmp feature of systemd breaks the usage of /tmp for the
shared Unix domain socket between nginx and unicorn, so discourage the
use of /tmp in that case.
While we're at it, use consistent paths for everything and use an
obviously intended-for-user-customization "/path/to" prefix instead
of "/tmp"
ML-Ref: CAKLVLx_t+9zWMhquMWDfStrxS7xrNoGmN0ZDsjSCUE=VxU+oyQ@mail.gmail.com
Reported-by: David Wilkins <dwilkins@conecuh.com>
|
|
As far as I can tell, this was never necessary.
|
|
commit a9474624a148fe58e0944664190b259787dcf51e in rack.git
|
|
This DRYs out our code and prevents snafus like the 4.6.0
release where UNICORN_VERSION stayed at 4.5.0
Reported-by: Maurizio De Santis <m.desantis@morganspa.com>
|
|
* hijack:
ignore normal Rack response at request-time hijack
support for Rack hijack in request and response
|
|
This would prevent Unicorn from adding default middleware,
as if RACK_ENV were always none. (not development nor deployment)
This should also be applied to `rainbows' and `zbatery' as well.
One of the reasons to add this is to avoid conflicting
RAILS_ENV and RACK_ENV. It would be helpful in the case
where a Rails application and Rack application are composed
together, while we want Rails app runs under development
and Rack app runs under none (if we don't want those default
middleware), and we don't really want to make RAILS_ENV
set to development and RACK_ENV to none because it might be
confusing. Note that Rails would also look into RACK_ENV.
Another reason for this is that only `rackup' would be
inserting those default middleware. Both `thin' and `puma'
would not do this, nor does Rack::Handler.get.run which is
used in Sinatra.
So using this option would make it work differently from
`rackup' but somehow more similar to `thin' or `puma'.
Discussion thread on the mailing list:
http://rubyforge.org/pipermail/mongrel-unicorn/2013-January/001675.html
Signed-off-by: Eric Wong <normalperson@yhbt.net>
|
|
Once a connection is hijacked, we ignore it completely and leave
the connection at the mercy of the application.
|
|
Rack 1.5.0 (protocol version [1,2]) adds support for
hijacking the client socket (removing it from the control
of unicorn (or any other Rack webserver)).
Tested with rack 1.5.0.
|
|
The new check_client_connection option allows unicorn to detect
most disconnected local clients before potentially expensive
application processing begins.
This feature is useful for applications experiencing spikes of
traffic leading to undesirable queue times, as clients will
disconnect (and perhaps even retry, compounding the problem)
before unicorn can even start processing the request.
To enable this feature, add the following line to a unicorn
config file:
check_client_connection true
This feature only works when nginx (or any other HTTP/1.0+
client) is on the same machine as unicorn.
A huge thanks to Tom Burns for implementing and testing this
change in production with real traffic (including mitigating
an unexpected DoS attack).
ref: http://mid.gmane.org/CAK4qKG3rkfVYLyeqEqQyuNEh_nZ8yw0X_cwTxJfJ+TOU+y8F+w@mail.gmail.com
This release fixes broken Rainbows! compatibility in 4.5.0pre1.
|
|
Rainbows! relies on the ERROR_XXX_RESPONSE constants of unicorn
4.x. Changing the constants in unicorn 4.x will break existing
versions of Rainbows!, so remove the dependency on the constants
and generate the error response dynamically.
Unlike Mongrel, unicorn is unlikely to see malicious traffic and
thus unlikely to benefit from making error messages constant.
For unicorn 5.x, we will drop these constants entirely.
(Rainbows! most likely cannot support check_client_connection
consistently across all concurrency models since some of them
pessimistically buffer all writes in userspace. However, the
extra concurrency of Rainbows! makes it less likely to be
overloaded than unicorn, so this feature is likely less useful
for Rainbows!)
|
|
The new check_client_connection option allows unicorn to detect
most disconnected clients before potentially expensive
application processing begins.
This feature is useful for applications experiencing spikes of
traffic leading to undesirable queue times, as clients will
disconnect (and perhaps even retry, compounding the problem)
before unicorn can even start processing the request.
To enable this feature, add the following line to a unicorn
config file:
check_client_connection true
A huge thanks to Tom Burns for implementing and testing this
change in production with real traffic (including mitigating
an unexpected DoS attack).
|
|
In my testing, only dropped clients over Unix domain sockets or
loopback TCP were detected with this option. Since many
nginx+unicorn combinations run on the same host, this is not a
problem.
Furthermore, tcp_nodelay:true appears to work over loopback,
so remove the requirement for tcp_nodelay:false.
|
|
This patch checks incoming connections and avoids calling the application
if the connection has been closed.
It works by sending the beginning of the HTTP response before calling
the application to see if the socket can successfully be written to.
By enabling this feature users can avoid wasting application rendering
time only to find the connection is closed when attempting to write, and
throwing out the result.
When a client disconnects while being queued or processed, Nginx will log
HTTP response 499 but the application will log a 200.
Enabling this feature will minimize the time window during which the problem
can arise.
The feature is disabled by default and can be enabled by adding
'check_client_connection true' to the unicorn config.
[ew: After testing this change, Tom Burns wrote:
So we just finished the US Black Friday / Cyber Monday weekend running
unicorn forked with the last version of the patch I had sent you. It
worked splendidly and helped us handle huge flash sales without
increased response time over the weekend.
Whereas in previous flash traffic scenarios we would see the number of
HTTP 499 responses grow past the number of real HTTP 200 responses,
over the weekend we saw no growth in 499s during flash sales.
Unexpectedly the patch also helped us ward off a DoS attack where the
attackers were disconnecting immediately after making a request.
ref: <CAK4qKG3rkfVYLyeqEqQyuNEh_nZ8yw0X_cwTxJfJ+TOU+y8F+w@mail.gmail.com>
]
Signed-off-by: Eric Wong <normalperson@yhbt.net>
|
|
Non-regular files are no longer reopened on SIGUSR1. This
allows users to specify FIFOs as log destinations.
TCP_NOPUSH/TCP_CORK is no longer set/unset by default. Use
:tcp_nopush explicitly with the "listen" directive if you wish
to enable TCP_NOPUSH/TCP_CORK.
Listen sockets are now bound _after_ loading the application for
preload_app(true) users. This prevents load balancers from
sending traffic to an application server while the application
is still loading.
There are also minor test suite cleanups.
|
|
If a user specifies a non-regular file for stderr_path or
stdout_path, we should not attempt to reopen or chown
it. This should also allow users to specify FIFOs as log
destinations.
|
|
On a certain FreeBSD 8.1 installation, explicitly setting
TCP_NOPUSH to zero (off) can cause EADDRNOTAVAIL errors and also
resets the listen backlog to 5. Enabling TCP_NOPUSH explicitly
did not exhibit this issue for the user who (privately) reported
this issue.
To be on the safe side, we won't set/unset TCP_NOPUSH/TCP_CORK
at all, which will leave it off on all current systems.
|
|
In the case where preload_app is true, delay binding new
listeners until after loading the application.
Some applications have very long load times (especially Rails
apps with Ruby 1.9.2). Binding listeners early may cause a load
balancer to incorrectly believe the unicorn workers are ready to
serve traffic even while the app is being loaded.
Once a listener is bound, connect() requests from the load
balancer succeed until the listen backlog is filled. This
allows requests to pile up for a bit (depending on backlog size)
before getting rejected by the kernel. By the time the
application is loaded and ready-to-run, requests in the
listen backlog are likely stale and not useful to process.
Processes inheriting listeners do not suffer this effect, as the
old process should still be capable of serving new requests.
This change does not improve the situation for the
preload_app=false (default) use case. There may not be a
solution for preload_app=false users using large applications.
Fortunately Ruby 1.9.3+ improves load times of large
applications significantly over 1.9.2 so this should be less of
a problem in the future.
Reported via private email sent on 2012-06-29T22:59:10Z
|
|
* Call shutdown(2) if a client EOFs on us during upload.
We can avoid holding a socket open if the Rack app forked a
process during uploads.
* ignore potential Errno::ENOTCONN errors (from shutdown(2)).
Even on LANs, connections can occasionally be accept()-ed but
be unusable afterwards.
Thanks to Joel Nimety <jnimety@continuity.net>,
Matt Smith <matt@nearapogee.com> and George <lists@southernohio.net>
on the mongrel-unicorn@rubyforge.org mailing list for their
feedback and testing for this release.
|
|
In case the Rack app forks before a client upload is complete,
shutdown(2) the socket to ensure the client isn't attempting to
read from us (even if it explicitly stopped writes).
|
|
Since there's nothing unicorn can do to avoid this error
on unconnected/halfway-connected clients, ignoring ENOTCONN
is a safe bet.
Rainbows! has long had this rescue as it called getpeername(2)
on untrusted sockets
|
|
* PATH_INFO (aka REQUEST_PATH) increased to 4096 (from 1024).
This allows requests with longer path components and matches
the system PATH_MAX value common to GNU/Linux systems for
serving filesystem components with long names.
* Apps that fork() (but do not exec()) internally for background
tasks now indicate the end-of-request immediately after
writing the Rack response.
Thanks to Hongli Lai, Lawrence Pit, Patrick Wenger and Nuo Yan
for their valuable feedback for this release.
|
|
Previously we relied on implicit socket shutdown() from the
close() syscall. However, some Rack applications fork()
(without calling exec()), creating a potentially long-lived
reference to the underlying socket in a child process. This
ends up causing nginx to wait on the socket shutdown when the
child process exits.
Calling shutdown() explicitly signals nginx (or whatever client)
that the unicorn worker is done with the socket, regardless of
the number of FD references to the underlying socket in
existence.
This was not an issue for applications which exec() since
FD_CLOEXEC is always set on the client socket.
Thanks to Patrick Wenger for discovering this. Thanks to
Hongli Lai for the tip on using shutdown() as is done in
Passenger.
ref: http://mid.gmane.org/CAOG6bOTseAPbjU5LYchODqjdF3-Ez4+M8jo-D_D2Wq0jkdc4Rw@mail.gmail.com
|
|
Found via rdoc-spellcheck
|
|
* Stale pid files are detected if a pid is recycled by processes
belonging to another user, thanks to Graham Bleach.
* nginx example config updates thanks to to Eike Herzbach.
* KNOWN_ISSUES now documents issues with apps/libs that install
conflicting signal handlers.
|
|
In some cases, EPERM may indicate a real configuration problem,
but it can also just mean the pid file is stale.
|
|
If unicorn doesn't get terminated cleanly (for example if the machine
has its power interrupted) and the pid in the pidfile gets used by
another process, the current unicorn code will exit and not start a
server. This tiny patch fixes that behaviour.
Acked-by: Eric Wong <normalperson@yhbt.net>
|
|
The GPLv3 is now an option to the Unicorn license. The existing GPLv2
and Ruby-only terms will always remain options, but the GPLv3 is
preferred.
Daemonization is correctly detected on all terminals for development
use (Brian P O'Rourke).
Unicorn::OobGC respects applications that disable GC entirely
during application dispatch (Yuichi Tateno).
Many test fixes for OpenBSD, which may help other *BSDs, too.
(Jeremy Evans).
There is now _optional_ SSL support (via the "kgio-monkey"
RubyGem). On fast, secure LANs, SSL is only intended for
detecting data corruption that weak TCP checksums cannot detect.
Our SSL support is remains unaudited by security experts.
There are also some minor bugfixes and documentation
improvements.
Ruby 2.0.0dev also has a copy-on-write friendly GC which can save memory
when combined with "preload_app true", so if you're in the mood, start
testing Unicorn with the latest Ruby!
|
|
No need to duplicate logic here
|
|
It's possible for a SIGUSR1 signal to be received in the
worker immediately before calling IO.select. In that case,
do not clutter logging with IOError and just process the
reopen log request.
|
|
Oops :x
|
|
Even LANs can break or be unreliable sometimes and socket
disconnect messages get lost, which means we fall back to
the global (kill -9) timeout in Unicorn.
While the default global timeout is much shorter (60s) than
typical TCP timeouts, some HTTP application dispatches take much
I/O or computational time (streaming many gigabytes), so the
global timeout becomes ineffective.
Under Linux, sysadmins are encouraged to lower the default
net.ipv4.tcp_keepalive_* knobs in sysctl. There should be
similar knobs in other operating systems (the default keepalive
intervals are usually ridiculously high, too high for anything).
When the listen socket has SO_KEEPALIVE set, the flag should be
inherited by accept()-ed sockets.
|
|
We favor low latency and consistency with the Unix socket
behavior even with TCP.
|
|
There's no practical difference between a timeout of 30 days and
68 years from an HTTP server standpoint.
POSIX limits us to 31 days, actually, but there could be
rounding error with floats used in Ruby time calculations and
there's no real difference between 30 and 31 days, either...
Thanks to Jeremy Evans for pointing out large values will throw
EINVAL (on select(2) under OpenBSD with Ruby 1.9.3 and
RangeError on older Rubies.
|
|
[ew: we need to explicitly enable GC if it is disabled
and respect applications that disable GC]
Acked-by: Eric Wong <normalperson@yhbt.net>
|
|
This will also be the foundation of SSL support in Rainbows!
and Zbatery. Some users may also want to use this in
Unicorn on LANs to meet certain security/auditing requirements.
Of course, Nightmare! (in whatever form) should also be able to
use it.
|
|
This prevents the stopping of all workers by SIGWINCH if you're
using a windowing system that will 'exec' unicorn from a process
that's already in a process group.
Acked-by: Eric Wong <normalperson@yhbt.net>
|
|
The old comment was confusing. We only zero the tick counter
when forking because application loading can take a long time.
Otherwise, it's always updated.
ref: http://mid.gmane.org/20110908191352.GA25251@dcvr.yhbt.net
|
|
There is no need to keep extra hashes or Proc objects around in
the heap.
|
|
Existing license terms (Ruby-specific) and GPLv2 remain
in place, but GPLv3 is preferred as it helps with
distribution of AGPLv3 code and is explicitly compatible
with Apache License (v2.0).
Many more reasons are documented by the FSF:
https://www.gnu.org/licenses/quick-guide-gplv3.html
http://gplv3.fsf.org/rms-why.html
ref: http://thread.gmane.org/gmane.comp.lang.ruby.unicorn.general/933
|
|
The last-resort timeout mechanism was inaccurate and often
delayed in activation since the 2.0.0 release. It is now fixed
and remains power-efficient in idle situations, especially with
the wakeup reduction in MRI 1.9.3+.
There is also a new document on application timeouts
intended to discourage the reliance on this last-resort
mechanism. It is visible on the web at:
http://unicorn.bogomips.org/Application_Timeouts.html
|