Date | Commit message (Collapse) |
|
* Use a frozen empty array and a class variable for TCP_Info to avoid
garbage. As far as I can tell, this shouldn't result in any garbage on
any requests (other than on the first request).
* Pass listener socket to #read to only check the client connection on
a TCP server.
* Short circuit CLOSE_WAIT after ESTABLISHED since in my testing it's
the most common state after ESTABLISHED, it makes the numbers
un-ordered, though. But comment should make it OK.
* Definition of of `check_client_connection` based on whether
Raindrops::TCP_Info is defined, instead of the class variable
approach.
* Changed the unit tests to pass a `nil` listener.
Tested on our staging environment, and still works like a dream.
I should note that I got the idea between this patch into Puma as well!
https://github.com/puma/puma/pull/1227
[ew: squashed in temporary change for oob_gc.rb, but we'll come
up with a different change to avoid breaking gctools
<https://github.com/tmm1/gctools>]
Acked-by: Eric Wong <e@80x24.org>
|
|
Killing the master process may lead to the worker dying on its
own (as designed); before kill(1) has had an opportunity to send
the second kill(2) syscall on the worker process.
Killing the worker before the master might also lead to a
needless respawn, so merely kill the master and let the worker
follow it in death.
This race condition occasionally caused test failures on slow,
uniprocessor hardware.
|
|
Fixes: 2af91a1fef70d654 ("Add after_worker_exit configuration option")
|
|
This option is executed in the master process following all
worker process exits. It is most useful in the case where
the worker process crashes the ruby interpreter, as the worker
process may not be able to send error notifications
appropriately.
For example, let's say you have a specific request that crashes a
worker process, which you expect to be due to a improperly
programmed C extension. By modifying your worker to save request
related data in a temporary file and using this option, you can get
a record of what request is crashing the application, which will
make debugging easier.
Example:
after_worker_exit do |server, worker, status|
server.logger.info "worker #{status.success? ? 'exit' : 'crash'}: #{status}"
file = "request.#{status.pid}.txt"
if File.exist?(file)
do_something_with(File.read(file)) unless status.success?
File.delete(file)
end
end
|
|
We'll be moving to direct ivar access to reduce the API (and
method entry) overhead of internal unicorn classes. This means
some tests like this one will reach deeper into internals.
This will be necessary for the upcoming changes to add new
configuration options to unicorn.
|
|
Oops, this was a half-baked change I was considering
but forgot about.
This reverts commit 69fd4f9bbff3708166fbf70163fa6e192dde1497.
|
|
This ensures we won't have duplicate objects in Ruby 2.0-2.4.
For Ruby 2.5.0dev+, this avoids any duplicate cleanup
introduced as of r57471: https://bugs.ruby-lang.org/issues/13085
|
|
|
|
IO#write already elides the write(2) syscall for empty buffers,
so there's no need to complicate our instruction sequence
footprint for the rare case of an empty buffer.
The only cases a Rack app will have an empty buffer are:
1) `env['rack.input'].read` without args
2) `env['rack.input'].gets`
Neither of these calls are safe for server-independent Rack apps
as the client can OOM the app.
unicorn itself provides no facility for limiting maximum
rack.input size. Instead, unicorn relies on nginx to limit
input size using the client_max_body_size directive.
|
|
This probably applies to other kernels, too, but I'm most
familiar with Linux.
|
|
While it is innocuous after compiling, it can be a confusing
source of errors for users with broken installations of Ruby
itself:
https://bogomips.org/unicorn-public/5ace6a20-e094-293d-93df-b557480e12d5@anyces.com/
https://bogomips.org/unicorn-public/02994a55-9c07-a3c5-f06b-a4c15551a67e@anyces.com/
rb_str_set_len has been provided since Ruby 1.8.7+, so we have
not needed it since we dropped all 1.8.x support in unicorn 5.x.
|
|
Most notably, this release allows us to support requests with
lines delimited by LF-only, as opposed to the standard CRLF
pair and allowed by RFC 2616 sec 19.3.
Thanks to Mishael A Sibiryakov for the explanation and change:
https://bogomips.org/unicorn-public/1476954332.1736.156.camel@junki.org/
Thanks to Let's Encrypt, the website also moves to HTTPS
<https://bogomips.org/unicorn/> to improve reader privacy. The
"unicorn.bogomips.org" subdomain will be retired soon to reduce
subjectAltName bloat and speed up certificate renewals.
There's also the usual round of documentation and example
updates, too.
Eric Wong (7):
examples/init.sh: update to reduce upgrade raciness
doc: systemd should only kill master in example
examples/logrotate.conf: update example for systemd
doc: update gmane URLs to point to our own archives
relocate website to https://bogomips.org/unicorn/
TODO: remove Rack 2.x item
build: "install-gem" target avoids network
Mishael A Sibiryakov (1):
Add some tolerance (RFC2616 sec. 19.3)
|
|
No need to go online when installing a locally-built gem.
|
|
Rack 2.x is less of a jump than initially expected,
and we've already supported it for a few releases, already.
|
|
* origin/website-move:
relocate website to https://bogomips.org/unicorn/
|
|
* origin/rfc2616-sec19.3:
Add some tolerance (RFC2616 sec. 19.3)
|
|
* origin/jr/init:
examples/init.sh: update to reduce upgrade raciness
|
|
HTTPS helps some with reader privacy and Let's Encrypt seems to
be working well enough the past few months.
This change will allow us to reduce subjectAltName bloat in our
TLS certificate over time. It will also promote domain name
agility to support mirrors or migrations to other domains
(including a Tor hidden service mirror).
http://bogomips.org/unicorn/ will remain available for people on
legacy systems without usable TLS. There is no plan for automatic
redirecting from HTTP to HTTPS at this time.
|
|
Hi all.
We're implementing client certificate authentication with nginx and
unicorn.
Nginx configured in the following way:
proxy_set_header X-SSL-Client-Cert $ssl_client_cert;
When client submits certificate and nginx passes it to the unicorn,
unicorn responds with 400 (Bad Request). This caused because nginx
doesn't use "\r\n" they using just "\n" and multilne headers is failed
to parse (I've added test).
Accorording to RFC2616 section 19.3:
https://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.3
"The line terminator for message-header fields is the sequence CRLF.
However, we recommend that applications, when parsing such headers,
recognize a single LF as a line terminator and ignore the leading CR."
CRLF changed to ("\r\n" | "\n")
Github commit
https://github.com/uno4ki/unicorn/commit/ed127b66e162aaf176de05720f6be758f8b41b1f
PS: Googling "nginx unicorn ssl_client_cert" shows the problem.
|
|
Gmane's NNTP server remains up, but the HTTP site is down:
https://lars.ingebrigtsen.no/2016/07/28/the-end-of-gmane/
Anyways, our own archives are designed to be mirror-able via git:
git clone --mirror https://bogomips.org/unicorn-public
And the code is self-hostable: git clone https://public-inbox.org
|
|
...And add placeholders for other systems
|
|
By default, systemd kills every process in the control group
when stopping a service. While it ought to be harmless to
signal workers, some Rack applications (and perhaps further
subprocesses) can misbehave when interrupted by a signal.
Ensure we only hit the master on graceful shutdown to avoid
tickling bugs in Rack apps.
This is the reason we switched to having the master send
"fake" signals for workers beginning with unicorn 4.8.0
back in 2013/2014.
|
|
Rework the "upgrade" target to only read the PID files once to
avoid misreading the wrong PID files in the middle of the
upgrade.
Additionally, introduce the UPGRADE_DELAY environment parameter
so users can increase/decrease according to their application
startup time.
PID files are inherently racy and people should be using a
process manager (systemd or similar) instead, but this should
mitigate most of the problems with the old target.
While we're at it, add LSB tags for systems which complain
about the lack of them and modernize things a bit using
$(command) construct instead of the more fragile `command`.
Thanks-to: Jesper Rønn-Jensen <jesperrr@gmail.com>
|
|
Note: no code changes since 5.1.0.pre1 from January.^WNo, wait,
last minute performance improvement added today. See below.
The big change is rack is not required (but still recommended).
Applications are expected to depend on rack on their own so they can
specify the version of rack they prefer without unicorn pulling
in a newer, potentially incompatible version.
unicorn will always attempt to work with multiple versions of rack
as practical.
The HTTP parser also switched to using the TypedData C-API for
extra type safety and memory usage accounting support in the
'objspace' extension.
Thanks to Adam Duke to bringing the rack change to our attention
and Aaron Patterson for helping with the matter.
Last minute change: we now support the new leftpad() syscall under
Linux for major performance and security improvement:
http://mid.gmane.org/1459463613-32473-1-git-send-email-richard@nod.at
8^H9 changes since 5.0.1:
http: TypedData C-API conversion
various documentation updates
doc: bump olddoc to ~> 1.2 for extra NNTP URL
rack is optional at runtime, required for dev
doc update for ClientShutdown exceptions class
unicorn 5.1.0.pre1 - rack is optional, again
doc: reference --keep-file-descriptors for "bundle exec"
doc: further trimming to reduce noise
use leftpad Linux syscall for speed!
|
|
It's not worth mentioning pre-Rack versions of Rails anymore,
and there are a few async Rack applications reliant on
EventMachine which we do not use.
Some uses of chunked request decoding are not well-handled
with nginx in front, anyways; so avoid mentioning them.
Additionally, avoid introducing new terms into the lexicon
and just refer to "mailing list" as a generic term.
|
|
"bundle exec" alone is not suitable for use with systemd-style
socket activation due to Ruby 2.0+ behavior of setting close-on-exec
for file descriptors above 2. However, the "--keep-file-descriptors"
option was added to bundler 1.4.0 to workaround this Ruby 2.0 change
and may be used to prevent Ruby 2.0+ from closing file descriptors
on exec.
Thanks to Amir Yalon and Christos Trochalakis for bringing up
this issue on the mailing list:
http://bogomips.org/unicorn-public/1457824748.3666627.547425122.2A828B07@webmail.messagingengine.com/
|
|
The big change is rack is not required (but still recommended).
Applications are expected to depend on rack on their own so they can
specify the version of rack they prefer without unicorn pulling
in a newer, potentially incompatible version.
unicorn will always attempt to work with multiple versions of rack
as practical.
The HTTP parser also switched to using the TypedData C-API for
extra type safety and memory usage accounting support in the
'objspace' extension.
Thanks to Adam Duke to bringing the rack change to our attention
and Aaron Patterson for helping with the matter.
There might be more documentation-related changes before 5.1.0
final. I am considering dropping pandoc from manpage generation
and relying on pod2man (from Perl) because it has a wider install
base.
5 changes since v5.0.1:
http: TypedData C-API conversion
various documentation updates
doc: bump olddoc to ~> 1.2 for extra NNTP URL
rack is optional at runtime, required for dev
doc update for ClientShutdown exceptions class
|
|
State explicitly applications should not rely on it, and instead
rescue the generic EOFError exception. This class will stick
around because there may inevitably be things which rely on it,
but we should not encourage it, either.
|
|
We do not want to pull in a newer or older version of rack depending
on an the application running under it requires. Furthermore, it
has always been possible to use unicorn without any middleware at
all.
Without rack, we'll be missing descriptive status text in the first
response line, but any valid HTTP/1.x parser should be able to
handle it properly.
ref:
http://bogomips.org/unicorn-public/20160121201255.GA6186@dcvr.yhbt.net/t/#u
Thanks-to: Adam Duke <adam.v.duke@gmail.com>
Thanks-to: Aaron Patterson <tenderlove@ruby-lang.org>
|
|
Additional advertising for the gmane NNTP server makes sense
from a robustness standpoint:
nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
Not advertising other HTTP-based URLs just yet. They could contain
images/frames/JS/CSS and add unnecessary clutter to the footer.
NNTP puts the client in control of UI.
|
|
* add nntp_url to the olddoc website footer
* update legacy support status for 4.x (not 4.8.x)
* update copyright range to 2016
* note all of our development tools are Free Software, too
* remove cgit mention; it may not always be cgit
(but URLs should remain compatible).
* discourage downloading snapshot tarballs;
"git clone" + periodic "git fetch" is more efficient
* remove most mentions of unicorn_rails as that
was meant for ancient Rails 1.x/2.x users
* update path reference to Ruby 2.3.0
* fix nginx upstream module link to avoid redirect
* shorten Message-ID example to avoid redirects
and inadvertant linkage
|
|
This provides some extra type safety if combined with other
C extensions, as well as allowing us to account for memory usage of
the HTTP parser in ObjectSpace.
This requires Ruby 1.9.3+ and has remained a stable API since
then. This will become officially supported when Ruby 2.3.0 is
released later this month.
This API has only been documented in doc/extension.rdoc (formerly
README.EXT) in the Ruby source tree since April 2015, r50318
|
|
Once again, we allow nil values in response headers. We've had
this bug since March 2009, and thus cannot expect existing
applications and middlewares running unicorn to fix this.
Unfortunately, supporting this bug contributes to application
server lock-in, but at least we'll document it as such.
Thanks to Owen Ou <o@heroku.com> for reporting this regression:
http://bogomips.org/unicorn-public/CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com/
Additionally, systemd examples are now in the examples/ directory
based on a post by Christos Trochalakis <yatiohi@ideopolis.gr>:
http://bogomips.org/unicorn-public/20150708130821.GA1361@luke.ws.skroutz.gr/
|
|
The "diff" function detection for C does not map well to
Ruby files, take advantage of gitattributes(5) to improve
method name detection in generated patches as well as
making "git diff -W" output more useful.
|
|
Since we have init scripts, we ought to have the equivalent for
systemd users who cannot upgrade via the normal SIGUSR2 mechanism;
but can use multiple services: "unicorn@1" + h"unicorn@2" to
accomplish the same thing.
Based on examples by Christos Trochalakis <yatiohi@ideopolis.gr>
ref:
http://bogomips.org/unicorn-public/20150708130821.GA1361@luke.ws.skroutz.gr/
|
|
This blatantly violates Rack SPEC, but we've had this bug since
March 2009[1]. Thus, we cannot expect all existing applications
and middlewares to fix this bug and will probably have to
support it forever.
Unfortunately, supporting this bug contributes to application
server lock-in, but at least we'll document it as such.
[1] commit 1835c9e2e12e6674b52dd80e4598cad9c4ea1e84
("HttpResponse: speed up non-multivalue headers")
Reported-by: Owen Ou <o@heroku.com>
Ref: <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
|
|
An evolutionary dead-end since its announcement[1] nearly six years
ago, this old-fashioned preforker has had enough bugs and missteps
that it's managed to hit version 5!
I wish I could say unicorn 5 is leaps and bounds better than 4, but
it is not. This major version change allows us to drop some cruft
and unused features which accumulated over the years, resulting in
several kilobytes of memory saved[2]!
Compatibility:
* The horrible, proprietary (:P) "Status:" response header is
finally gone, saving at least 16 precious bytes in every HTTP
response. This should make it easier to write custom HTTP clients
which are compatible across all HTTP servers. It will hopefully
make migrating between different Rack servers easier for new
projects.
* Ruby 1.8 support removed. Ruby 1.9.3 is currently the earliest
supported version. However, expect minor, likely-unnoticeable
performance regressions if you use Ruby 2.1 or earlier. Going
forward, unicorn will favor the latest version (currently 2.2) of
the mainline Ruby implementation, potentially sacrificing
performance on older Rubies.
* Some internal, undocumented features and APIs used by
derivative servers are gone; removing bloat and slightly lowering
memory use. We have never and will never endorse the use of any
applications or middleware with a dependency on unicorn,
applications should be written for Rack instead.
Note: Rainbows! 5.0 will be released next week or so to be
compatible with unicorn 5.x
New features:
* sd_listen_fds(3) emulation added for systemd compatibility.
You may now stop using PID files and other process monitoring
software when using systemd.
* Newly-set TCP socket options are now applied to inherited sockets.
* Dynamic changes in the application to Rack::Utils::HTTP_STATUS
hash is now supported; allowing users to set custom status lines
in Rack to be reflected in unicorn. This feature causes a minor
performance regression, but is made up for Ruby 2.2 users with
other optimizations.
* The monotonic clock is used under Ruby 2.1+, making the
timeout feature immune to system clock changes.
As unicorn may be used anonymously without registration, the
project is committed to supporting anonymous and pseudonymous
help requests, contributions and feedback via plain-text mail to:
unicorn-public@bogomips.org
The mail submission port (587) is open to those behind firewalls
and allows access via Tor and anonymous remailers.
Archives are accessible via: http://bogomips.org/unicorn-public/
and mirrored to various other places, so you do not need to use
a valid address when posting.
Finally, rest assured the core design of unicorn will never change.
It will never use new-fangled things like threads, kqueue or epoll;
but will always remain a preforking server serving one client
per-process.
[1] http://mid.gmane.org/20090211230457.GB22926@dcvr.yhbt.net
[2] this would've been like, totally gnarly in the 80s!
|
|
We cannot rely on users reading release notes.
|
|
ISSUES: note images are considered spam as well as HTML.
Links: Clarify we may only endorse the Free versions of nginx, not the
non-Free versions.
Add a link to Starman as a unicorn derivative, as I even use Starman
myself. Remove yahns, since it's really the complete opposite of
unicorn and probably not appropriate to place next to Starman and
gunicorn
|
|
Older RubyGems (1.8.23.2 at least) does not seem to support
multiple version requirements for the Ruby version; so drop
the lower 1.9.3 requirement for now.
|
|
The PID of a process can never be zero as kill(2) interprets a '0'
PID arg as "every process in caller's process group", so there's no
risk of the 'nil.to_i => 0' conversion resulting in a truth value
when compared to $$.
|
|
For some reason, I thought invalid descriptors passed to UNICORN_FD
would be automatically closed by the master process; but apparently
this hasn't been the case. On the other hand, this bug has been
around for over 6 years now and nobody noticed or cared enough to
tell us, so fixing it might break existing setups.
Since there may be users relying on this behavior, we cannot change
the behavior anymore; so update the documentation and write at test
to ensure we can never "fix" this bug at the expense of breaking
any working setups which may be out there.
Keep in mind that a before_exec hook may always be used to modify
the UNICORN_FD environment by setting the close_on_exec flag and
removing the appropriate descriptor from the environment.
I originally intended to add the ability to inherit new listeners
without a config file specification so systemd users can avoid
repeating themselves in the systemd and unicorn config files,
but apparently there is nothing to change in our code.
|
|
Re-enable and expand on the test case while we're at it for new
Rubies. The bug is now fixed in Ruby 2.3.0dev as of r51576. We
shall assume anybody running a pre-release 2.3.0 at this point is
running a fairly recent snapshot, so we won't bother doing a
finer-grained check in the test for an exact revision number.
|
|
The statement about C exts hasn't been true since 2010 when kgio was
unfortunately introduced. However, I've been working on killing off
kgio. Maybe raindrops isn't worth it given the limits of SMP, either.
And I'm even tempted to rewrite the HTTP parser in Ruby...
Furthermore, Ruby Enterprise Edition is long gone and Ruby 2.0
is already old, so update that bit about CoW-friendliness.
While we're at it, avoid mentioning kgio at all in the Links
document, too.
|
|
Ruby 2.0+ has a copy-on-write-friendly memory layout by default,
and REE is long dead and just confusing to new users.
|
|
public-inbox supports read-only NNTP access nowadays to make it
easier to follow archives. It is read-only to encourage Cc:-ing
all participants (which avoids reliance on the few-points-of-failure
behavior of NNTP). Unlike email, NNTP also lacks good anti-spam
filtering.
Additionally, the gmane group also got redirected to the
bogomips.org address at some point since RubyForge died.
While we're at it, link to my post about enabling the submission
port (587). It's been a year and nothing bad has happened, yet.
Finally, remove most of the documentation for ssoma since it's
unlikely anybody will use it given the existence of NNTP access.
It did little besides clutter the page. However, git:// (used
by ssoma) remains strictly more efficient than NNTP.
Vebavpnyyl, gur AAGC freire sbe choyvp-vaobk pna unaqyr
gubhfnaqf bs fybj pyvragf. Fbzrguvat havpbea jvyy arire or noyr
gb qb :C
|
|
It does not look like we'll be compatible with Ruby 3.0 with
the plan for immutable string literals.
However, keep in mind 3.0 is still many years away and
decisions can change, so it would be premature to stop
assuming frozen string literals this year.
ref: https://bugs.ruby-lang.org/issues/11473
|
|
We no longer need Ruby 1.8 compatibility, so use String#clear
to reduce argument passing and code size.
|
|
They'll continue to be maintained, but we're no longer advertising
them. Also, favor lowercase "unicorn" while we're at it since that
matches the executable and gem name to avoid unnecessary escaping
for RDoc.
|
|
Linux users are effectively capped to 128 on stock installations
and may wonder why connections get rejected with overloaded apps
sooner rather than later.
|