Date | Commit message (Collapse) |
|
Note: no code changes since 5.1.0.pre1 from January.^WNo, wait,
last minute performance improvement added today. See below.
The big change is rack is not required (but still recommended).
Applications are expected to depend on rack on their own so they can
specify the version of rack they prefer without unicorn pulling
in a newer, potentially incompatible version.
unicorn will always attempt to work with multiple versions of rack
as practical.
The HTTP parser also switched to using the TypedData C-API for
extra type safety and memory usage accounting support in the
'objspace' extension.
Thanks to Adam Duke to bringing the rack change to our attention
and Aaron Patterson for helping with the matter.
Last minute change: we now support the new leftpad() syscall under
Linux for major performance and security improvement:
http://mid.gmane.org/1459463613-32473-1-git-send-email-richard@nod.at
8^H9 changes since 5.0.1:
http: TypedData C-API conversion
various documentation updates
doc: bump olddoc to ~> 1.2 for extra NNTP URL
rack is optional at runtime, required for dev
doc update for ClientShutdown exceptions class
unicorn 5.1.0.pre1 - rack is optional, again
doc: reference --keep-file-descriptors for "bundle exec"
doc: further trimming to reduce noise
use leftpad Linux syscall for speed!
|
|
It's not worth mentioning pre-Rack versions of Rails anymore,
and there are a few async Rack applications reliant on
EventMachine which we do not use.
Some uses of chunked request decoding are not well-handled
with nginx in front, anyways; so avoid mentioning them.
Additionally, avoid introducing new terms into the lexicon
and just refer to "mailing list" as a generic term.
|
|
"bundle exec" alone is not suitable for use with systemd-style
socket activation due to Ruby 2.0+ behavior of setting close-on-exec
for file descriptors above 2. However, the "--keep-file-descriptors"
option was added to bundler 1.4.0 to workaround this Ruby 2.0 change
and may be used to prevent Ruby 2.0+ from closing file descriptors
on exec.
Thanks to Amir Yalon and Christos Trochalakis for bringing up
this issue on the mailing list:
http://bogomips.org/unicorn-public/1457824748.3666627.547425122.2A828B07@webmail.messagingengine.com/
|
|
The big change is rack is not required (but still recommended).
Applications are expected to depend on rack on their own so they can
specify the version of rack they prefer without unicorn pulling
in a newer, potentially incompatible version.
unicorn will always attempt to work with multiple versions of rack
as practical.
The HTTP parser also switched to using the TypedData C-API for
extra type safety and memory usage accounting support in the
'objspace' extension.
Thanks to Adam Duke to bringing the rack change to our attention
and Aaron Patterson for helping with the matter.
There might be more documentation-related changes before 5.1.0
final. I am considering dropping pandoc from manpage generation
and relying on pod2man (from Perl) because it has a wider install
base.
5 changes since v5.0.1:
http: TypedData C-API conversion
various documentation updates
doc: bump olddoc to ~> 1.2 for extra NNTP URL
rack is optional at runtime, required for dev
doc update for ClientShutdown exceptions class
|
|
State explicitly applications should not rely on it, and instead
rescue the generic EOFError exception. This class will stick
around because there may inevitably be things which rely on it,
but we should not encourage it, either.
|
|
We do not want to pull in a newer or older version of rack depending
on an the application running under it requires. Furthermore, it
has always been possible to use unicorn without any middleware at
all.
Without rack, we'll be missing descriptive status text in the first
response line, but any valid HTTP/1.x parser should be able to
handle it properly.
ref:
http://bogomips.org/unicorn-public/20160121201255.GA6186@dcvr.yhbt.net/t/#u
Thanks-to: Adam Duke <adam.v.duke@gmail.com>
Thanks-to: Aaron Patterson <tenderlove@ruby-lang.org>
|
|
Additional advertising for the gmane NNTP server makes sense
from a robustness standpoint:
nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
Not advertising other HTTP-based URLs just yet. They could contain
images/frames/JS/CSS and add unnecessary clutter to the footer.
NNTP puts the client in control of UI.
|
|
* add nntp_url to the olddoc website footer
* update legacy support status for 4.x (not 4.8.x)
* update copyright range to 2016
* note all of our development tools are Free Software, too
* remove cgit mention; it may not always be cgit
(but URLs should remain compatible).
* discourage downloading snapshot tarballs;
"git clone" + periodic "git fetch" is more efficient
* remove most mentions of unicorn_rails as that
was meant for ancient Rails 1.x/2.x users
* update path reference to Ruby 2.3.0
* fix nginx upstream module link to avoid redirect
* shorten Message-ID example to avoid redirects
and inadvertant linkage
|
|
This provides some extra type safety if combined with other
C extensions, as well as allowing us to account for memory usage of
the HTTP parser in ObjectSpace.
This requires Ruby 1.9.3+ and has remained a stable API since
then. This will become officially supported when Ruby 2.3.0 is
released later this month.
This API has only been documented in doc/extension.rdoc (formerly
README.EXT) in the Ruby source tree since April 2015, r50318
|
|
Once again, we allow nil values in response headers. We've had
this bug since March 2009, and thus cannot expect existing
applications and middlewares running unicorn to fix this.
Unfortunately, supporting this bug contributes to application
server lock-in, but at least we'll document it as such.
Thanks to Owen Ou <o@heroku.com> for reporting this regression:
http://bogomips.org/unicorn-public/CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com/
Additionally, systemd examples are now in the examples/ directory
based on a post by Christos Trochalakis <yatiohi@ideopolis.gr>:
http://bogomips.org/unicorn-public/20150708130821.GA1361@luke.ws.skroutz.gr/
|
|
The "diff" function detection for C does not map well to
Ruby files, take advantage of gitattributes(5) to improve
method name detection in generated patches as well as
making "git diff -W" output more useful.
|
|
Since we have init scripts, we ought to have the equivalent for
systemd users who cannot upgrade via the normal SIGUSR2 mechanism;
but can use multiple services: "unicorn@1" + h"unicorn@2" to
accomplish the same thing.
Based on examples by Christos Trochalakis <yatiohi@ideopolis.gr>
ref:
http://bogomips.org/unicorn-public/20150708130821.GA1361@luke.ws.skroutz.gr/
|
|
This blatantly violates Rack SPEC, but we've had this bug since
March 2009[1]. Thus, we cannot expect all existing applications
and middlewares to fix this bug and will probably have to
support it forever.
Unfortunately, supporting this bug contributes to application
server lock-in, but at least we'll document it as such.
[1] commit 1835c9e2e12e6674b52dd80e4598cad9c4ea1e84
("HttpResponse: speed up non-multivalue headers")
Reported-by: Owen Ou <o@heroku.com>
Ref: <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
|
|
An evolutionary dead-end since its announcement[1] nearly six years
ago, this old-fashioned preforker has had enough bugs and missteps
that it's managed to hit version 5!
I wish I could say unicorn 5 is leaps and bounds better than 4, but
it is not. This major version change allows us to drop some cruft
and unused features which accumulated over the years, resulting in
several kilobytes of memory saved[2]!
Compatibility:
* The horrible, proprietary (:P) "Status:" response header is
finally gone, saving at least 16 precious bytes in every HTTP
response. This should make it easier to write custom HTTP clients
which are compatible across all HTTP servers. It will hopefully
make migrating between different Rack servers easier for new
projects.
* Ruby 1.8 support removed. Ruby 1.9.3 is currently the earliest
supported version. However, expect minor, likely-unnoticeable
performance regressions if you use Ruby 2.1 or earlier. Going
forward, unicorn will favor the latest version (currently 2.2) of
the mainline Ruby implementation, potentially sacrificing
performance on older Rubies.
* Some internal, undocumented features and APIs used by
derivative servers are gone; removing bloat and slightly lowering
memory use. We have never and will never endorse the use of any
applications or middleware with a dependency on unicorn,
applications should be written for Rack instead.
Note: Rainbows! 5.0 will be released next week or so to be
compatible with unicorn 5.x
New features:
* sd_listen_fds(3) emulation added for systemd compatibility.
You may now stop using PID files and other process monitoring
software when using systemd.
* Newly-set TCP socket options are now applied to inherited sockets.
* Dynamic changes in the application to Rack::Utils::HTTP_STATUS
hash is now supported; allowing users to set custom status lines
in Rack to be reflected in unicorn. This feature causes a minor
performance regression, but is made up for Ruby 2.2 users with
other optimizations.
* The monotonic clock is used under Ruby 2.1+, making the
timeout feature immune to system clock changes.
As unicorn may be used anonymously without registration, the
project is committed to supporting anonymous and pseudonymous
help requests, contributions and feedback via plain-text mail to:
unicorn-public@bogomips.org
The mail submission port (587) is open to those behind firewalls
and allows access via Tor and anonymous remailers.
Archives are accessible via: http://bogomips.org/unicorn-public/
and mirrored to various other places, so you do not need to use
a valid address when posting.
Finally, rest assured the core design of unicorn will never change.
It will never use new-fangled things like threads, kqueue or epoll;
but will always remain a preforking server serving one client
per-process.
[1] http://mid.gmane.org/20090211230457.GB22926@dcvr.yhbt.net
[2] this would've been like, totally gnarly in the 80s!
|
|
We cannot rely on users reading release notes.
|
|
ISSUES: note images are considered spam as well as HTML.
Links: Clarify we may only endorse the Free versions of nginx, not the
non-Free versions.
Add a link to Starman as a unicorn derivative, as I even use Starman
myself. Remove yahns, since it's really the complete opposite of
unicorn and probably not appropriate to place next to Starman and
gunicorn
|
|
Older RubyGems (1.8.23.2 at least) does not seem to support
multiple version requirements for the Ruby version; so drop
the lower 1.9.3 requirement for now.
|
|
The PID of a process can never be zero as kill(2) interprets a '0'
PID arg as "every process in caller's process group", so there's no
risk of the 'nil.to_i => 0' conversion resulting in a truth value
when compared to $$.
|
|
For some reason, I thought invalid descriptors passed to UNICORN_FD
would be automatically closed by the master process; but apparently
this hasn't been the case. On the other hand, this bug has been
around for over 6 years now and nobody noticed or cared enough to
tell us, so fixing it might break existing setups.
Since there may be users relying on this behavior, we cannot change
the behavior anymore; so update the documentation and write at test
to ensure we can never "fix" this bug at the expense of breaking
any working setups which may be out there.
Keep in mind that a before_exec hook may always be used to modify
the UNICORN_FD environment by setting the close_on_exec flag and
removing the appropriate descriptor from the environment.
I originally intended to add the ability to inherit new listeners
without a config file specification so systemd users can avoid
repeating themselves in the systemd and unicorn config files,
but apparently there is nothing to change in our code.
|
|
Re-enable and expand on the test case while we're at it for new
Rubies. The bug is now fixed in Ruby 2.3.0dev as of r51576. We
shall assume anybody running a pre-release 2.3.0 at this point is
running a fairly recent snapshot, so we won't bother doing a
finer-grained check in the test for an exact revision number.
|
|
The statement about C exts hasn't been true since 2010 when kgio was
unfortunately introduced. However, I've been working on killing off
kgio. Maybe raindrops isn't worth it given the limits of SMP, either.
And I'm even tempted to rewrite the HTTP parser in Ruby...
Furthermore, Ruby Enterprise Edition is long gone and Ruby 2.0
is already old, so update that bit about CoW-friendliness.
While we're at it, avoid mentioning kgio at all in the Links
document, too.
|
|
Ruby 2.0+ has a copy-on-write-friendly memory layout by default,
and REE is long dead and just confusing to new users.
|
|
public-inbox supports read-only NNTP access nowadays to make it
easier to follow archives. It is read-only to encourage Cc:-ing
all participants (which avoids reliance on the few-points-of-failure
behavior of NNTP). Unlike email, NNTP also lacks good anti-spam
filtering.
Additionally, the gmane group also got redirected to the
bogomips.org address at some point since RubyForge died.
While we're at it, link to my post about enabling the submission
port (587). It's been a year and nothing bad has happened, yet.
Finally, remove most of the documentation for ssoma since it's
unlikely anybody will use it given the existence of NNTP access.
It did little besides clutter the page. However, git:// (used
by ssoma) remains strictly more efficient than NNTP.
Vebavpnyyl, gur AAGC freire sbe choyvp-vaobk pna unaqyr
gubhfnaqf bs fybj pyvragf. Fbzrguvat havpbea jvyy arire or noyr
gb qb :C
|
|
It does not look like we'll be compatible with Ruby 3.0 with
the plan for immutable string literals.
However, keep in mind 3.0 is still many years away and
decisions can change, so it would be premature to stop
assuming frozen string literals this year.
ref: https://bugs.ruby-lang.org/issues/11473
|
|
We no longer need Ruby 1.8 compatibility, so use String#clear
to reduce argument passing and code size.
|
|
They'll continue to be maintained, but we're no longer advertising
them. Also, favor lowercase "unicorn" while we're at it since that
matches the executable and gem name to avoid unnecessary escaping
for RDoc.
|
|
Linux users are effectively capped to 128 on stock installations
and may wonder why connections get rejected with overloaded apps
sooner rather than later.
|
|
Turns out ruby does have trouble emulating systemd, for now:
[ruby-core:69895] https://bugs.ruby-lang.org/issues/11336
When we re-enable this test, we'll only enable it for fixed Rubies.
The actual socket inheritance functionality works in any version of
Ruby, of course, it's just that emulating systemd won't work until
ruby-core fixes mainline Ruby.
|
|
There is a minor TCP socket options are now applied to inherited
sockets, and we have native support for inheriting sockets from
systemd (by emulating the sd_listen_fds(3) function).
Dynamic changes in the application to Rack::Utils::HTTP_STATUS
codes is now supported, so you can use your own custom status
lines.
Ruby 2.2 and later is now favored for performance.
Optimizations by using constants which made sense in earlier
versions of Ruby are gone: so users of old Ruby versions
will see performance regressions. Ruby 2.2 users should
see the same or better performance, and we have less code
as a result.
* doc: update some invalid URLs
* apply TCP socket options on inherited sockets
* reflect changes in Rack::Utils::HTTP_STATUS_CODES
* reduce constants and optimize for Ruby 2.2
* http_response: reduce size of multi-line header path
* emulate sd_listen_fds for systemd support
* test/unit/test_response.rb: compatibility with older test-unit
This also includes all changes in unicorn 5.0.0.pre1:
http://bogomips.org/unicorn-public/m/20150615225652.GA16164@dcvr.yhbt.net.html
|
|
assert_predicate really isn't that useful even if it seems
preferred in another project I work on. Avoid having folks
download the latest test-unit if they're on an old version of
Ruby (e.g. 1.9.3) which bundled it.
|
|
systemd socket emulation shares FDs across execve, just like
the built-in SIGUSR2 upgrade process in unicorn. Thus it is
easy to support inheriting sockets from systemd.
Tested-by: Christos Trochalakis <yatiohi@ideopolis.gr>
|
|
This should save over 100 bytes of bytecode overhead due to
reduced method dispatch points. The performance difference
when this is actually hit could go either way depending on
how String#<< and realloc(3) interact, but it's uncommon
enough that nobody is expected to notice either way.
|
|
Ruby (MRI) 2.1 optimizes allocations away on String#freeze with
literal strings.
Furthermore, Ruby 2.2 optimizes away literal string allocations
when they are used as arguments to Hash#[] and Hash#[]=
Thus we can avoid expensive constant lookups and cache overhead
by taking advantage of advancements in Ruby.
Since Ruby 2.2 has been out for 7 months, now; it ought to be safe
to introduce minor performance regressions for folks using older
Rubies (1.9.3+ remains supported) to benefit folks on the latest
Ruby.
This should recover the performance lost in the
"reflect changes in Rack::Utils::HTTP_STATUS_CODES" change
in synthetic benchmarks.
|
|
Applications may want to alter the message associated with HTTP
status codes in Rack::Utils::HTTP_STATUS_CODES. Avoid memoizing
status lines ahead-of-time
Note: this introduces a minor performance regression, but ought to
be unnoticeable unless you're running "Hello world"-type apps.
|
|
TCP socket options are now set when inheriting existing sockets from
a parent process. I'm fairly certain all the TCP setsockopt knobs
we use are idempotent and harmless to change.
If anything, the only directive I'd be uncomfortable changing is
shortening the listen(2) (aka :backlog) size, but we've always
changed that anyways since it also applies to UNIX sockets.
Note: removing a configuration knob in a unicorn config file can not
reset the value to the OS-provided default setting. Inherited
sockets must use a new setting to override existing ones.
(or the socket needs to be closed and re-created in the process
launcher before unicorn inherits it).
Noticed-by: Christos Trochalakis <yatiohi@ideopolis.gr>
<20150626114129.GA25883@luke.ws.skroutz.gr>
|
|
Most of these were found by the `linkchecker' package
in Debian.
|
|
This release finally drops Ruby 1.8 support and requires Ruby 1.9.3
or later. The horrible "Status:" header in our HTTP response is
finally gone, saving at least 16 precious bytes in every single HTTP
response.
Under Ruby 2.1 and later, the monotonic clock is used for timeout
handling for better accuracy.
Several experimental, unused and undocumented features are removed.
There's also tiny, minor performance and memory improvements from
dropping 1.8 compatibility, but probably nothing noticeable on a
typical real-life (bloated) app.
The biggest performance improvement we made was to our website by
switching to olddoc. Depending on connection speed, latency, and
renderer performance, it typically loads two to four times faster.
Finally, for the billionth time: unicorn must never be exposed
to slow clients, as it will never ever use new-fangled things
like non-blocking socket I/O, threads, epoll or kqueue. unicorn
must be used with a fully-buffering reverse proxy such as nginx
for slow clients.
* ISSUES: update with mailing list subscription
* GIT-VERSION-GEN: start 5.0.0 development
* http: remove xftrust options
* FAQ: add entry for Rails autoflush_log
* dev: remove isolate dependency
* unicorn.gemspec: depend on test-unit 3.0
* http_response: remove Status: header
* remove RubyForge and Freecode references
* remove mongrel.rubyforge.org references
* http: remove the keepalive requests limit
* http: reduce parser from 72 to 56 bytes on 64-bit
* examples: add run_once to before_fork hook example
* worker: remove old tmp accessor
* http_server: save 450+ bytes of memory on x86-64
* t/t0002-parser-error.sh: relax test for rack 1.6.0
* remove SSL support
* tmpio: drop the "size" method
* switch docs + website to olddoc
* README: clarify/reduce references to unicorn_rails
* gemspec: fixup olddoc migration
* use the monotonic clock under Ruby 2.1+
* http: -Wshorten-64-to-32 warnings on clang
* remove old inetd+git examples and exec_cgi
* http: standalone require + reduction in binary size
* GNUmakefile: fix clean gem build + reduce build cruft
* socket_helper: reduce constant lookups and caching
* remove 1.8, <= 1.9.1 fallback for missing IO#autoclose=
* favor IO#close_on_exec= over fcntl in 1.9+
* use require_relative to reduce syscalls at startup
* doc: update support status for Ruby versions
* fix uninstalled testing and reduce require paths
* test_socket_helper: do not depend on SO_REUSEPORT
* favor "a.b(&:c)" form over "a.b { |x| x.c }"
* ISSUES: add section for bugs in other projects
* http_server: favor ivars over constants
* explain 11 byte magic number for self-pipe
* const: drop constants used by Rainbows!
* reduce and localize constant string use
* Links: mark Rainbows! as historical, reference yahns
* save about 200 bytes of memory on x86-64
* http: remove deprecated reset method
* http: remove experimental dechunk! method
* socket_helper: update comments
* doc: document UNICORN_FD in manpage
* doc: document Etc.nprocessors for worker_processes
* favor more string literals for cold call sites
* tee_input: support for Rack::TempfileReaper middleware
* support TempfileReaper in deployment and development envs
* favor kgio_wait_readable for single FD over select
* Merge tag 'v4.9.0'
* http_request: support rack.hijack by default
* avoid extra allocation for hijack proc creation
* FAQ: add note about ECONNRESET errors from bodies
* process SIGWINCH unless stdin is a TTY
* ISSUES: discourage HTML mail strongly, welcome nyms
* http: use rb_hash_clear in Ruby 2.0+
* http_response: avoid special-casing for Rack < 1.5
* www: install NEWS.atom.xml properly
* http_server: remove a few more accessors and constants
* http_response: simplify regular expression
* move the socket into Rack env for hijacking
* http: move response_start_sent into the C ext
* FAQ: reorder bit on Rack 1.1.x and Rails 2.3.x
* ensure body is closed during hijack
|
|
Middlewares such as Rack::Lock (used by Rails) break badly unless
the response body is closed on hijack, so we will close it to follow
the lead of other popular Rack servers.
While it's unclear if there's anybody using rack.hijack with unicorn,
we'll try to emulate the behavior of other servers as much as
possible.
ref: https://github.com/ngauthier/tubesock/issues/10
|
|
These things were a while ago, and while apps using them still
exist, they should not be near the top of the FAQ.
|
|
Combined with the previous commit to eliminate the `@socket'
instance variable, this eliminates the last instance variable
in the Unicorn::HttpRequest class.
Eliminating the last instance variable avoids the creation of a
internal hash table used for implementing the "generic" instance
variables found in non-pure-Ruby classes. Method entry overhead
remains the same.
While this change doesn't do a whole lot for unicorn memory usage
where the HttpRequest is a singleton, it helps other HTTP servers
which rely on this code where thousands of clients may be connected.
|
|
This avoids the expensive generic instance variable for @socket
and exposes the socket as `env["unicorn.socket"]' to the Rack
application.
As as nice side-effect, applications may access
`env["unicorn.socket"]' as part of the API may be useful for
3rd-party bits such as Raindrops::TCP_Info for reading the tcp_info
struct on Linux-based systems.
Yes, `env["unicorn.socket"]' is a proprietary API in unicorn!
News at 11! But then again, unicorn is not the first Rack server
to implement `env["#{servername}.socket"]', either...
|
|
It's ugly and less-readable to have redundant \z statements, and
according to ObjectSpace.memsize_of, this saves 4 bytes on x86-64.
|
|
Unnecessarily exposed accessors and constants take up unnecessary
memory in constant/method tables as well as using extra space in
instruction sequences.
Preforking servers like unicorn are a bloated pigs anyways,
but saving a few hundred bytes here and there can add up and
make them marginally less bad.
|
|
I just noticed the 4.9.0 release was not properly reflected in
the Atom news feed at http://unicorn.bogomips.org/NEWS.atom.xml
|
|
Rack 1.4 and earlier will soon die out, so avoid having extra,
overengineered code and method dispatch to silently drop support
for mis-hijacking with old Rack versions.
This will cause improperly hijacked responses in all versions of
Rack to fail, but allows properly hijacked responses to work
regardless of Rack version.
Followup-to: commit fdf09e562733f9509d275cb13c1c1a04e579a68a
("http_request: support rack.hijack by default")
|
|
Calling the function directly avoids the overhead of Ruby method
table lookup and global method cache. The only downside is this is
now hidden from tracers and cannot be overridden from Ruby, but I
doubt anybody cares about that.
|
|
HTML email is too likely to be lost, so more strongly discourage it.
While we're at it, make it clear we allow anonymous and pseudonymous
contributions, unlike many projects nowadays.
|
|
Some process managers such as foreman and daemontools rely on
unicorn not daemonizing, but we still want to be able to process
SIGWINCH in that case.
stdout and stderr may be redirected to a pipe (for cronolog or
similar process), so those are less likely to be attached to a TTY
than stdin. This also allows users to process SIGWINCH when running
inside a regular terminal if they redirect stdin to /dev/null.
Reported-by: Dan Moore <dan@vaporwa.re>
References: <etPan.555b4293.5b47a5b7.e617@danbookpro>
<20150519232858.GA23515@dcvr.yhbt.net>
|
|
Thanks to Michael Fischer and Gabe Martin-Dempesy for bringing this
to light on the mailing list.
Ref: <CABHxtY7Sn5yaiR5a3gDk1G4XySE+UtfuqUTcOSdmwneXLD5rcg@mail.gmail.com>
Ref: <FC91211E-FD32-432C-92FC-0318714C2170@zendesk.com>
Cc: Michael Fischer <mfischer@zendesk.com>
Cc: Gabe Martin-Dempesy <gabe@zendesk.com>
|
|
proc creation is expensive, so merely use a 48-byte generic ivar
hash slot for @socket instead.
|