about summary refs log tree commit homepage
AgeCommit message (Collapse)AuthorFilesLines
2017-03-07check_client_connection: use tcp state on linux ccc-tcpSimon Eskildsen4-25/+57
* Use a frozen empty array and a class variable for TCP_Info to avoid garbage. As far as I can tell, this shouldn't result in any garbage on any requests (other than on the first request). * Pass listener socket to #read to only check the client connection on a TCP server. * Short circuit CLOSE_WAIT after ESTABLISHED since in my testing it's the most common state after ESTABLISHED, it makes the numbers un-ordered, though. But comment should make it OK. * Definition of of `check_client_connection` based on whether Raindrops::TCP_Info is defined, instead of the class variable approach. * Changed the unit tests to pass a `nil` listener. Tested on our staging environment, and still works like a dream. I should note that I got the idea between this patch into Puma as well! https://github.com/puma/puma/pull/1227 [ew: squashed in temporary change for oob_gc.rb, but we'll come up with a different change to avoid breaking gctools <https://github.com/tmm1/gctools>] Acked-by: Eric Wong <e@80x24.org>
2017-03-01t0011-active-unix-socket.sh: fix race condition in testEric Wong1-1/+1
Killing the master process may lead to the worker dying on its own (as designed); before kill(1) has had an opportunity to send the second kill(2) syscall on the worker process. Killing the worker before the master might also lead to a needless respawn, so merely kill the master and let the worker follow it in death. This race condition occasionally caused test failures on slow, uniprocessor hardware.
2017-02-23Fix code example in after_worker_exit documentationJeremy Evans1-1/+1
Fixes: 2af91a1fef70d654 ("Add after_worker_exit configuration option")
2017-02-21Add after_worker_exit configuration optionJeremy Evans2-2/+23
This option is executed in the master process following all worker process exits. It is most useful in the case where the worker process crashes the ruby interpreter, as the worker process may not be able to send error notifications appropriately. For example, let's say you have a specific request that crashes a worker process, which you expect to be due to a improperly programmed C extension. By modifying your worker to save request related data in a temporary file and using this option, you can get a record of what request is crashing the application, which will make debugging easier. Example: after_worker_exit do |server, worker, status| server.logger.info "worker #{status.success? ? 'exit' : 'crash'}: #{status}" file = "request.#{status.pid}.txt" if File.exist?(file) do_something_with(File.read(file)) unless status.success? File.delete(file) end end
2017-02-21t/t0012-reload-empty-config.sh: access ivars directly if neededEric Wong1-1/+2
We'll be moving to direct ivar access to reduce the API (and method entry) overhead of internal unicorn classes. This means some tests like this one will reach deeper into internals. This will be necessary for the upcoming changes to add new configuration options to unicorn.
2017-02-15Revert "remove response_start_sent"Eric Wong1-0/+14
Oops, this was a half-baked change I was considering but forgot about. This reverts commit 69fd4f9bbff3708166fbf70163fa6e192dde1497.
2017-02-13http_request: freeze constant strings passed IO#writeEric Wong1-4/+1
This ensures we won't have duplicate objects in Ruby 2.0-2.4. For Ruby 2.5.0dev+, this avoids any duplicate cleanup introduced as of r57471: https://bugs.ruby-lang.org/issues/13085
2017-02-13remove response_start_sentEric Wong1-14/+0
2017-02-10tee_input: simplify condition for IO#writeEric Wong1-3/+1
IO#write already elides the write(2) syscall for empty buffers, so there's no need to complicate our instruction sequence footprint for the rare case of an empty buffer. The only cases a Rack app will have an empty buffer are: 1) `env['rack.input'].read` without args 2) `env['rack.input'].gets` Neither of these calls are safe for server-independent Rack apps as the client can OOM the app. unicorn itself provides no facility for limiting maximum rack.input size. Instead, unicorn relies on nginx to limit input size using the client_max_body_size directive.
2016-11-28TUNING: document THP caveat for Linux usersEric Wong1-1/+19
This probably applies to other kernels, too, but I'm most familiar with Linux.
2016-11-09drop rb_str_set_len compatibility replacementEric Wong2-21/+1
While it is innocuous after compiling, it can be a confusing source of errors for users with broken installations of Ruby itself: https://bogomips.org/unicorn-public/5ace6a20-e094-293d-93df-b557480e12d5@anyces.com/ https://bogomips.org/unicorn-public/02994a55-9c07-a3c5-f06b-a4c15551a67e@anyces.com/ rb_str_set_len has been provided since Ruby 1.8.7+, so we have not needed it since we dropped all 1.8.x support in unicorn 5.x.
2016-10-31unicorn 5.2.0 v5.2.0Eric Wong1-1/+1
Most notably, this release allows us to support requests with lines delimited by LF-only, as opposed to the standard CRLF pair and allowed by RFC 2616 sec 19.3. Thanks to Mishael A Sibiryakov for the explanation and change: https://bogomips.org/unicorn-public/1476954332.1736.156.camel@junki.org/ Thanks to Let's Encrypt, the website also moves to HTTPS <https://bogomips.org/unicorn/> to improve reader privacy. The "unicorn.bogomips.org" subdomain will be retired soon to reduce subjectAltName bloat and speed up certificate renewals. There's also the usual round of documentation and example updates, too. Eric Wong (7): examples/init.sh: update to reduce upgrade raciness doc: systemd should only kill master in example examples/logrotate.conf: update example for systemd doc: update gmane URLs to point to our own archives relocate website to https://bogomips.org/unicorn/ TODO: remove Rack 2.x item build: "install-gem" target avoids network Mishael A Sibiryakov (1): Add some tolerance (RFC2616 sec. 19.3)
2016-10-31build: "install-gem" target avoids networkEric Wong1-1/+1
No need to go online when installing a locally-built gem.
2016-10-31TODO: remove Rack 2.x itemEric Wong1-2/+0
Rack 2.x is less of a jump than initially expected, and we've already supported it for a few releases, already.
2016-10-30Merge remote-tracking branch 'origin/website-move'Eric Wong15-31/+31
* origin/website-move: relocate website to https://bogomips.org/unicorn/
2016-10-30Merge remote-tracking branch 'origin/rfc2616-sec19.3'Eric Wong2-1/+19
* origin/rfc2616-sec19.3: Add some tolerance (RFC2616 sec. 19.3)
2016-10-30Merge remote-tracking branch 'origin/jr/init'Eric Wong1-8/+36
* origin/jr/init: examples/init.sh: update to reduce upgrade raciness
2016-10-25relocate website to https://bogomips.org/unicorn/ website-moveEric Wong15-31/+31
HTTPS helps some with reader privacy and Let's Encrypt seems to be working well enough the past few months. This change will allow us to reduce subjectAltName bloat in our TLS certificate over time. It will also promote domain name agility to support mirrors or migrations to other domains (including a Tor hidden service mirror). http://bogomips.org/unicorn/ will remain available for people on legacy systems without usable TLS. There is no plan for automatic redirecting from HTTP to HTTPS at this time.
2016-10-20Add some tolerance (RFC2616 sec. 19.3) rfc2616-sec19.3Mishael A Sibiryakov2-1/+19
Hi all. We're implementing client certificate authentication with nginx and unicorn.  Nginx configured in the following way: proxy_set_header X-SSL-Client-Cert $ssl_client_cert; When client submits certificate and nginx passes it to the unicorn, unicorn responds with 400 (Bad Request). This caused because nginx doesn't use "\r\n" they using just "\n" and multilne headers is failed to parse (I've added test). Accorording to RFC2616 section 19.3: https://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.3 "The line terminator for message-header fields is the sequence CRLF. However, we recommend that applications, when parsing such headers, recognize a single LF as a line terminator and ignore the leading CR." CRLF changed to ("\r\n" | "\n") Github commit https://github.com/uno4ki/unicorn/commit/ed127b66e162aaf176de05720f6be758f8b41b1f PS: Googling "nginx unicorn ssl_client_cert" shows the problem. 
2016-07-28doc: update gmane URLs to point to our own archivesEric Wong4-7/+9
Gmane's NNTP server remains up, but the HTTP site is down: https://lars.ingebrigtsen.no/2016/07/28/the-end-of-gmane/ Anyways, our own archives are designed to be mirror-able via git: git clone --mirror https://bogomips.org/unicorn-public And the code is self-hostable: git clone https://public-inbox.org
2016-06-20examples/logrotate.conf: update example for systemdEric Wong1-1/+16
...And add placeholders for other systems
2016-06-13doc: systemd should only kill master in exampleEric Wong1-0/+5
By default, systemd kills every process in the control group when stopping a service. While it ought to be harmless to signal workers, some Rack applications (and perhaps further subprocesses) can misbehave when interrupted by a signal. Ensure we only hit the master on graceful shutdown to avoid tickling bugs in Rack apps. This is the reason we switched to having the master send "fake" signals for workers beginning with unicorn 4.8.0 back in 2013/2014.
2016-06-07examples/init.sh: update to reduce upgrade raciness jr/initEric Wong1-8/+36
Rework the "upgrade" target to only read the PID files once to avoid misreading the wrong PID files in the middle of the upgrade. Additionally, introduce the UPGRADE_DELAY environment parameter so users can increase/decrease according to their application startup time. PID files are inherently racy and people should be using a process manager (systemd or similar) instead, but this should mitigate most of the problems with the old target. While we're at it, add LSB tags for systems which complain about the lack of them and modernize things a bit using $(command) construct instead of the more fragile `command`. Thanks-to: Jesper Rønn-Jensen <jesperrr@gmail.com>
2016-04-01unicorn 5.1.0 - rack is optional, again v5.1.0Eric Wong1-1/+1
Note: no code changes since 5.1.0.pre1 from January.^WNo, wait, last minute performance improvement added today. See below. The big change is rack is not required (but still recommended). Applications are expected to depend on rack on their own so they can specify the version of rack they prefer without unicorn pulling in a newer, potentially incompatible version. unicorn will always attempt to work with multiple versions of rack as practical. The HTTP parser also switched to using the TypedData C-API for extra type safety and memory usage accounting support in the 'objspace' extension. Thanks to Adam Duke to bringing the rack change to our attention and Aaron Patterson for helping with the matter. Last minute change: we now support the new leftpad() syscall under Linux for major performance and security improvement: http://mid.gmane.org/1459463613-32473-1-git-send-email-richard@nod.at 8^H9 changes since 5.0.1: http: TypedData C-API conversion various documentation updates doc: bump olddoc to ~> 1.2 for extra NNTP URL rack is optional at runtime, required for dev doc update for ClientShutdown exceptions class unicorn 5.1.0.pre1 - rack is optional, again doc: reference --keep-file-descriptors for "bundle exec" doc: further trimming to reduce noise use leftpad Linux syscall for speed!
2016-03-31doc: further trimming to reduce noiseEric Wong2-11/+4
It's not worth mentioning pre-Rack versions of Rails anymore, and there are a few async Rack applications reliant on EventMachine which we do not use. Some uses of chunked request decoding are not well-handled with nginx in front, anyways; so avoid mentioning them. Additionally, avoid introducing new terms into the lexicon and just refer to "mailing list" as a generic term.
2016-03-17doc: reference --keep-file-descriptors for "bundle exec"Eric Wong2-3/+5
"bundle exec" alone is not suitable for use with systemd-style socket activation due to Ruby 2.0+ behavior of setting close-on-exec for file descriptors above 2. However, the "--keep-file-descriptors" option was added to bundler 1.4.0 to workaround this Ruby 2.0 change and may be used to prevent Ruby 2.0+ from closing file descriptors on exec. Thanks to Amir Yalon and Christos Trochalakis for bringing up this issue on the mailing list: http://bogomips.org/unicorn-public/1457824748.3666627.547425122.2A828B07@webmail.messagingengine.com/
2016-01-27unicorn 5.1.0.pre1 - rack is optional, again v5.1.0.pre1Eric Wong1-1/+1
The big change is rack is not required (but still recommended). Applications are expected to depend on rack on their own so they can specify the version of rack they prefer without unicorn pulling in a newer, potentially incompatible version. unicorn will always attempt to work with multiple versions of rack as practical. The HTTP parser also switched to using the TypedData C-API for extra type safety and memory usage accounting support in the 'objspace' extension. Thanks to Adam Duke to bringing the rack change to our attention and Aaron Patterson for helping with the matter. There might be more documentation-related changes before 5.1.0 final. I am considering dropping pandoc from manpage generation and relying on pod2man (from Perl) because it has a wider install base. 5 changes since v5.0.1: http: TypedData C-API conversion various documentation updates doc: bump olddoc to ~> 1.2 for extra NNTP URL rack is optional at runtime, required for dev doc update for ClientShutdown exceptions class
2016-01-27doc update for ClientShutdown exceptions classEric Wong1-1/+3
State explicitly applications should not rely on it, and instead rescue the generic EOFError exception. This class will stick around because there may inevitably be things which rely on it, but we should not encourage it, either.
2016-01-27rack is optional at runtime, required for devEric Wong3-25/+37
We do not want to pull in a newer or older version of rack depending on an the application running under it requires. Furthermore, it has always been possible to use unicorn without any middleware at all. Without rack, we'll be missing descriptive status text in the first response line, but any valid HTTP/1.x parser should be able to handle it properly. ref: http://bogomips.org/unicorn-public/20160121201255.GA6186@dcvr.yhbt.net/t/#u Thanks-to: Adam Duke <adam.v.duke@gmail.com> Thanks-to: Aaron Patterson <tenderlove@ruby-lang.org>
2016-01-09doc: bump olddoc to ~> 1.2 for extra NNTP URLEric Wong2-2/+4
Additional advertising for the gmane NNTP server makes sense from a robustness standpoint: nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general Not advertising other HTTP-based URLs just yet. They could contain images/frames/JS/CSS and add unnecessary clutter to the footer. NNTP puts the client in control of UI.
2016-01-07various documentation updatesEric Wong7-19/+13
* add nntp_url to the olddoc website footer * update legacy support status for 4.x (not 4.8.x) * update copyright range to 2016 * note all of our development tools are Free Software, too * remove cgit mention; it may not always be cgit (but URLs should remain compatible). * discourage downloading snapshot tarballs; "git clone" + periodic "git fetch" is more efficient * remove most mentions of unicorn_rails as that was meant for ancient Rails 1.x/2.x users * update path reference to Ruby 2.3.0 * fix nginx upstream module link to avoid redirect * shorten Message-ID example to avoid redirects and inadvertant linkage
2015-12-13http: TypedData C-API conversionEric Wong2-12/+37
This provides some extra type safety if combined with other C extensions, as well as allowing us to account for memory usage of the HTTP parser in ObjectSpace. This requires Ruby 1.9.3+ and has remained a stable API since then. This will become officially supported when Ruby 2.3.0 is released later this month. This API has only been documented in doc/extension.rdoc (formerly README.EXT) in the Ruby source tree since April 2015, r50318
2015-11-17unicorn 5.0.1 - continuing to violate Rack SPEC v5.0.1Eric Wong1-1/+1
Once again, we allow nil values in response headers. We've had this bug since March 2009, and thus cannot expect existing applications and middlewares running unicorn to fix this. Unfortunately, supporting this bug contributes to application server lock-in, but at least we'll document it as such. Thanks to Owen Ou <o@heroku.com> for reporting this regression: http://bogomips.org/unicorn-public/CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com/ Additionally, systemd examples are now in the examples/ directory based on a post by Christos Trochalakis <yatiohi@ideopolis.gr>: http://bogomips.org/unicorn-public/20150708130821.GA1361@luke.ws.skroutz.gr/
2015-11-17add .gitattributes for Ruby method detectionEric Wong1-0/+5
The "diff" function detection for C does not map well to Ruby files, take advantage of gitattributes(5) to improve method name detection in generated patches as well as making "git diff -W" output more useful.
2015-11-17examples: add systemd socket and service filesEric Wong2-0/+37
Since we have init scripts, we ought to have the equivalent for systemd users who cannot upgrade via the normal SIGUSR2 mechanism; but can use multiple services: "unicorn@1" + h"unicorn@2" to accomplish the same thing. Based on examples by Christos Trochalakis <yatiohi@ideopolis.gr> ref: http://bogomips.org/unicorn-public/20150708130821.GA1361@luke.ws.skroutz.gr/
2015-11-17http_response: allow nil values in response headersEric Wong2-1/+10
This blatantly violates Rack SPEC, but we've had this bug since March 2009[1]. Thus, we cannot expect all existing applications and middlewares to fix this bug and will probably have to support it forever. Unfortunately, supporting this bug contributes to application server lock-in, but at least we'll document it as such. [1] commit 1835c9e2e12e6674b52dd80e4598cad9c4ea1e84 ("HttpResponse: speed up non-multivalue headers") Reported-by: Owen Ou <o@heroku.com> Ref: <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
2015-11-01unicorn 5.0.0 - most boring major release. EVER. v5.0.0Eric Wong1-1/+1
An evolutionary dead-end since its announcement[1] nearly six years ago, this old-fashioned preforker has had enough bugs and missteps that it's managed to hit version 5! I wish I could say unicorn 5 is leaps and bounds better than 4, but it is not. This major version change allows us to drop some cruft and unused features which accumulated over the years, resulting in several kilobytes of memory saved[2]! Compatibility: * The horrible, proprietary (:P) "Status:" response header is finally gone, saving at least 16 precious bytes in every HTTP response. This should make it easier to write custom HTTP clients which are compatible across all HTTP servers. It will hopefully make migrating between different Rack servers easier for new projects. * Ruby 1.8 support removed. Ruby 1.9.3 is currently the earliest supported version. However, expect minor, likely-unnoticeable performance regressions if you use Ruby 2.1 or earlier. Going forward, unicorn will favor the latest version (currently 2.2) of the mainline Ruby implementation, potentially sacrificing performance on older Rubies. * Some internal, undocumented features and APIs used by derivative servers are gone; removing bloat and slightly lowering memory use. We have never and will never endorse the use of any applications or middleware with a dependency on unicorn, applications should be written for Rack instead. Note: Rainbows! 5.0 will be released next week or so to be compatible with unicorn 5.x New features: * sd_listen_fds(3) emulation added for systemd compatibility. You may now stop using PID files and other process monitoring software when using systemd. * Newly-set TCP socket options are now applied to inherited sockets. * Dynamic changes in the application to Rack::Utils::HTTP_STATUS hash is now supported; allowing users to set custom status lines in Rack to be reflected in unicorn. This feature causes a minor performance regression, but is made up for Ruby 2.2 users with other optimizations. * The monotonic clock is used under Ruby 2.1+, making the timeout feature immune to system clock changes. As unicorn may be used anonymously without registration, the project is committed to supporting anonymous and pseudonymous help requests, contributions and feedback via plain-text mail to: unicorn-public@bogomips.org The mail submission port (587) is open to those behind firewalls and allows access via Tor and anonymous remailers. Archives are accessible via: http://bogomips.org/unicorn-public/ and mirrored to various other places, so you do not need to use a valid address when posting. Finally, rest assured the core design of unicorn will never change. It will never use new-fangled things like threads, kqueue or epoll; but will always remain a preforking server serving one client per-process. [1] http://mid.gmane.org/20090211230457.GB22926@dcvr.yhbt.net [2] this would've been like, totally gnarly in the 80s!
2015-11-01manpage: reference systemd socket activation featureEric Wong1-0/+5
We cannot rely on users reading release notes.
2015-11-01doc updatesEric Wong2-4/+5
ISSUES: note images are considered spam as well as HTML. Links: Clarify we may only endorse the Free versions of nginx, not the non-Free versions. Add a link to Starman as a unicorn derivative, as I even use Starman myself. Remove yahns, since it's really the complete opposite of unicorn and probably not appropriate to place next to Starman and gunicorn
2015-11-01gemspec: relax Ruby version requirement for old RubyGemsEric Wong1-2/+10
Older RubyGems ( at least) does not seem to support multiple version requirements for the Ruby version; so drop the lower 1.9.3 requirement for now.
2015-11-01golf down conditional for socket activationEric Wong1-1/+1
The PID of a process can never be zero as kill(2) interprets a '0' PID arg as "every process in caller's process group", so there's no risk of the 'nil.to_i => 0' conversion resulting in a truth value when compared to $$.
2015-10-27inheriting sockets from UNICORN_FD does not close themEric Wong2-3/+21
For some reason, I thought invalid descriptors passed to UNICORN_FD would be automatically closed by the master process; but apparently this hasn't been the case. On the other hand, this bug has been around for over 6 years now and nobody noticed or cared enough to tell us, so fixing it might break existing setups. Since there may be users relying on this behavior, we cannot change the behavior anymore; so update the documentation and write at test to ensure we can never "fix" this bug at the expense of breaking any working setups which may be out there. Keep in mind that a before_exec hook may always be used to modify the UNICORN_FD environment by setting the close_on_exec flag and removing the appropriate descriptor from the environment. I originally intended to add the ability to inherit new listeners without a config file specification so systemd users can avoid repeating themselves in the systemd and unicorn config files, but apparently there is nothing to change in our code.
2015-10-27sd_listen_fds emulation cleanupEric Wong2-19/+25
Re-enable and expand on the test case while we're at it for new Rubies. The bug is now fixed in Ruby 2.3.0dev as of r51576. We shall assume anybody running a pre-release 2.3.0 at this point is running a fairly recent snapshot, so we won't bother doing a finer-grained check in the test for an exact revision number.
2015-10-15doc: DESIGN: update old statements.Eric Wong2-7/+2
The statement about C exts hasn't been true since 2010 when kgio was unfortunately introduced. However, I've been working on killing off kgio. Maybe raindrops isn't worth it given the limits of SMP, either. And I'm even tempted to rewrite the HTTP parser in Ruby... Furthermore, Ruby Enterprise Edition is long gone and Ruby 2.0 is already old, so update that bit about CoW-friendliness. While we're at it, avoid mentioning kgio at all in the Links document, too.
2015-10-15unicorn.conf.rb: remove mention of REE-specific settingEric Wong1-4/+1
Ruby 2.0+ has a copy-on-write-friendly memory layout by default, and REE is long dead and just confusing to new users.
2015-10-05doc: update mail archive infoEric Wong2-27/+23
public-inbox supports read-only NNTP access nowadays to make it easier to follow archives. It is read-only to encourage Cc:-ing all participants (which avoids reliance on the few-points-of-failure behavior of NNTP). Unlike email, NNTP also lacks good anti-spam filtering. Additionally, the gmane group also got redirected to the bogomips.org address at some point since RubyForge died. While we're at it, link to my post about enabling the submission port (587). It's been a year and nothing bad has happened, yet. Finally, remove most of the documentation for ssoma since it's unlikely anybody will use it given the existence of NNTP access. It did little besides clutter the page. However, git:// (used by ssoma) remains strictly more efficient than NNTP. Vebavpnyyl, gur AAGC freire sbe choyvp-vaobk pna unaqyr gubhfnaqf bs fybj pyvragf. Fbzrguvat havpbea jvyy arire or noyr gb qb :C
2015-08-22gemspec: limit to 1.9.3 and 2.xEric Wong1-0/+1
It does not look like we'll be compatible with Ruby 3.0 with the plan for immutable string literals. However, keep in mind 3.0 is still many years away and decisions can change, so it would be premature to stop assuming frozen string literals this year. ref: https://bugs.ruby-lang.org/issues/11473
2015-08-22stream_input: favor String#clear over String#replaceEric Wong1-3/+3
We no longer need Ruby 1.8 compatibility, so use String#clear to reduce argument passing and code size.
2015-07-15doc: remove references to old serversEric Wong15-93/+78
They'll continue to be maintained, but we're no longer advertising them. Also, favor lowercase "unicorn" while we're at it since that matches the executable and gem name to avoid unnecessary escaping for RDoc.
2015-07-15configurator: document net.core.somaxconn sysctl dependencyEric Wong1-0/+5
Linux users are effectively capped to 128 on stock installations and may wonder why connections get rejected with overloaded apps sooner rather than later.