about summary refs log tree commit homepage
DateCommit message (Collapse)
2020-04-27unicorn 5.5.5 v5.5.5
This release fixes a bug for users of multiple listeners setups where a busy listen socket could starve other listeners. Thanks to Stan Hu for reporting and testing. No need to upgrade if you're using a single listen socket. Link: https://yhbt.net/unicorn-public/CAMBWrQ=Yh42MPtzJCEO7XryVknDNetRMuA87irWfqVuLdJmiBQ@mail.gmail.com/
2020-04-16prevent single listener from monopolizing a worker
In setups with multiple listeners, it's possible for our greedy select(2)-avoidance optimization to get pinned on a single, busy listener and starve the other listener(s). Prevent starvation by retrying the select(2)-avoidance optimization if and only if all listeners were active. This should have no effect on the majority of deployments with only a single listener. Thanks to Stan Hu for reporting and testing. Reported-by: Stan Hu <stanhu@gmail.com> Tested-by: Stan Hu <stanhu@gmail.com> Link: https://yhbt.net/unicorn-public/CAMBWrQ=Yh42MPtzJCEO7XryVknDNetRMuA87irWfqVuLdJmiBQ@mail.gmail.com/
2020-03-24unicorn 5.5.4 v5.5.4
One change to improve RFC 7230 conformance in the HTTP parser: https://yhbt.net/unicorn-public/20200319022823.32472-1-bofh@yhbt.net/
2020-03-19http: improve RFC 7230 conformance
We need to favor "Transfer-Encoding: chunked" over "Content-Length" in the request header if they both exist. Furthermore, we now reject redundant chunking and cases where "chunked" is not the final encoding. We currently do not and have no plans to decode "gzip", "deflate", or "compress" encoding as described by RFC 7230. That's a job more appropriate for middleware, anyways. cf. https://tools.ietf.org/html/rfc7230 https://www.rfc-editor.org/errata_search.php?rfc=7230
2020-01-31unicorn 5.5.3 v5.5.3
Documentation updates to switch bogomips.org to yhbt.net since the .org TLD won't be affordable in the near future. There's also a few minor test cleanups.
2020-01-20test_upload: use spawn to simplify redirects
We can start using Ruby 1.9 APIs, nowadays
2020-01-20test_helper: remove unused `chunked_spawn'
It was added nearly 11 years ago in commit 6945342a1f0a4caa ("Transfer-Encoding: chunked streaming input support") but never used.
2020-01-20test/exec/test_exec: bring worker_processes down to 2
My hardware gets worse and worse every year :<
2020-01-20doc: s/bogomips.org/yhbt.net/g
bogomips.org is due to expire, soon, and I'm not willing to pay extortionist fees to Ethos Capital/PIR/ICANN to keep a .org. So it's at yhbt.net, for now, but it will change again to whatever's affordable... Identity is overrated. Tor users can use .onions and kick ICANN to the curb: torsocks w3m http://unicorn.ou63pmih66umazou.onion/ torsocks git clone http://ou63pmih66umazou.onion/unicorn.git/ torsocks w3m http://ou63pmih66umazou.onion/unicorn-public/ While we're at it, `s/news.gmane.org/news.gmane.io/g', too. (but I suspect that'll need to be resynched since our mail "List-Id:" header is changing).
2019-12-20unicorn 5.5.2 v5.5.2
Thanks to Terry Scheingeld, we now workaround a Ruby bug and can now run with taint checks enabled: <https://bugs.ruby-lang.org/issues/14485> <https://bogomips.org/unicorn-public/CABg1sXrvGv9G6CDQxePDUqTe6N-5UpLXm7eG3YQO=dda-Cgg7A@mail.gmail.com/> There's also a few documentation updates and building packages from source is easier since pandoc is no longer a dependency (and I can no longer afford the bandwidth or space to install it). Eric Wong (7): test/benchmark/ddstream: demo for slowly reading clients test/benchmark/readinput: demo for slowly uploading clients test/benchmark/uconnect: test for accept loop speed examples/unicorn@.service: note the NonBlocking flag Merge remote-tracking branch 'origin/ts/tmpio' test_util: get rid of some unused variables in tests doc: replace pandoc-"Markdown" with real manpages Terry Scheingeld (1): tmpio: workaround File#path being tainted on unlink
2019-12-15doc: replace pandoc-"Markdown" with real manpages
Trying to install pandoc on an x86-64 Debian stable system says: > Need to get 15.2 MB of archives. > After this operation, 117 MB of additional disk space will be used. My laptop is on metered Internet nowadays and already low on disk space, so installing pandoc is not realistic. Maybe it wasn't realistic to other hackers with limited resources in the past. There's also dozens of subtly incompatible Markdown flavors out there, most of which can't really handle manpages. Anyways, roff isn't too bad and at least groff is well-documented. Updating the website now requires olddoc 1.8.0 (which is much smaller than pandoc), but I'm the only one with that burden. On the flipside more users can update and read the manpages locally without extra software, since nearly every developer's *nix system has man(1) command, unlike pandoc.
2019-12-15test_util: get rid of some unused variables in tests
Ruby 2.7.0dev warns on them
2019-12-15Merge remote-tracking branch 'origin/ts/tmpio'
* origin/ts/tmpio: tmpio: workaround File#path being tainted on unlink
2019-12-11tmpio: workaround File#path being tainted on unlink
Ruby mistakenly taints the file path, causing File.unlink to fail: https://bugs.ruby-lang.org/issues/14485 Workaround the Ruby bug by keeping the path as a local variable and passing that to File.unlink, instead of the return value of File#path. Link: https://bogomips.org/unicorn-public/CABg1sXrvGv9G6CDQxePDUqTe6N-5UpLXm7eG3YQO=dda-Cgg7A@mail.gmail.com/
2019-07-03examples/unicorn@.service: note the NonBlocking flag
It's racy otherwise when starting simultaneous instanced units. Without specifying NonBlocking=true, systemd will clear the O_NONBLOCK flag every time it starts a new service instance. There's a small window where systemd can clear O_NONBLOCK immediately after it's set by Ruby (or kgio): unicorn@1 |systemd |unicorn@2 ---------------------------+----------------+-------------------- F_SETFL, O_NONBLOCK|O_RDWR | | (not running, yet) |F_SETFL, O_RDWR | |fork | | exec unicorn@2 | accept4(...) # blocks! | | (now started by systemd) | |F_SETFL,O_NONBLOCK|O_RDWR | |accept4(...) non-blocking
2019-05-12test/benchmark/uconnect: test for accept loop speed
In preparation for kgio removal, I want to ensure we can maintain existing performance when swapping kgio_tryaccept for accept_nonblock on Ruby 2.3+ There's plenty of TCP benchmarking tools, but TCP port reuse delays hurt predictability since unicorn doesn't do persistent connections. So this is exclusively for Unix sockets and uses Perl instead of Ruby since I don't want to be bothered with GC unpredictability on the client side.
2019-05-12test/benchmark/readinput: demo for slowly uploading clients
This is intended to demonstrate how badly we suck at dealing with slow clients making uploads. It can help users evaluate alternative fully-buffering reverse proxies, because nginx should not be the only option.
2019-05-12test/benchmark/ddstream: demo for slowly reading clients
This is intended to demonstrate how badly we suck at dealing with slow clients. It can help users evaluate alternative fully-buffering reverse proxies, because nginx should not be the only option. Update the benchmark README while we're at it
2019-05-06unicorn 5.5.1 v5.5.1
This release fixes and works around issues from v5.5.0 (2019-03-04) Stephen Demjanenko worked around a pipe resource accounting bug present in old Linux kernels. Linux 3.x users from 3.16.57 and on are unaffected. Linux 4.x users from 4.9 and on are unaffected. https://bogomips.org/unicorn-public/1556922018-24096-1-git-send-email-sdemjanenko@gmail.com/ Stan Pitucha reported a bug with the old `unicorn_rails' wrapper (intended for Rails 2.x users) which was promptly fixed by Jeremy Evans: https://bogomips.org/unicorn-public/CAJ2_uEPifcv_ux4sX=t0C4zHTHGhqVfcLcSB2kTU3Rb_6pQ3nw@mail.gmail.com/ There's also some doc updates to warn users off `unicorn_rails'; the homepage is now energy-efficient for OLEDs and CRTs; and I'm no longer advertising mailing list subscriptions (because I hate centralization and mail archives are the priority) Eric Wong (3): doc: unicorn_rails: clarify that it is intended for rails <= 2.x doc: stop advertising mailing list subscription doc: switch homepage to dark216 Jeremy Evans (1): unicorn_rails: fix regression with Rails >= 3.x in app build Stephen Demjanenko (1): Rescue failed pipe resizes due to permissions
2019-05-03Rescue failed pipe resizes due to permissions
When running: ``` require 'kgio' require 'raindrops' F_SETPIPE_SZ = 1031 if RUBY_PLATFORM =~ /linux/ Kgio::Pipe.new.each do |io| io.close_on_exec = true if defined?(F_SETPIPE_SZ) begin puts "setting" io.fcntl(F_SETPIPE_SZ, Raindrops::PAGE_SIZE) rescue Errno::EINVAL puts "rescued" rescue => e puts ["FAILED HARD", e].inspect end end end ``` on a few servers to test some Unicorn boot failures I saw: ``` ["FAILED HARD", #<Errno::EPERM: Operation not permitted>] ``` The `EPERM` error gets raised by the Linux kernel if: ``` (too_many_pipe_buffers_hard(pipe->user) || too_many_pipe_buffers_soft(pipe->user)) && !capable(CAP_SYS_RESOURCE) && !capable(CAP_SYS_ADMIN) ``` Given that the resize is not strictly necessary Unicorn should rescue the error and continue booting.
2019-04-22doc: switch homepage to dark216
Because Earth Day!: https://80x24.org/olddoc-public/20190422014906.4253-1-e@80x24.org/
2019-04-15doc: stop advertising mailing list subscription
It's still available, but no point in advertising something which increases the dependency on a centralized subscriber list. Subscription will never be required to write to our centralization-resistant public inbox (which anybody can read/mirror using HTTP(S) or NNTP). The mailing list has only attracted a tiny handful of subscribers since Rubyforge died, and more than half of the subscribers imported from Rubyforge have bounced off (or unsubscribed).
2019-04-15doc: unicorn_rails: clarify that it is intended for rails <= 2.x
Hopefully the wording is a little more explicit and clearer by stating its purpose in the first line of the description.
2019-03-06unicorn_rails: fix regression with Rails >= 3.x in app build
Note: `unicorn_rails' was only intended for Rails <= 2.x projects in the old days. Fixes: 5985dd50a9bd7238 ("Support default_middleware configuration option") From: Jeremy Evans <code@jeremyevans.net> cf. https://bogomips.org/unicorn-public/20190306055734.GC61406@jeremyevans.local/ Signed-off-by: Eric Wong <e@80x24.org> [ew: commit message]
2019-03-04unicorn 5.5.0 v5.5.0
Mostly identical to 5.5.0.pre1, which I didn't hear any feedback from: https://bogomips.org/unicorn-public/20181220222842.GA27382@dcvr/ > Jeremy Evans contributed the "default_middleware" configuration option: > > https://bogomips.org/unicorn-public/20180913192055.GD48926@jeremyevans.local/ > > Jeremy also contributed the ability to use separate groups for the process > and log files: > > https://bogomips.org/unicorn-public/20180913192449.GE48926@jeremyevans.local/ > > There's also a couple of uninteresting minor optimizations and > documentation additions. Otherwise, there's one extra change to use rb_gc_register_mark_object which is finally a documented part of the Ruby C-API, but has existed since the 1.9 days.
2018-12-28Links: add historical mongrel dev list mirror
unicorn has no future, but at least some of our past is archived. This was taken from nntp://news.gmane.org/gmane.comp.lang.ruby.mongrel.devel as the HTTP parts of gmane are dead. Unfortunately, Gmane only started archiving mongrel-development@rubyforge.org in 2008 and discussions prior to that may be lost. If anybody has mbox, Maildir or MH archives from those days, I'd be glad to import them.
2018-12-26use rb_gc_register_mark_object
Since Ruby 2.6, it's a documented part of the API and we may depend on it: https://bugs.ruby-lang.org/issues/9894 It's been around since the early Ruby 1.9 days, and reduces overhead compared to relying on rb_global_variable: https://bogomips.org/unicorn-public/20170301002854.29198-1-e@80x24.org/
2018-12-20unicorn 5.5.0.pre1 v5.5.0.pre1
Jeremy Evans contributed the "default_middleware" configuration option: https://bogomips.org/unicorn-public/20180913192055.GD48926@jeremyevans.local/ Jeremy also contributed the ability to use separate groups for the process and log files: https://bogomips.org/unicorn-public/20180913192449.GE48926@jeremyevans.local/ There's also a couple of uninteresting minor optimizations and documentation additions. Eric Wong (10): remove random seed reset atfork use IO#wait instead of kgio_wait_readable Merge branch '5.4-stable' shrink pipes under Linux socket_helper: add hint for FreeBSD users for accf_http(9) tests: ensure -N/--no-default-middleware not supported in config.ru doc: update more URLs to use HTTPS and avoid redirects deduplicate strings VM-wide in Ruby 2.5+ doc/ISSUES: add links to git clone-able mail archives of our dependencies README: minor updates and additional disclaimer Jeremy Evans (2): Make Worker#user support different process primary group and log file group Support default_middleware configuration option
2018-12-19README: minor updates and additional disclaimer
Nowadays, I mainly rely on systemd (and not USR2) for zero-downtime upgrades. Also, CoW-friendliness is standard in mainline Ruby since 2.0. There also needs to be a disclaimer to point out the unfortunate side-effect of robustness for hosting buggy apps.
2018-12-13doc/ISSUES: add links to git clone-able mail archives of our dependencies
Archives are crucial to preserving history and knowledge in Free Software projects, so promote them for projects we depend on. Naq lrf, gur nepuviny fbsgjner qrirybcrq sbe nepuvivat gur havpbea znvyvat yvfg unf ybat fhecnffrq gur hfrshyarff bs havpbea vgfrys :C
2018-12-12deduplicate strings VM-wide in Ruby 2.5+
String#-@ deduplicates strings starting with Ruby 2.5.0 Hash#[]= deduplicates strings starting in Ruby 2.6.0-rc1 This allows us to save a small amount of memory by sharing objects with other parts of the stack (e.g. Rack).
2018-10-18doc: update more URLs to use HTTPS and avoid redirects
Latency from redirects is painful, and HTTPS can protect privacy in some cases.
2018-09-21Support default_middleware configuration option
This allows for the equivalent of the -N/--no-default_middleware command line option to be specified in the configuration file so it doesn't need to be specified on the command line every time unicorn is executed. It explicitly excludes the use of -N/--no-default_middleware as an embedded configuration option in the rackup file, by ignoring the options after ARGV is parsed. In order to allow the configuration method to work, have the lambda that Unicorn.builder returns accept two arguments. Technically, only one argument is needed for the HttpServer instance, but I'm guessing if the lambda accepts a single argument, we expect that to be a rack application instead of a lambda that returns a rack application. The command line option option to disable default middleware will take precedence over the unicorn configuration file option if both are present. For backwards compatibility, if the lambda passed to HttpServer accepts 0 arguments, then call it without arguments. [ew: fix precedence for arity checking in build_app! configurator: ensure -N is respected when set in command-line]
2018-09-18tests: ensure -N/--no-default-middleware not supported in config.ru
Continue to make it easy to migrate AWAY from unicorn because vendor lock-in is the worst thing, especially if it's on us.
2018-09-13Make Worker#user support different process primary group and log file group
Previously, Unicorn always used the process's primary group as the the group of the log file. However, there are reasons to use a separate group for the log files, such as when you have many applications where each application uses it's own user and primary group, but you want to be able to have a user read the log files for all applications. Some operating systems have a fairly small limit on the number of groups per user, and it may not be feasible to have a user be in the primary group for all applications. a primary group
2018-08-20socket_helper: add hint for FreeBSD users for accf_http(9)
Because I forget to load accf_http on new FreeBSD installs, too :x
2018-08-20shrink pipes under Linux
We have never had any need for pipes with the default 64K capacity on Linux. Our pipes are only used for tiny writes in signal handlers and to perform parent shutdown detection. With the current /proc/sys/fs/pipe-user-pages-soft default, only 1024 pipes can be created by an unprivileged user before Linux clamps down the pipe size to 4K (a single page) for newly-created pipes[1]. So avoid penalizing OTHER pipe users who could benefit from the increased capacity and use only a single page for ourselves. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/pipe.c?h=v4.18#n642
2018-07-23Merge branch '5.4-stable'
* 5.4-stable: unicorn 5.4.1
2018-07-23use IO#wait instead of kgio_wait_readable
Slowly reducing dependencies on kgio. 'io/wait' is required by 'socket' these days, so it's no extra relocations for .so loading, either.
2018-07-23remove random seed reset atfork
It's not unicorn 6, yet, but we dropped Ruby 1.8 support at unicorn 5. Stable Ruby 1.9+ releases have always reseeded the PRNG at fork.
2018-07-23unicorn 5.4.1 v5.4.1 5.4-stable
This release quiets some warnings for Ruby 2.6 preview releases and enables tests to pass under Ruby 1.9.3. Otherwise, nothing interesting for Ruby 2.0..2.5 users. *YAWN* Eric Wong (1): quiet some mismatched indentation warnings Fumiaki MATSUSHIMA (1): Use IO#wait instead to fix test for Ruby 1.9
2018-05-01quiet some mismatched indentation warnings
Ruby trunk started warning about more mismatched indentations starting around r62836.
2018-02-24Use IO#wait instead to fix test for Ruby 1.9
IO#wait_readable is introduced since 2.0 I confirmed we can pass tests for all versions of Ruby with this patch. https://github.com/mtsmfm/unicorn/pull/2
2017-12-23unicorn 5.4.0 v5.4.0
Rack hijack support improves as the app code can capture and use the Rack `env' privately without copying it (to avoid clobbering by another client). Thanks to Sam Saffron for reporting and testing this new feature: https://bogomips.org/unicorn-public/CAAtdryPG3nLuyo0jxfYW1YHu1Q+ZpkLkd4KdWC8vA46B5haZxw@mail.gmail.com/T/ We also now support $DEBUG being set by the Rack app (instead of relying on the "-d" CLI switch). Thanks to James P Robinson Jr for reporting this bug: https://bogomips.org/unicorn-public/D6324CB4.7BC3E%25james.robinson3@cigna.com/T/ (Coincidentally, this fix will be irrelevant for Ruby 2.5 which requires 'pp' by default) There's a few minor test cleanups and documentation updates, too. All commits since v5.3.1 (2017-10-03): reduce method calls with String#start_with? require 'pp' if $DEBUG is set by Rack app avoid reusing env on hijack tests: cleanup some unused variable warnings ISSUES: add a note about Debian BTS interopability Roughly all mailing discussions since the last release: https://bogomips.org/unicorn-public/?q=d:20171004..20171223
2017-12-23ISSUES: add a note about Debian BTS interopability
This isn't anything new, just explicitly stating what's always been the case. In other news, I hate formal things.
2017-12-22tests: cleanup some unused variable warnings
Add a new "check-warnings" target to the GNUmakefile to make checking for this easier. Warnings aren't fatal, and newer versions of Ruby tend to increase warnings.
2017-12-16avoid reusing env on hijack
Hijackers may capture and reuse `env' indefinitely, so we must not use it in those cases for future requests. For non-hijack requests, we continue to reuse the `env' object to reduce memory recycling. Reported-and-tested-by: Sam Saffron <sam.saffron@gmail.com>
2017-11-16require 'pp' if $DEBUG is set by Rack app
While "unicorn -d" requires 'pp' when setting $DEBUG, we did not account for (rare) Rack applications setting $DEBUG at load time. Thanks-to: James P (Jim) Robinson Jr <James.Robinson3@Cigna.com>
2017-10-03Merge remote-tracking branch 'origin/5.3-stable'
* origin/5.3-stable: unicorn 5.3.1 fix GC issue on rb_global_variable array
2017-10-03unicorn 5.3.1 v5.3.1 5.3-stable
This release fixes an occasional GC problem introduced in v5.3.0 to reduce global variable overhead (commit 979ebcf91705709b) Thanks to Xuanzhong Wei for the patch which lead to this release: https://bogomips.org/unicorn-public/20171003182054.76392-1-azrlew@gmail.com/T/#u https://bogomips.org/unicorn-public/20171003145718.30404-1-azrlew@gmail.com/T/#u Xuanzhong Wei (1): fix GC issue on rb_global_variable array