| Date | Commit message (Collapse) |
|---|
|
It's a little less DRY, and there'll be no NEWS file generated,
but it's one less thing to install, so perhaps that's worth it.
The website at https://bogomips.org/unicorn/ will continue
to use olddoc, of course,
|
|
* origin/worker_exec:
Don't pass a block for fork when forking workers
Add worker_exec configuration option
|
|
* ccc-tcp-v3:
test_ccc: use a pipe to synchronize test
http_request: support proposed Raindrops::TCP states on non-Linux
|
|
This quiets down warnings when run with '-w'
|
|
rack 2.x exists nowadays still allows rewindable input as an
option, and we will still enable it by default to avoid breaking
any existing code.
Hide the internal documentation since we do not want people
depending on unicorn internals; so there's no reason to confuse
or overwhelm people with documentation about it. Arguably,
TeeInput and StreamInput should not be documented publically at
all, but I guess that ship has long sailed...
|
|
Email was never private, and won't further burden myself or
any future maintainers with trying to maintain someone elses'
privacy.
Offering private support is also unfair to readers on public
lists who may get a watered down or improperly translated
summary (if at all).
Instead, encourage the use of anonymity tools and scrubbing of
sensitive information when the sender deems necessary.
|
|
Sleeping 1 second to test 100 requests is too long for some
systems; and not long enough for others.
We need to also finish reading the sleeper response to ensure
the server actually got the second request in, before sending
SIGQUIT to terminate it; as it's possible for the test client
to connect and abort 100 clients before the server even
increments the request counter for the 2nd request.
|
|
Tis better to pull than push, or something like that.
|
|
raindrops 0.18+ will have Raindrops::TCP state hash for portable
mapping of TCP states to their respective numeric values. This
was necessary because TCP state numbers (and even macro names)
differ between FreeBSD and Linux (and possibly other OSes).
Favor using the Raindrops::TCP state hash if available, but
fall back to the hard-coded values since older versions of
raindrops did not support TCP_INFO on non-Linux systems.
While we're in the area, favor "const_defined?" over "defined?"
to reduce the inline constant cache footprint for branches
which are only evaluated once.
Patches to implement Raindrops::TCP for FreeBSD are available at:
https://bogomips.org/raindrops-public/20170316031652.17433-1-e@80x24.org/T/
|
|
* origin/ccc-tcp-v3:
http_request: reduce insn size for check_client_connection
support "struct tcp_info" on non-Linux and Ruby 2.2+
revert signature change to HttpServer#process_client
new test for check_client_connection
check_client_connection: use tcp state on linux
|
|
bogomips.org is dropping prefixes to reduce subjectAltName bloat
in TLS certificates.
|
|
On FreeBSD 10.3, the value of SO_KEEPALIVE returned by
getsockopt is 8, even when set to '1' via setsockopt.
Relax the test to only ensure the boolean value is
interpreted as "true".
Verified independently of Ruby using the following:
--------8<---------
#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
static int err(const char *msg)
{
perror(msg);
return 1;
}
int main(void)
{
int sv[2];
int set = 1;
int got;
socklen_t len = (socklen_t)sizeof(int);
int rc;
rc = socketpair(PF_LOCAL, SOCK_STREAM, 0, sv);
if (rc) return err("socketpair failed");
rc = setsockopt(sv[0], SOL_SOCKET, SO_KEEPALIVE, &set, len);
if (rc) return err("setsockopt failed");
rc = getsockopt(sv[0], SOL_SOCKET, SO_KEEPALIVE, &got, &len);
if (rc) return err("getsockopt failed");
printf("got: %d\n", got);
return 0;
}
|
|
Some versions of test-unit will fail if an unspecified test is
attempted via "-n", so we need to define an empty test.
We cannot use "skip", either, as that seems exclusive to
minitest; and we won't use minitest since it has more
incompatible changes than test-unit over the last 8 years.
The memory leak test is gone since we're more versed in the
Ruby C API nowadays, modern GCs + mallocs may be less
predictable about releasing memory back to the OS.
|
|
GNU expr supports '+' to match one or more occurrences, but
it seems the expr(1) on my FreeBSD installation does not.
|
|
Accept filters can only be set on listen sockets, and it also
fails with EINVAL if it's already set.
Untested, but I suppose changing the accept filter on a listening
socket is not supported, either; since that could affect in-flight
sockets.
|
|
Unlike constants and instance variables, class variable access
is not optimized in the mainline Ruby VM. Use a constant
instead, to take advantage of inline constant caching.
This further reduces runtime instruction size by avoiding a
branch by allocating the Raindrops::TCP_Info object up front.
This reduces the method size by roughly 300 bytes on 64-bit.
|
|
This reduces the stack depth, making GC more efficient.
|
|
The worker_exec configuration option makes all worker processes
exec after forking. This initializes the worker processes with
separate memory layouts, defeating address space discovery
attacks on operating systems supporting address space layout
randomization, such as Linux, MacOS X, NetBSD, OpenBSD, and
Solaris.
Support for execing workers is very similar to support for reexecing
the master process. The main difference is the worker's to_i and
master pipes also need to be inherited after worker exec just as the
listening sockets need to be inherited after reexec.
Because execing working is similar to reexecing the master, this
extracts a couple of methods from reexec (listener_sockets and
close_sockets_on_exec), so they can be reused in worker_spawn.
|
|
Maybe oob_gc probably isn't heavily used anymore, maybe
some Ruby 2.2+ users will benefit from this constant
reduction.
Followup-to: fb2f10e1d7a72e67 ("reduce constants and optimize for Ruby 2.2")
|
|
rb_global_variable registers the address of the variable which
refers to the object, instead of the object itself. This adds
extra overhead to each global variable for our case, where the
variable is frozen and never changed.
Given there are currently 59 elements in this array, this saves
58 singly-linked list entries and associated malloc calls and
associated overhead in the current mainline Ruby 2.x
implementation. On 64-bit GNU libc malloc, this is already
16 * 58 = 928 bytes; more than the extra object slot and array
slack space used by the new mark array.
Mainline Ruby 1.9+ currently has a rb_gc_register_mark_object
public function which would suite our needs, too, but it is
currently undocumented, and may not be available in the future.
|
|
* origin/chroot:
Add after_worker_ready configuration option
Add support for chroot to Worker#user
|
|
Ruby 2.2+ can show "struct tcp_info" as a string via
Socket::Option#inspect, and we can attempt to parse it
out to extract the information we need.
Parsing this string is inefficient, but does not depend on the
ordering of the tcp_info struct.
|
|
We can force kgio_tryaccept to return an internal class
for TCP objects by subclassing Kgio::TCPServer.
This avoids breakage in any unfortunate projects which depend on
our undocumented internal APIs, such as gctools
<https://github.com/tmm1/gctools>
|
|
This was a bit tricky to test, but it's probably more reliable
now that we're relying on TCP_INFO.
Based on test by Simon Eskildsen <simon.eskildsen@shopify.com>:
https://bogomips.org/unicorn-public/CAO3HKM49+aLD=KLij3zhJqkWnR7bCWVan0mOvxD85xfrW8RXOw@mail.gmail.com/
|
|
* Use a frozen empty array and a class variable for TCP_Info to avoid
garbage. As far as I can tell, this shouldn't result in any garbage on
any requests (other than on the first request).
* Pass listener socket to #read to only check the client connection on
a TCP server.
* Short circuit CLOSE_WAIT after ESTABLISHED since in my testing it's
the most common state after ESTABLISHED, it makes the numbers
un-ordered, though. But comment should make it OK.
* Definition of of `check_client_connection` based on whether
Raindrops::TCP_Info is defined, instead of the class variable
approach.
* Changed the unit tests to pass a `nil` listener.
Tested on our staging environment, and still works like a dream.
I should note that I got the idea between this patch into Puma as well!
https://github.com/puma/puma/pull/1227
[ew: squashed in temporary change for oob_gc.rb, but we'll come
up with a different change to avoid breaking gctools
<https://github.com/tmm1/gctools>]
Acked-by: Eric Wong <e@80x24.org>
|
|
Killing the master process may lead to the worker dying on its
own (as designed); before kill(1) has had an opportunity to send
the second kill(2) syscall on the worker process.
Killing the worker before the master might also lead to a
needless respawn, so merely kill the master and let the worker
follow it in death.
This race condition occasionally caused test failures on slow,
uniprocessor hardware.
|
|
This adds a hook that is called after the application has
been loaded by the worker process, directly before it starts
accepting requests. This hook is necessary if your application
needs to gain access to resources during initialization,
and then drop privileges before serving requests.
This is especially useful in conjunction with chroot support
so the app can load all the normal ruby libraries it needs
to function, and then chroot before accepting requests.
If you are preloading the app, it's possible to drop privileges
or chroot in after_fork, but if you are not preloading the app,
the only way to currently do this is to override the private
HttpServer#init_worker_process method, and overriding private
methods is a recipe for future breakage if the internals are
modified. This hook allows for such functionality to be
supported and not break in future versions of Unicorn.
|
|
Any chrooting would need to happen inside Worker#user, because
you can't chroot until after you have parsed the list of groups,
and you must chroot before dropping root privileges.
chroot adds an extra layer of security, so that if the unicorn
process is exploited, file system access is limited to the chroot
directory instead of the entire file system.
|
|
Fixes: 2af91a1fef70d654 ("Add after_worker_exit configuration option")
|
|
This option is executed in the master process following all
worker process exits. It is most useful in the case where
the worker process crashes the ruby interpreter, as the worker
process may not be able to send error notifications
appropriately.
For example, let's say you have a specific request that crashes a
worker process, which you expect to be due to a improperly
programmed C extension. By modifying your worker to save request
related data in a temporary file and using this option, you can get
a record of what request is crashing the application, which will
make debugging easier.
Example:
after_worker_exit do |server, worker, status|
server.logger.info "worker #{status.success? ? 'exit' : 'crash'}: #{status}"
file = "request.#{status.pid}.txt"
if File.exist?(file)
do_something_with(File.read(file)) unless status.success?
File.delete(file)
end
end
|
|
We'll be moving to direct ivar access to reduce the API (and
method entry) overhead of internal unicorn classes. This means
some tests like this one will reach deeper into internals.
This will be necessary for the upcoming changes to add new
configuration options to unicorn.
|
|
Oops, this was a half-baked change I was considering
but forgot about.
This reverts commit 69fd4f9bbff3708166fbf70163fa6e192dde1497.
|
|
This ensures we won't have duplicate objects in Ruby 2.0-2.4.
For Ruby 2.5.0dev+, this avoids any duplicate cleanup
introduced as of r57471: https://bugs.ruby-lang.org/issues/13085
|
|
|
|
IO#write already elides the write(2) syscall for empty buffers,
so there's no need to complicate our instruction sequence
footprint for the rare case of an empty buffer.
The only cases a Rack app will have an empty buffer are:
1) `env['rack.input'].read` without args
2) `env['rack.input'].gets`
Neither of these calls are safe for server-independent Rack apps
as the client can OOM the app.
unicorn itself provides no facility for limiting maximum
rack.input size. Instead, unicorn relies on nginx to limit
input size using the client_max_body_size directive.
|
|
This probably applies to other kernels, too, but I'm most
familiar with Linux.
|
|
While it is innocuous after compiling, it can be a confusing
source of errors for users with broken installations of Ruby
itself:
https://bogomips.org/unicorn-public/5ace6a20-e094-293d-93df-b557480e12d5@anyces.com/
https://bogomips.org/unicorn-public/02994a55-9c07-a3c5-f06b-a4c15551a67e@anyces.com/
rb_str_set_len has been provided since Ruby 1.8.7+, so we have
not needed it since we dropped all 1.8.x support in unicorn 5.x.
|
|
Most notably, this release allows us to support requests with
lines delimited by LF-only, as opposed to the standard CRLF
pair and allowed by RFC 2616 sec 19.3.
Thanks to Mishael A Sibiryakov for the explanation and change:
https://bogomips.org/unicorn-public/1476954332.1736.156.camel@junki.org/
Thanks to Let's Encrypt, the website also moves to HTTPS
<https://bogomips.org/unicorn/> to improve reader privacy. The
"unicorn.bogomips.org" subdomain will be retired soon to reduce
subjectAltName bloat and speed up certificate renewals.
There's also the usual round of documentation and example
updates, too.
Eric Wong (7):
examples/init.sh: update to reduce upgrade raciness
doc: systemd should only kill master in example
examples/logrotate.conf: update example for systemd
doc: update gmane URLs to point to our own archives
relocate website to https://bogomips.org/unicorn/
TODO: remove Rack 2.x item
build: "install-gem" target avoids network
Mishael A Sibiryakov (1):
Add some tolerance (RFC2616 sec. 19.3)
|
|
No need to go online when installing a locally-built gem.
|
|
Rack 2.x is less of a jump than initially expected,
and we've already supported it for a few releases, already.
|
|
* origin/website-move:
relocate website to https://bogomips.org/unicorn/
|
|
* origin/rfc2616-sec19.3:
Add some tolerance (RFC2616 sec. 19.3)
|
|
* origin/jr/init:
examples/init.sh: update to reduce upgrade raciness
|
|
HTTPS helps some with reader privacy and Let's Encrypt seems to
be working well enough the past few months.
This change will allow us to reduce subjectAltName bloat in our
TLS certificate over time. It will also promote domain name
agility to support mirrors or migrations to other domains
(including a Tor hidden service mirror).
http://bogomips.org/unicorn/ will remain available for people on
legacy systems without usable TLS. There is no plan for automatic
redirecting from HTTP to HTTPS at this time.
|
|
Hi all.
We're implementing client certificate authentication with nginx and
unicorn.
Nginx configured in the following way:
proxy_set_header X-SSL-Client-Cert $ssl_client_cert;
When client submits certificate and nginx passes it to the unicorn,
unicorn responds with 400 (Bad Request). This caused because nginx
doesn't use "\r\n" they using just "\n" and multilne headers is failed
to parse (I've added test).
Accorording to RFC2616 section 19.3:
https://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.3
"The line terminator for message-header fields is the sequence CRLF.
However, we recommend that applications, when parsing such headers,
recognize a single LF as a line terminator and ignore the leading CR."
CRLF changed to ("\r\n" | "\n")
Github commit
https://github.com/uno4ki/unicorn/commit/ed127b66e162aaf176de05720f6be758f8b41b1f
PS: Googling "nginx unicorn ssl_client_cert" shows the problem.
|
|
Gmane's NNTP server remains up, but the HTTP site is down:
https://lars.ingebrigtsen.no/2016/07/28/the-end-of-gmane/
Anyways, our own archives are designed to be mirror-able via git:
git clone --mirror https://bogomips.org/unicorn-public
And the code is self-hostable: git clone https://public-inbox.org
|
|
...And add placeholders for other systems
|
|
By default, systemd kills every process in the control group
when stopping a service. While it ought to be harmless to
signal workers, some Rack applications (and perhaps further
subprocesses) can misbehave when interrupted by a signal.
Ensure we only hit the master on graceful shutdown to avoid
tickling bugs in Rack apps.
This is the reason we switched to having the master send
"fake" signals for workers beginning with unicorn 4.8.0
back in 2013/2014.
|
|
Rework the "upgrade" target to only read the PID files once to
avoid misreading the wrong PID files in the middle of the
upgrade.
Additionally, introduce the UPGRADE_DELAY environment parameter
so users can increase/decrease according to their application
startup time.
PID files are inherently racy and people should be using a
process manager (systemd or similar) instead, but this should
mitigate most of the problems with the old target.
While we're at it, add LSB tags for systems which complain
about the lack of them and modernize things a bit using
$(command) construct instead of the more fragile `command`.
Thanks-to: Jesper Rønn-Jensen <jesperrr@gmail.com>
|
|
Note: no code changes since 5.1.0.pre1 from January.^WNo, wait,
last minute performance improvement added today. See below.
The big change is rack is not required (but still recommended).
Applications are expected to depend on rack on their own so they can
specify the version of rack they prefer without unicorn pulling
in a newer, potentially incompatible version.
unicorn will always attempt to work with multiple versions of rack
as practical.
The HTTP parser also switched to using the TypedData C-API for
extra type safety and memory usage accounting support in the
'objspace' extension.
Thanks to Adam Duke to bringing the rack change to our attention
and Aaron Patterson for helping with the matter.
Last minute change: we now support the new leftpad() syscall under
Linux for major performance and security improvement:
http://mid.gmane.org/1459463613-32473-1-git-send-email-richard@nod.at
8^H9 changes since 5.0.1:
http: TypedData C-API conversion
various documentation updates
doc: bump olddoc to ~> 1.2 for extra NNTP URL
rack is optional at runtime, required for dev
doc update for ClientShutdown exceptions class
unicorn 5.1.0.pre1 - rack is optional, again
doc: reference --keep-file-descriptors for "bundle exec"
doc: further trimming to reduce noise
use leftpad Linux syscall for speed!
|