unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: Steven Stewart-Gallus <sstewartgallus00@mylangara.bc.ca>
Cc: unicorn-public@bogomips.org, Jesse Storimer <jstorimer@gmail.com>
Subject: Re: Bug, Unicorn can drop signals in extreme conditions
Date: Wed, 18 Feb 2015 09:15:17 +0000	[thread overview]
Message-ID: <20150218091517.GA21920@dcvr.yhbt.net> (raw)
In-Reply-To: <fc15a0f85b70.54e15396@langara.bc.ca>

Steven Stewart-Gallus <sstewartgallus00@mylangara.bc.ca> wrote:
> Hello,
> 
> While reading the article at
> http://www.sitepoint.com/the-self-pipe-trick-explained/ I realized
> that the signal handling code shown there and of the same type in your
> application is broken.  As you have a single nonblocking pipe in your
> application you can drop signals entirely (and not just fold multiple
> signals of the same type into one).  The simplest fix is to use a pipe
> for each individual type of signal or to just use pselect or similar.
> I'm pretty sure that there is a way to use a single pipe but it seems
> complicated and very difficult to implement correctly.

(I've Cc-ed Jesse for this)

I wasn't sure exactly what you were referring to, but now I see where
the sitepoint.com article makes calls in the wrong order:

     self_writer.write_nonblock('.') # XXX may fail!
     SIGNAL_QUEUE << signal

In contrast, unicorn enqueues the signal before attempting to write to
the pipe, so we don't care at all if the write fails.  It doesn't matter
if the non-blocking write fails due to the pipe being full at all; as
any existing data in the pipe is sufficient to cause the reader to wake
up.

The correct order would be:

     SIGNAL_QUEUE << signal
     self_writer.write_nonblock('.') # we don't care if this fails

Furthermore, this avoids a race condition in multi-threaded situations.
Order is critical, even outside of signal handlers, this ordering of
events is fundamental to correct usage of things like condition
variables and waitqueues.


Btw, MRI 1.9.3+ also uses the self-pipe trick internally, too (see
thread_pthread.c) for signals and thread switching.  Current versions
use two pipes, one for high-priority wakeups, and one for low-priority
wakeups.

And on a related note, using pselect/ppoll/epoll_pwait/signalfd-style
syscalls which affect the signal mask is not feasible with runtimes
which already implement a high-level signal handling API.  I ripped out
signalfd support from sleepy_penguin a few years back because it would
always conflict with the signal handling API in Ruby itself...

And eventfd is cheaper and usable in place of a self-pipe from Ruby, of
course(*), but I haven't convinced ruby-core it's worth the maintenance
effort for thread_pthread.c; so a conservative project like unicorn
won't use it, yet.

Anyways, thanks for bringing this to our attention.


(*) I use it in yet another horribly-named server :)

  reply	other threads:[~2015-02-18  9:15 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-02-16  2:19 Bug, Unicorn can drop signals in extreme conditions Steven Stewart-Gallus
2015-02-18  9:15 ` Eric Wong [this message]
2015-02-18 17:27   ` Steven Stewart-Gallus

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://yhbt.net/unicorn/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150218091517.GA21920@dcvr.yhbt.net \
    --to=e@80x24.org \
    --cc=jstorimer@gmail.com \
    --cc=sstewartgallus00@mylangara.bc.ca \
    --cc=unicorn-public@bogomips.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhbt.net/unicorn.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).