Re: Worker Timeout Debugging

[Eric Wong <normalperson@yhbt.net>]

Bill Vieux <billv@yahoo.com> wrote:
> I am getting occasional worker timeouts for a Rails app hosted on
> Heroku. I have rack-timeout set at the top of the middleware with a
> shorter timeout than unicorn workers, but it is not firing for some
> reason.

Which version of Ruby is this and what C extensions are you using?
This is probably a buggy C extension which blocks the VM.

> Are there any recommended techniques to determine the call stack when
> the worker is reaped?

Not the call stack, but you can get the Rails endpoint regardless of
Ruby version:

  Ensure your Rails logger is configured to log the PID at the start
  of every request.  (I think Rails logs parameters by default for
  every request).

  Match up the killed workers logging from unicorn to the PIDs that
  started a request (but never logged a completion) in the Rails log.

> The solutions that come to mind for me seem to require running a
> customized build of unicorn. For example: start a script (e.g., gdb to
> attach and core dump the worker) before (or in place of) sending the
> SIGKILL.

If you're using Ruby 1.9 or later, maybe sending SIGBUS/SIGSEGV can work
to trigger a Ruby core dump.

Do not attempt to install SIGSEGV/BUS handler(s) via Ruby, Ruby 1.9
already handles those internally.  Ruby 2.0.0 prevents trapping SEGV/BUS
with Ruby-level Signal#trap handlers, even.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying