From: Eric Wong <normalperson@yhbt.net>
To: unicorn list <mongrel-unicorn@rubyforge.org>
Cc: "Carlos Peñas" <theistian@gmx.com>
Subject: Re: Is it possible to log better info in timeouts?
Date: Thu, 2 Jan 2014 01:31:06 +0000 [thread overview]
Message-ID: <20140102013106.GA2106@dcvr.yhbt.net> (raw)
In-Reply-To: <20140102004150.GA11314@tydirium.the-staging.com>
Carlos Peñas <theistian@gmx.com> wrote:
> Recently we had an issue with an external service for an app served with
> unicorn. And some of our requests died with unicorn's timeout because a
> not properly secured call to this external service. We had 20s
> timeout but in production with ten worker and a big percent of urls
> hitting external services 20 secons of timeout sooner or later brings
> down the entire site.
>
> in the logs when this hapens this line is left for debugging:
>
> E, [2013-12-25T23:56:49.363426 #26324] ERROR -- : worker=2 PID:27956 timeout (21s > 20s), killing
> E, [2014-12-25T23:56:49.396005 #26324] ERROR -- : reaped #<Process::Status: pid 27956 SIGKILL (signal 9)> worker=2
>
>
> Is it possible to opt to print to the error log some other info perhaps
> the bactrace of the process or the offending request?
The SIGKILL sent by the master isn't avoidable/trapable in userspace,
the KILL-ed process has no chance to dump a backtrace.
> Even better whould be possible to hack a watchdog that could log "slow
> requests" like mysql slow_queries?
You need to do this in your application. The timeout in unicorn is a
last resort: http://unicorn.bogomips.org/Application_Timeouts.html
I don't encourage relying on the unicorn timeout, it's a hack for broken
libraries or fatal bugs in Ruby (which are rarer nowadays).
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
prev parent reply other threads:[~2014-01-02 1:31 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-01-02 0:41 Is it possible to log better info in timeouts? Carlos Peñas
2014-01-02 1:31 ` Eric Wong [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://yhbt.net/unicorn/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140102013106.GA2106@dcvr.yhbt.net \
--to=normalperson@yhbt.net \
--cc=mongrel-unicorn@rubyforge.org \
--cc=theistian@gmx.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).