From: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
To: Rainbows! list <rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org>
Subject: Re: Rainbows! + EventMachine + Sinatra::Synchrony == pegged CPU when idle?
Date: Mon, 23 Jul 2012 17:21:40 -0700 [thread overview]
Message-ID: <20120724002140.GA25177@dcvr.yhbt.net> (raw)
In-Reply-To: <CAAsVqN91_KiexPc4u9byuVstyknj7TgT4V7evs1x65W+6izzkQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
Ilya Maykov <ivmaykov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Hi Eric,
>
> Sorry for the delayed response. I've added inline answers to your
> questions. We've since resolved this issue by disabling keepalive in
> our rainbows config. So, this probably had to do with the keepalive
> implementation either in Rainbows itself or in the base Unicorn code.
> Answers to your other questions are inlined below.
Thank you very much for the follow-up (I wish more folks would do this
:)
If Rainbows! is using EventMachine, it'll use the
EM.set_comm_inactivity_timeout method in EventMachine.
> On Tue, Jun 19, 2012 at 10:54 AM, Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> > Ilya Maykov <ivmaykov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > Can you also try "keepalive_timeout 0" to disable keepalive? (EM
> > handles it internally, but I'm not sure how well)
> >
>
> This turned out to be the problem. Disabling keepalive got rid of the
> CPU pegging. Surprisingly, it also made our average latency drop from
> about 50 ms to about 20 ms per request, even though every request now
> has to negotiate a connection handshake. So, we're just going to keep
> it disabled for now. The bug may be inside the keepalive code in
> Rainbows or Unicorn (not familiar with the codebase so not sure where
> that code lives).
Are you setting Content-Length or "Transfer-Encoding: chunked" in
responses? Rack::ContentLength or Rack::Chunked middleware might need
to be loaded if your framework doesn't already include it.
Lack of these headers may confuse clients, even... Where did you
measure the 50 -> 20ms latency drop, from the client?
About the latency drop:
Was :tcp_cork enabled in your listen directive? I wonder if there's
some bad interaction with :tcp_cork + EM which might explain the
latency (but not the CPU usage). Disabling keepalive would force data
out immediately and avoid any bad effects of :tcp_cork.j
Can you also try listen() with :tcp_defer_accept => 0? That might
help if you're accept()-ing many connections at once.
Anyways, the TCP connection handshake is very fast on LANs/localhost,
but hurts on high-latency connections (I seem to remember mainstream web
browsers double simultaneous requests to compensate for lack of
keepalive).
Which version of EM are you using? The keepalive implementation for
EM+Rainbows! is entirely handled by EM.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
prev parent reply other threads:[~2012-07-24 0:21 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-06-19 9:06 Rainbows! + EventMachine + Sinatra::Synchrony == pegged CPU when idle? Ilya Maykov
[not found] ` <CAAsVqN8=HxMbbQkfonojsoCHgJAPsDhC7hqqcBK9GnYsO2ww7g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-06-19 17:54 ` Eric Wong
[not found] ` <20120619175409.GA27303-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-07-23 23:34 ` Ilya Maykov
[not found] ` <CAAsVqN91_KiexPc4u9byuVstyknj7TgT4V7evs1x65W+6izzkQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-07-24 0:21 ` Eric Wong [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://yhbt.net/rainbows/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120724002140.GA25177@dcvr.yhbt.net \
--to=normalperson-rmlxzr9ms24@public.gmane.org \
--cc=rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
http://yhbt.net/rainbows.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).