raindrops RubyGem user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
From: Eric Wong <normalperson@yhbt.net>
To: raindrops@librelist.com
Subject: Re: queued is always 0
Date: Tue, 29 Jun 2010 19:45:49 -0700	[thread overview]
Message-ID: <20100630024549.GA9169@dcvr.yhbt.net> (raw)
In-Reply-To: AANLkTimHNhnu7vkzo7yu3IK4ODIhoYRvm8zmaBepm0SA@mail.gmail.com

Ryan King <ryan@twitter.com> wrote:
> On Mon, Jun 28, 2010 at 5:50 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Ryan King <ryan@twitter.com> wrote:
> >> I'm trying to see if raindrops will help us instrument our stack, so
> >> I'm doing some tests.
> >>
> >> I have 16 unicorns running our app with rainbows and am hitting it
> >> with ab with a concurrency of 100. However, according to
> >> linux-tcp-listener-stats.rb I have 0 queued requests. Maybe I'm
> >> missing something, but shouldn't there be queued requests somewhere
> >> here?
> >
> > Hi Ryan,
> >
> > The Unicorns could be accept()-ing connections fast enough and the
> > queued connections aren't noticeable[1].

<snip>

> I did as you describe here and I got the expected results.
> 
> I think I'm just confused at this point. We have 16 unicorns running,
> with 100 clients, but no queued connections, which makes me assume
> that we have accepted connections that we can't actually service yet.
> Is this true?

Not true with Unicorn.  Unicorn won't accept connections it can't
service, ever.

It could be ab itself isn't able to keep up with the requests/responses.

How fast are your response times?

What happens when you run fewer Unicorn workers or increase concurrency
with ab?

How large are your responses?

If you have small responses, Unicorn could've also finished writing the
response to the socket buffers and accepted another connection by the
time ab gets around to reading the socket.

Under Linux, Unicorn (and Mongrel) also uses TCP_DEFER_ACCEPT with a
value of 1 (second)[1], so it won't even register in the queue if
ab opened the socket and didn't have a chance to write to it, yet...

You could try commenting out the setsockopt call to set
TCP_DEFER_ACCEPT in lib/unicorn/socket_helper.rb for testing
and see if the queue jumps up, too..


I noticed that the usage string for linux-tcp-listener-stats.rb was
wrong yesterday (and updated it in rainbows.git), but you can try
running it in a tight delay loop with "-d <seconds>":

  ruby linux-tcp-listener-stats.rb -d 0.01 | awk '$3 != 0 { print $0}'

The above will only print lines if there are queued connections.  You
can replace "-d 0.01" with a smaller number, but Raindrops (and the
underlying tcpdiag kernel module) can only give a snapshot of the
current queue size).



[1] - I should actually make the defer timeout configurable, especially
      for Rainbows!  nginx sets this to 60s and it seems to work fine.

-- 
Eric Wong


  reply	other threads:[~2010-06-30  2:46 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-06-29  0:26 queued is always 0 Ryan King
2010-06-29  0:50 ` Eric Wong
2010-06-30  1:12   ` Ryan King
2010-06-30  2:45     ` Eric Wong [this message]
2010-06-30 18:51       ` Ryan King
2010-06-30 23:18         ` Ryan King
2010-07-01 10:18         ` Eric Wong
2010-07-01 18:37           ` Ryan King
2010-07-02  2:51             ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://yhbt.net/raindrops/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20100630024549.GA9169@dcvr.yhbt.net \
    --to=normalperson@yhbt.net \
    --cc=raindrops@librelist.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	http://yhbt.net/raindrops.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).