raindrops RubyGem user+dev discussion/patches/pulls/bugs/help
 help / color / Atom feed
From: Eric Wong <normalperson@yhbt.net>
To: raindrops@librelist.com
Subject: Re: queued is always 0
Date: Tue, 29 Jun 2010 19:45:49 -0700
Message-ID: <20100630024549.GA9169@dcvr.yhbt.net> (raw)
In-Reply-To: <AANLkTimHNhnu7vkzo7yu3IK4ODIhoYRvm8zmaBepm0SA@mail.gmail.com>

Ryan King <ryan@twitter.com> wrote:
> On Mon, Jun 28, 2010 at 5:50 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Ryan King <ryan@twitter.com> wrote:
> >> I'm trying to see if raindrops will help us instrument our stack, so
> >> I'm doing some tests.
> >>
> >> I have 16 unicorns running our app with rainbows and am hitting it
> >> with ab with a concurrency of 100. However, according to
> >> linux-tcp-listener-stats.rb I have 0 queued requests. Maybe I'm
> >> missing something, but shouldn't there be queued requests somewhere
> >> here?
> >
> > Hi Ryan,
> >
> > The Unicorns could be accept()-ing connections fast enough and the
> > queued connections aren't noticeable[1].


> I did as you describe here and I got the expected results.
> I think I'm just confused at this point. We have 16 unicorns running,
> with 100 clients, but no queued connections, which makes me assume
> that we have accepted connections that we can't actually service yet.
> Is this true?

Not true with Unicorn.  Unicorn won't accept connections it can't
service, ever.

It could be ab itself isn't able to keep up with the requests/responses.

How fast are your response times?

What happens when you run fewer Unicorn workers or increase concurrency
with ab?

How large are your responses?

If you have small responses, Unicorn could've also finished writing the
response to the socket buffers and accepted another connection by the
time ab gets around to reading the socket.

Under Linux, Unicorn (and Mongrel) also uses TCP_DEFER_ACCEPT with a
value of 1 (second)[1], so it won't even register in the queue if
ab opened the socket and didn't have a chance to write to it, yet...

You could try commenting out the setsockopt call to set
TCP_DEFER_ACCEPT in lib/unicorn/socket_helper.rb for testing
and see if the queue jumps up, too..

I noticed that the usage string for linux-tcp-listener-stats.rb was
wrong yesterday (and updated it in rainbows.git), but you can try
running it in a tight delay loop with "-d <seconds>":

  ruby linux-tcp-listener-stats.rb -d 0.01 | awk '$3 != 0 { print $0}'

The above will only print lines if there are queued connections.  You
can replace "-d 0.01" with a smaller number, but Raindrops (and the
underlying tcpdiag kernel module) can only give a snapshot of the
current queue size).

[1] - I should actually make the defer timeout configurable, especially
      for Rainbows!  nginx sets this to 60s and it seems to work fine.

Eric Wong

  reply index

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-06-29  0:26 Ryan King
2010-06-29  0:50 ` Eric Wong
2010-06-30  1:12   ` Ryan King
2010-06-30  2:45     ` Eric Wong [this message]
2010-06-30 18:51       ` Ryan King
2010-06-30 23:18         ` Ryan King
2010-07-01 10:18         ` Eric Wong
2010-07-01 18:37           ` Ryan King
2010-07-02  2:51             ` Eric Wong

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information: https://yhbt.net/raindrops/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20100630024549.GA9169@dcvr.yhbt.net \
    --to=normalperson@yhbt.net \
    --cc=raindrops@librelist.com \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

raindrops RubyGem user+dev discussion/patches/pulls/bugs/help

Archives are clonable:
	git clone --mirror https://yhbt.net/raindrops-public
	git clone --mirror http://ou63pmih66umazou.onion/raindrops-public

Example config snippet for mirrors

Newsgroups are available over NNTP:

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git