From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS6939 64.71.128.0/18 X-Spam-Status: No, score=-1.9 required=3.0 tests=AWL,BAYES_00, MSGID_FROM_MTA_HEADER shortcircuit=no autolearn=unavailable version=3.3.2 Path: news.gmane.org!not-for-mail From: Eric Wong Newsgroups: gmane.comp.lang.ruby.raindrops.general Subject: Re: queued is always 0 Date: Tue, 29 Jun 2010 19:45:49 -0700 Message-ID: <20100630024549.GA9169@dcvr.yhbt.net> References: <20100629005007.GA25438@dcvr.yhbt.net> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: dough.gmane.org 1277865964 20195 80.91.229.12 (30 Jun 2010 02:46:04 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Wed, 30 Jun 2010 02:46:04 +0000 (UTC) To: raindrops@librelist.com Original-X-From: raindrops@librelist.com Wed Jun 30 04:46:02 2010 Return-path: Envelope-to: gclrrg-raindrops@m.gmane.org List-Archive: List-Help: List-Id: List-Post: List-Subscribe: List-Unsubscribe: Precedence: list Original-Sender: raindrops@librelist.com Xref: news.gmane.org gmane.comp.lang.ruby.raindrops.general:8 Archived-At: Received: from zedshaw.xen.prgmr.com ([64.71.167.205]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1OTnJJ-0007W2-Bc for gclrrg-raindrops@m.gmane.org; Wed, 30 Jun 2010 04:46:01 +0200 Received: from zedshaw.xen.prgmr.com (localhost [IPv6:::1]) by zedshaw.xen.prgmr.com (Postfix) with ESMTP id 581FC21B7EC for ; Wed, 30 Jun 2010 02:54:22 +0000 (UTC) Ryan King wrote: > On Mon, Jun 28, 2010 at 5:50 PM, Eric Wong wrote: > > Ryan King wrote: > >> I'm trying to see if raindrops will help us instrument our stack, so > >> I'm doing some tests. > >> > >> I have 16 unicorns running our app with rainbows and am hitting it > >> with ab with a concurrency of 100. However, according to > >> linux-tcp-listener-stats.rb I have 0 queued requests. Maybe I'm > >> missing something, but shouldn't there be queued requests somewhere > >> here? > > > > Hi Ryan, > > > > The Unicorns could be accept()-ing connections fast enough and the > > queued connections aren't noticeable[1]. > I did as you describe here and I got the expected results. > > I think I'm just confused at this point. We have 16 unicorns running, > with 100 clients, but no queued connections, which makes me assume > that we have accepted connections that we can't actually service yet. > Is this true? Not true with Unicorn. Unicorn won't accept connections it can't service, ever. It could be ab itself isn't able to keep up with the requests/responses. How fast are your response times? What happens when you run fewer Unicorn workers or increase concurrency with ab? How large are your responses? If you have small responses, Unicorn could've also finished writing the response to the socket buffers and accepted another connection by the time ab gets around to reading the socket. Under Linux, Unicorn (and Mongrel) also uses TCP_DEFER_ACCEPT with a value of 1 (second)[1], so it won't even register in the queue if ab opened the socket and didn't have a chance to write to it, yet... You could try commenting out the setsockopt call to set TCP_DEFER_ACCEPT in lib/unicorn/socket_helper.rb for testing and see if the queue jumps up, too.. I noticed that the usage string for linux-tcp-listener-stats.rb was wrong yesterday (and updated it in rainbows.git), but you can try running it in a tight delay loop with "-d ": ruby linux-tcp-listener-stats.rb -d 0.01 | awk '$3 != 0 { print $0}' The above will only print lines if there are queued connections. You can replace "-d 0.01" with a smaller number, but Raindrops (and the underlying tcpdiag kernel module) can only give a snapshot of the current queue size). [1] - I should actually make the defer timeout configurable, especially for Rainbows! nginx sets this to 60s and it seems to work fine. -- Eric Wong