From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: * X-Spam-ASN: AS33070 50.56.128.0/17 X-Spam-Status: No, score=1.1 required=3.0 tests=AWL,HK_RANDOM_FROM, MSGID_FROM_MTA_HEADER,PLING_QUERY,TVD_RCVD_IP shortcircuit=no autolearn=no version=3.3.2 Path: news.gmane.org!not-for-mail From: Eric Wong Newsgroups: gmane.comp.lang.ruby.rainbows.general Subject: Re: Rainbows! + EventMachine + Sinatra::Synchrony == pegged CPU when idle? Date: Mon, 23 Jul 2012 17:21:40 -0700 Message-ID: <20120724002140.GA25177@dcvr.yhbt.net> References: <20120619175409.GA27303@dcvr.yhbt.net> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: dough.gmane.org 1343089316 17505 80.91.229.3 (24 Jul 2012 00:21:56 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Tue, 24 Jul 2012 00:21:56 +0000 (UTC) To: Rainbows! list Original-X-From: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Tue Jul 24 02:21:56 2012 Return-path: Envelope-to: gclrrg-rainbows-talk@m.gmane.org X-Original-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Delivered-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.rainbows.general:383 Archived-At: Received: from 50-56-192-79.static.cloud-ips.com ([50.56.192.79] helo=rubyforge.org) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1StSsp-00027m-1c for gclrrg-rainbows-talk@m.gmane.org; Tue, 24 Jul 2012 02:21:51 +0200 Received: from localhost.localdomain (localhost [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id A7DBC2E067; Tue, 24 Jul 2012 00:21:49 +0000 (UTC) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by rubyforge.org (Postfix) with ESMTP id 1E0922E067 for ; Tue, 24 Jul 2012 00:21:40 +0000 (UTC) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 37DB01F438; Tue, 24 Jul 2012 00:21:40 +0000 (UTC) Ilya Maykov wrote: > Hi Eric, > > Sorry for the delayed response. I've added inline answers to your > questions. We've since resolved this issue by disabling keepalive in > our rainbows config. So, this probably had to do with the keepalive > implementation either in Rainbows itself or in the base Unicorn code. > Answers to your other questions are inlined below. Thank you very much for the follow-up (I wish more folks would do this :) If Rainbows! is using EventMachine, it'll use the EM.set_comm_inactivity_timeout method in EventMachine. > On Tue, Jun 19, 2012 at 10:54 AM, Eric Wong wrote: > > Ilya Maykov wrote: > > Can you also try "keepalive_timeout 0" to disable keepalive? (EM > > handles it internally, but I'm not sure how well) > > > > This turned out to be the problem. Disabling keepalive got rid of the > CPU pegging. Surprisingly, it also made our average latency drop from > about 50 ms to about 20 ms per request, even though every request now > has to negotiate a connection handshake. So, we're just going to keep > it disabled for now. The bug may be inside the keepalive code in > Rainbows or Unicorn (not familiar with the codebase so not sure where > that code lives). Are you setting Content-Length or "Transfer-Encoding: chunked" in responses? Rack::ContentLength or Rack::Chunked middleware might need to be loaded if your framework doesn't already include it. Lack of these headers may confuse clients, even... Where did you measure the 50 -> 20ms latency drop, from the client? About the latency drop: Was :tcp_cork enabled in your listen directive? I wonder if there's some bad interaction with :tcp_cork + EM which might explain the latency (but not the CPU usage). Disabling keepalive would force data out immediately and avoid any bad effects of :tcp_cork.j Can you also try listen() with :tcp_defer_accept => 0? That might help if you're accept()-ing many connections at once. Anyways, the TCP connection handshake is very fast on LANs/localhost, but hurts on high-latency connections (I seem to remember mainstream web browsers double simultaneous requests to compensate for lack of keepalive). Which version of EM are you using? The keepalive implementation for EM+Rainbows! is entirely handled by EM. _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying