From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS14383 205.234.109.0/24 X-Spam-Status: No, score=-0.6 required=5.0 tests=MSGID_FROM_MTA_HEADER, RP_MATCHES_RCVD,T_DKIM_INVALID shortcircuit=no autolearn=unavailable version=3.3.2 Path: news.gmane.org!not-for-mail From: Andrew Grim Newsgroups: gmane.comp.lang.ruby.unicorn.general Subject: Re: anything left before 1.0? Date: Tue, 15 Jun 2010 18:05:41 -0700 Message-ID: References: <20100616000907.GA28484@dcvr.yhbt.net> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Trace: dough.gmane.org 1276651744 10463 80.91.229.12 (16 Jun 2010 01:29:04 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Wed, 16 Jun 2010 01:29:04 +0000 (UTC) To: unicorn list Original-X-From: mongrel-unicorn-bounces@rubyforge.org Wed Jun 16 03:29:00 2010 Return-path: Envelope-to: gclrug-mongrel-unicorn@m.gmane.org X-Original-To: mongrel-unicorn@rubyforge.org Delivered-To: mongrel-unicorn@rubyforge.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type:content-transfer-encoding; bh=6n1ucTI5IxtpfID98+slclYIWV8Whgi5jBmBj0lPSsI=; b=KN11yVvX7Dznp66tovwRot6bvsdQMVo213tTiRtIOdy+1qQAhM808ImiCaMnW9jNBV MW81HeaLTDJYW6R0PDCNg/Rd/gpj3sLnD8mHoiMZU2n7lkmR3CbdDg4Yi8aoqehqq8gy dkvtcOa0a3XpEJnwReUHOnO+swg2k2KtfAFck= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=q2ayhTWisU4qPYa8nfDtdJKgD/yVUIA5j7pu4Im0caBms5pxr4zcX5SoLu2tyh9MXk /ae5tyqONdKjhswdF+uh/nqQzdWvbl/yfN8uC8XZdqogjCIIsdv+HX17quQ+HZTga7uu SHra7D/yMuE9jzJw/aurD4AY/rP6aeFl9giqg= In-Reply-To: <20100616000907.GA28484@dcvr.yhbt.net> X-Google-Sender-Auth: urWF6pOljpJt0ioh8pU133RPBCI X-BeenThere: mongrel-unicorn@rubyforge.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: mongrel-unicorn-bounces@rubyforge.org Errors-To: mongrel-unicorn-bounces@rubyforge.org Xref: news.gmane.org gmane.comp.lang.ruby.unicorn.general:583 Archived-At: Received: from rubyforge.org ([205.234.109.19]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1OOhR4-0007cA-Av for gclrug-mongrel-unicorn@m.gmane.org; Wed, 16 Jun 2010 03:28:58 +0200 Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 080961858388; Tue, 15 Jun 2010 21:28:56 -0400 (EDT) Received: from mail-vw0-f50.google.com (mail-vw0-f50.google.com [209.85.212.50]) by rubyforge.org (Postfix) with ESMTP id 795A1185836B for ; Tue, 15 Jun 2010 21:05:42 -0400 (EDT) Received: by vws10 with SMTP id 10so8164390vws.23 for ; Tue, 15 Jun 2010 18:05:41 -0700 (PDT) Received: by 10.220.48.131 with SMTP id r3mr4358685vcf.221.1276650341195; Tue, 15 Jun 2010 18:05:41 -0700 (PDT) Received: by 10.220.166.205 with HTTP; Tue, 15 Jun 2010 18:05:41 -0700 (PDT) > Let us know if there's anything else missing, pipe up within the next 24 > hours or so... > > You can always email me privately if you don't want your address on the > public mailing list. =A0Don't send HTML mail, though. Hey Eric, I was hoping to spend some more time debugging myself, but since you were going to release 1.0 I thought I'd get your thoughts on this. Quick overview, I work for kongregate.com, one of the larger rails sites, and one of the only large sites (that I know of personally) running ruby 1.9. We are currently running mostly mongrels, but I've got one server testing Unicorn. Things are mostly great, we are seeing nearly 20ms improvement in average response time, which is awesome. Now to the issue. SYMPTOM: The master process is killing the workers fairly frequently based on the workers timing out. CAUSE: I've added some logging to get the backtrace when I send a SIGTERM, and it is always stuck on line 68 in http_response.rb: body.each { |chunk| socket.write(chunk) } I ran some straces, and here's an example of the last few lines where it gets killed: 06:50:23.239931 clock_gettime(CLOCK_REALTIME, {1276523423, 239967000}) =3D 0 <0.000172> 06:50:23.240213 write(12, "HTTP/1.1 200 OK\r\nDate: Mon, 14 J"..., 1896) =3D 1896 <0.000087> 06:50:23.242072 write(12, " 06:50:23.242230 select(13, NULL, [12], NULL, NULL 06:51:22.167122 +++ killed by SIGKILL +++ So it's writing and then (to my understanding) waiting on the socket to return, but you can see that for a full 60s it isn't. My best guess off-hand is that the large size of the string being written to the socket is causing an issue, and I have noticed that it is happening primarily on requests that return larger payloads. At the same time, it isn't that much data, so I'm a little surprised it would be an issue. I am planning on trying to split the body up into smaller chunks in a rack middleware or something. Or I could be totally off. Just wanted to see if you have any ideas, I'm not even sure this is a Unicorn issue, definitely could be ruby 1.9 bug too. Sorry about the long email, but I appreciate any help you can give. Thanks, Andrew _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying