From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: * X-Spam-ASN: AS33070 50.56.128.0/17 X-Spam-Status: No, score=1.0 required=3.0 tests=AWL,HK_RANDOM_FROM, MSGID_FROM_MTA_HEADER,TVD_RCVD_IP shortcircuit=no autolearn=no version=3.3.2 Path: news.gmane.org!not-for-mail From: Eric Wong Newsgroups: gmane.comp.lang.ruby.rainbows.general Subject: Re: Unicorn is killing our rainbows workers Date: Fri, 27 Jul 2012 20:40:40 +0000 Message-ID: <20120727204040.GA2192@dcvr.yhbt.net> References: <20120719002641.GA17210@dcvr.yhbt.net> <20120719201633.GA8203@dcvr.yhbt.net> <20120719213125.GA17708@dcvr.yhbt.net> <20120726234845.GA29453@dcvr.yhbt.net> <20120727001125.GA30957@dcvr.yhbt.net> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: dough.gmane.org 1343421655 5967 80.91.229.3 (27 Jul 2012 20:40:55 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Fri, 27 Jul 2012 20:40:55 +0000 (UTC) Cc: Cody Fauser , ops , Harry Brundage , Jonathan Rudenberg To: Rainbows! list Original-X-From: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Fri Jul 27 22:40:54 2012 Return-path: Envelope-to: gclrrg-rainbows-talk@m.gmane.org X-Original-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Delivered-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) X-BeenThere: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.rainbows.general:388 Archived-At: Received: from 50-56-192-79.static.cloud-ips.com ([50.56.192.79] helo=rubyforge.org) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1SurL6-0004DX-TQ for gclrrg-rainbows-talk@m.gmane.org; Fri, 27 Jul 2012 22:40:49 +0200 Received: from localhost.localdomain (localhost [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id CA08B2E05F; Fri, 27 Jul 2012 20:40:47 +0000 (UTC) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by rubyforge.org (Postfix) with ESMTP id 688712E05F for ; Fri, 27 Jul 2012 20:40:42 +0000 (UTC) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 413331F5B5; Fri, 27 Jul 2012 20:40:41 +0000 (UTC) Samuel Kadolph wrote: > On Thu, Jul 26, 2012 at 8:11 PM, Eric Wong wrote: > >> Our ops guys have been busy so I don't have the output from lsof but > >> it didn't look like it was spawning any extra threads or opening any > >> unexplainable connections. But I think we should have been checking > >> the worker processes and not the master, right? > > > > Definitely check the master, too. It's the master that seems to > > believe it's suspended, so that makes me believe something is wrong > > with the master (and this is likely due to preload_app). > > > >> Haven't tried disabling preload_app yet but we have tried > > I've got the output of lsof and ls at https://gist.github.com/3190171. Thanks, that's the output for the master? I don't see anything obviously wrong. I seem to recall the Ruby library responsible for the following log file also spawns its own background thread, but your "ls" only shows 2 tasks (instead of 3): > ruby 26564 root 9w REG 202,1 51221 529742 APP_PATH/shared/log/newrelic_agent.log > $ ls /proc/26564/task/ > 26564 27052 (While the Ruby code for the module responsible for that log file is technically "open", it's not Free, so I'm not comfortable looking at that code). _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying