From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS33070 50.56.128.0/17 X-Spam-Status: No, score=-0.2 required=5.0 tests=AWL shortcircuit=no autolearn=unavailable version=3.3.2 X-Original-To: archivist@yhbt.net Delivered-To: archivist@dcvr.yhbt.net Received: from rubyforge.org (50-56-192-79.static.cloud-ips.com [50.56.192.79]) by dcvr.yhbt.net (Postfix) with ESMTP id 32C0B32B5D for ; Thu, 29 Nov 2012 23:32:18 +0000 (UTC) Received: from localhost.localdomain (localhost [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 369952E07F; Thu, 29 Nov 2012 23:32:19 +0000 (UTC) X-Original-To: mongrel-unicorn@rubyforge.org Delivered-To: mongrel-unicorn@rubyforge.org Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by rubyforge.org (Postfix) with ESMTP id 907572E079 for ; Thu, 29 Nov 2012 23:32:11 +0000 (UTC) Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id C972C32B28; Thu, 29 Nov 2012 23:32:08 +0000 (UTC) Date: Thu, 29 Nov 2012 23:32:08 +0000 From: Eric Wong To: unicorn list Subject: Re: Fwd: Maintaining capacity during deploys Message-ID: <20121129233208.GB2618@dcvr.yhbt.net> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: mongrel-unicorn@rubyforge.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: mongrel-unicorn-bounces@rubyforge.org Errors-To: mongrel-unicorn-bounces@rubyforge.org Tony Arcieri wrote: > We're using unicornctl restart with the default before/after hook > behavior, which is to reap old Unicorn workers via SIGQUIT after the > new one has finished booting. > > Unfortunately, while the new workers are forking and begin processing > requests, we're still seeing significant spikes in our haproxy request > queue. It seems as if after we restart, the unwarmed workers get > swamped by the incoming requests. As far as I can tell, the momentary > loss of capacity we experience translates fairly quickly into a > thundering herd. > > We've experimented with rolling restarts at the server level but these > do not resolve the problem. So it's one haproxy -> multiple nginx/unicorn servers? Do you mark the server down or lower the weight in haproxy when deploying the Ruby app? (Perhaps using monitor-uri in haproxy) If the server is down to haproxy, but still up, you can send some warmup requests to the server before enabling the monitor-uri for haproxy. _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying