From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS14383 205.234.109.0/24 X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,MSGID_FROM_MTA_HEADER, RP_MATCHES_RCVD,T_DKIM_INVALID shortcircuit=no autolearn=unavailable version=3.3.2 Path: news.gmane.org!not-for-mail From: Jamie Wilkinson Newsgroups: gmane.comp.lang.ruby.unicorn.general Subject: Re: scaling unicorn Date: Mon, 21 Jun 2010 19:34:33 -0700 Message-ID: References: <20100622001632.GA10082@dcvr.yhbt.net> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable X-Trace: dough.gmane.org 1277174622 9848 80.91.229.12 (22 Jun 2010 02:43:42 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Tue, 22 Jun 2010 02:43:42 +0000 (UTC) To: unicorn list Original-X-From: mongrel-unicorn-bounces@rubyforge.org Tue Jun 22 04:43:39 2010 Return-path: Envelope-to: gclrug-mongrel-unicorn@m.gmane.org X-Original-To: mongrel-unicorn@rubyforge.org Delivered-To: mongrel-unicorn@rubyforge.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:content-type :mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; bh=DVg+z+iHWMvWqE5K23KcHiryOJ+4RRqsD+FAY9qM8J4=; b=NSTAeq1JsOFqCBrFFym0mVJR5ayk/Bj+eLZPZZlWTPwxTEHuigrHLwrX6F9RNqJNG5 PH/mHg8Km6Sy84EwSewW3mxNfLc5UMhU3xtONQlDDm+dnbBsr8abi0zwzy+c9neW7PHC mEl1d2f2yV38h5cbPcXeUADR6rXt8334Ll8aw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; b=ZOzFl9Xug4EbTckpp1VfsF3NndPRL+C2CBF4TtN9dtR8zcm07Q5cPG6yUhxTZbOE6q tL/t9XB7MUzggTFdMQd1+U/AynfRaJkD6tgCiQuZgcha9vzi/4kBjUbmutZdoDH+c4fb 6UqUGbGVwKBK7YyAX3UO2It2pkom+gmO2uFgU= In-Reply-To: <20100622001632.GA10082@dcvr.yhbt.net> X-Mailer: Apple Mail (2.1078) X-BeenThere: mongrel-unicorn@rubyforge.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: mongrel-unicorn-bounces@rubyforge.org Errors-To: mongrel-unicorn-bounces@rubyforge.org Xref: news.gmane.org gmane.comp.lang.ruby.unicorn.general:598 Archived-At: Received: from rubyforge.org ([205.234.109.19]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1OQtSd-0000pj-3U for gclrug-mongrel-unicorn@m.gmane.org; Tue, 22 Jun 2010 04:43:39 +0200 Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 7C67B1858344; Mon, 21 Jun 2010 22:43:38 -0400 (EDT) Received: from mail-px0-f178.google.com (mail-px0-f178.google.com [209.85.212.178]) by rubyforge.org (Postfix) with ESMTP id 3229E1858344 for ; Mon, 21 Jun 2010 22:34:36 -0400 (EDT) Received: by pxi3 with SMTP id 3so18696pxi.23 for ; Mon, 21 Jun 2010 19:34:36 -0700 (PDT) Received: by 10.115.66.30 with SMTP id t30mr4844016wak.161.1277174076039; Mon, 21 Jun 2010 19:34:36 -0700 (PDT) Received: from [10.0.1.31] (173-13-169-18-sfba.hfc.comcastbusiness.net [173.13.169.18]) by mx.google.com with ESMTPS id h9sm23068264wal.12.2010.06.21.19.34.34 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 21 Jun 2010 19:34:35 -0700 (PDT) On Jun 21, 2010, at 5:16 PM, Eric Wong wrote: >> overloading the server or hitting memory bandwidth issues. The >> backlog is at the somaxconn default of 128, I'm still not sure if we >> will bump that up or not. > = > The default backlog we try to specify is actually 1024 (same as > Mongrel). But it's always a murky value anyways, as it's > kernel/sysctl-dependent. With Unix domain sockets, some folks use > crazy values like 2048 to look better on synthetic benchmarks :) Somewhat related -- I've been meaning to discuss the finer points of backlo= g tuning. I've been experimenting with the multi-server socket+TCP megaunicorn config= uration from your CDT: http://rubyforge.org/pipermail/mongrel-unicorn/2009-September/000033.html Which I think is what this sentence from TUNING is talking about? "Setting a very low value for the :backlog parameter in =93listen=94 direc= tives can allow failover to happen more quickly if your cluster is configur= ed for it." Our app can catch a batch of requests which will be slow (1-3s), and these = can pool on one individual server in our load-balanced EC2 cluster -- exact= ly the case for the multi-server failover setup. = I've put this into production under a healthy load (5000+ RPM) and it appea= rs to work really well! Produces very high requests/s rates at significant= ly higher concurrency than without, and serves zero 502 errors (part of the= goal) I currently I have the unix socket set to a backlog of 64, then failing ove= r to a TCP listener using backlog 1024 (so that things are queued rather th= an 502'd) I can imagine there might be a case for keeping the TCP backlog low as well= & serving errors when overloaded, rather than getting caught in an unrecov= erable back-queue tarpit I'm currently failing-over to a dedicated "backup" instance, so that I coul= d measure exactly how much traffic is being offloaded. This means my benchm= arks w/o failover are 1 server, but with failover is actually 2 servers. We= 're reconfiguring to something more like the original diagram at which poin= t I'll do some cluster-wide stress-tests & share data/scripts/process. = BTW, this configuration needs a cool name! -jamie http://jamiedubs.com http://fffff.at _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying