From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: * X-Spam-ASN: AS14383 205.234.109.0/24 X-Spam-Status: No, score=1.0 required=3.0 tests=AWL,HK_RANDOM_FROM, MSGID_FROM_MTA_HEADER shortcircuit=no autolearn=no version=3.3.2 Path: news.gmane.org!not-for-mail From: Eric Wong Newsgroups: gmane.comp.lang.ruby.rainbows.general Subject: load balancing with keepalive is hard, but we try anyways Date: Thu, 20 Jan 2011 11:15:35 +0000 Message-ID: <20110120111535.GA27078@dcvr.yhbt.net> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: dough.gmane.org 1295524250 19423 80.91.229.12 (20 Jan 2011 11:50:50 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Thu, 20 Jan 2011 11:50:50 +0000 (UTC) To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Original-X-From: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Thu Jan 20 12:50:46 2011 Return-path: Envelope-to: gclrrg-rainbows-talk@m.gmane.org X-Original-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Delivered-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Content-Disposition: inline X-Motd: Rainbows! is too cool to be mainstream User-Agent: Mutt/1.5.18 (2008-05-17) X-BeenThere: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.rainbows.general:193 Archived-At: Received: from rubyforge.org ([205.234.109.19]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Pft2L-0002sK-N9 for gclrrg-rainbows-talk@m.gmane.org; Thu, 20 Jan 2011 12:50:45 +0100 Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 2CE83185838E for ; Thu, 20 Jan 2011 06:50:45 -0500 (EST) Received: from dcvr.yhbt.net (unknown [64.71.152.64]) by rubyforge.org (Postfix) with ESMTP id 65F4D18583AE for ; Thu, 20 Jan 2011 06:16:04 -0500 (EST) Received: from localhost (unknown [127.0.2.5]) by dcvr.yhbt.net (Postfix) with ESMTP id 784921F4E1; Thu, 20 Jan 2011 11:15:35 +0000 (UTC) (I've alluded to some of these problems in previous messages/release notes/commit messages, but I've been meaning to write more on this subject for ages, now, so here goes...) >>From what I can tell, all concurrency options suck at it. Native threads eat too much memory, event loops (that I've seen in the wild) don't utilize multicore effectively, yadda yadda... Disabling keepalive gives you the best load balancing for short request-response cycles: this is what Unicorn does (with help from nginx). It's crap for slow/trickle stuff (what Rainbows! is for) and also suboptimal for "hello world"-type applications that can do several thousand requests a second (Rainbows! can do this, too, sorta). And even in a perfect world where we could use teeny stacks, clone() and no GVL, it all goes out the door when you have more than one machine. How Rainbows! deals with keepalive load balancing now: * Short keepalive timeout by default (5s). This reduces the memory/cycles spent on idle clients. * keepalive requests limited to 100 by default. This is to prevent aggressive clients from monopolizing a single thread or process. I considered allowing this to be a range and random per-client, but other factors (other clients, machine load, GC, different request profiles) provide enough randomness in my testing that it wasn't worth the extra code. Both of these strategies force clients to reconnect more frequently than they need to, allowing them to migrate to a different, potentially less-loaded worker process/thread. It ends up being a see-saw effect that seems "good enough" load balancing to the mix but you're never in an ideal (nor terrible) situation for long. I'm working on yet another strategy, but it probably won't be useful immediately under Ruby 1.9 and it won't be useful for multiple hosts... Also keep in mind that hardly anybody uses nor needs Rainbows!, but these issues affect other servers in other languages and platforms, too. -- Eric Wong _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying