Rainbows! Rack HTTP server user/dev discussion
 help / color / mirror / code / Atom feed
From: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
Subject: load balancing with keepalive is hard, but we try anyways
Date: Thu, 20 Jan 2011 11:15:35 +0000	[thread overview]
Message-ID: <20110120111535.GA27078@dcvr.yhbt.net> (raw)

(I've alluded to some of these problems in previous messages/release
notes/commit messages, but I've been meaning to write more on this
subject for ages, now, so here goes...)

>From what I can tell, all concurrency options suck at it.  Native
threads eat too much memory, event loops (that I've seen in the wild)
don't utilize multicore effectively, yadda yadda...

Disabling keepalive gives you the best load balancing for short
request-response cycles: this is what Unicorn does (with help from
nginx).  It's crap for slow/trickle stuff (what Rainbows! is for) and
also suboptimal for "hello world"-type applications that can do several
thousand requests a second (Rainbows! can do this, too, sorta).

And even in a perfect world where we could use teeny stacks, clone() and
no GVL, it all goes out the door when you have more than one machine.

How Rainbows! deals with keepalive load balancing now:

* Short keepalive timeout by default (5s).  This reduces the
  memory/cycles spent on idle clients.

* keepalive requests limited to 100 by default. This is to prevent
  aggressive clients from monopolizing a single thread or process.
  I considered allowing this to be a range and random per-client, but
  other factors (other clients, machine load, GC, different request
  profiles) provide enough randomness in my testing that it wasn't
  worth the extra code.

Both of these strategies force clients to reconnect more frequently than
they need to, allowing them to migrate to a different, potentially
less-loaded worker process/thread.  It ends up being a see-saw effect
that seems "good enough" load balancing to the mix but you're never in
an ideal (nor terrible) situation for long.

I'm working on yet another strategy, but it probably won't be useful
immediately under Ruby 1.9 and it won't be useful for multiple hosts...

Also keep in mind that hardly anybody uses nor needs Rainbows!, but
these issues affect other servers in other languages and platforms, too.

Eric Wong
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
Do not quote signatures (like this one) or top post when replying

             reply	other threads:[~2011-01-20 11:50 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-01-20 11:15 Eric Wong [this message]
     [not found] ` <20110120111535.GA27078-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2011-01-20 11:34   ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information: https://yhbt.net/rainbows/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110120111535.GA27078@dcvr.yhbt.net \
    --to=normalperson-rmlxzr9ms24@public.gmane.org \
    --cc=rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this inbox:


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).