Rainbows! Rack HTTP server user/dev discussion
 help / color / mirror / code / Atom feed
* load balancing with keepalive is hard, but we try anyways
@ 2011-01-20 11:15 Eric Wong
       [not found] ` <20110120111535.GA27078-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
  0 siblings, 1 reply; 2+ messages in thread
From: Eric Wong @ 2011-01-20 11:15 UTC (permalink / raw)
  To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw

(I've alluded to some of these problems in previous messages/release
notes/commit messages, but I've been meaning to write more on this
subject for ages, now, so here goes...)

>From what I can tell, all concurrency options suck at it.  Native
threads eat too much memory, event loops (that I've seen in the wild)
don't utilize multicore effectively, yadda yadda...

Disabling keepalive gives you the best load balancing for short
request-response cycles: this is what Unicorn does (with help from
nginx).  It's crap for slow/trickle stuff (what Rainbows! is for) and
also suboptimal for "hello world"-type applications that can do several
thousand requests a second (Rainbows! can do this, too, sorta).

And even in a perfect world where we could use teeny stacks, clone() and
no GVL, it all goes out the door when you have more than one machine.

How Rainbows! deals with keepalive load balancing now:

* Short keepalive timeout by default (5s).  This reduces the
  memory/cycles spent on idle clients.

* keepalive requests limited to 100 by default. This is to prevent
  aggressive clients from monopolizing a single thread or process.
  I considered allowing this to be a range and random per-client, but
  other factors (other clients, machine load, GC, different request
  profiles) provide enough randomness in my testing that it wasn't
  worth the extra code.

Both of these strategies force clients to reconnect more frequently than
they need to, allowing them to migrate to a different, potentially
less-loaded worker process/thread.  It ends up being a see-saw effect
that seems "good enough" load balancing to the mix but you're never in
an ideal (nor terrible) situation for long.

I'm working on yet another strategy, but it probably won't be useful
immediately under Ruby 1.9 and it won't be useful for multiple hosts...


Also keep in mind that hardly anybody uses nor needs Rainbows!, but
these issues affect other servers in other languages and platforms, too.

-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2011-01-20 11:51 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-01-20 11:15 load balancing with keepalive is hard, but we try anyways Eric Wong
     [not found] ` <20110120111535.GA27078-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2011-01-20 11:34   ` Eric Wong

Code repositories for project(s) associated with this inbox:

	../../../rainbows.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).