unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
From: Eric Wong <normalperson@yhbt.net>
To: unicorn list <mongrel-unicorn@rubyforge.org>
Subject: Re: Unicorn and HAProxy, 500 Internal errors after checks
Date: Wed, 1 Dec 2010 19:58:59 +0000	[thread overview]
Message-ID: <20101201195858.GD12001@dcvr.yhbt.net> (raw)
In-Reply-To: <AANLkTimHFP7H+2FoEoe48gky4s8Z_BSUrmFLV41juP5F@mail.gmail.com>

Pierre <oct@fotopedia.com> wrote:
> On Wed, Dec 1, 2010 at 5:52 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Hi Pierre, HAProxy should be configured to send proper HTTP checks and
> > not just TCP connection checks, the problem will go away then.
> I understood this could be fixed this way and we will probably do that
> soon. However, I think this is also the responsibility of Unicorn not
> to reply anything when there's no request or at least log the error
> somewhere :)

I'm not sure how Unicorn is actually replying to anything, does HAProxy
write *anything* to the socket?

Logging those bad connections is another DoS vector I'd rather avoid,
and for connections where nothing is written, not even possible...

If you have the TCP_DEFER_ACCEPT (Linux) or accept filters (FreeBSD),
it's highly likely the Unicorn would never see the connection if the
client never wrote to it.

> > Also, I
> > can not recommend HAProxy unless you're certain all your clients are on
> > a LAN and can be trusted to never trickle uploads nor reading large
> > responses.
> While I understand that uploads are very complicated to handle on the
> stack (even nginx can be confused at upload sometimes), HAProxy proved
> it was very good at managing tons of connections and high volume
> traffic from the Internet. All the more so as it allows a very high
> level of redundancy at a very small cost that cannot be achieved
> simply otherwise. Do you have any pointers about your worrying
> non-recommendation of HAProxy ?

HAProxy starts writing request bodies to Unicorn as soon as the upload
starts, which means the Unicorn process will be bounded by the speed of
the original client connection.  If multiple clients upload slowly,
then you'll end up hogging many Unicorn workers.

nginx can also limit the size of client uploads (default 1M) to prevent
unnecessary I/O.

If you serve large responses that can't fit in kernel socket buffers,
then Unicorn will get stuck writing out to a client that isn't reading
fast enough.

AFAIK, HAProxy also does not yet maintain keep-alive connections to
clients, whereas nginx does. Keep-alive is important to client browsers,
they can halve their active connections to a site if keep-alive is

Eric Wong
Unicorn mailing list - mongrel-unicorn@rubyforge.org
Do not quote signatures (like this one) or top post when replying

  reply	other threads:[~2010-12-01 20:22 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-12-01 11:59 Unicorn and HAProxy, 500 Internal errors after checks Pierre
2010-12-01 16:12 ` Clifton King
2010-12-01 17:02   ` Pierre
2010-12-01 16:52 ` Eric Wong
2010-12-01 17:14   ` Pierre
2010-12-01 19:58     ` Eric Wong [this message]
2010-12-02  0:42       ` Lawrence Pit
2010-12-02  4:59         ` Eric Wong
2010-12-02  8:38       ` Pierre
2010-12-02 17:39         ` Eric Wong
2010-12-03  8:41           ` Pierre
2010-12-04 23:38             ` Eric Wong
2010-12-05  1:04               ` russell muetzelfeldt
2010-12-06 18:39                 ` Eric Wong
2010-12-02 17:41     ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information: https://yhbt.net/unicorn/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20101201195858.GD12001@dcvr.yhbt.net \
    --to=normalperson@yhbt.net \
    --cc=mongrel-unicorn@rubyforge.org \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).