unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
From: Pierre <oct@fotopedia.com>
To: unicorn list <mongrel-unicorn@rubyforge.org>
Subject: Re: Unicorn and HAProxy, 500 Internal errors after checks
Date: Thu, 2 Dec 2010 09:38:38 +0100	[thread overview]
Message-ID: <AANLkTimHCSGv9c-a9y+zN42_Gf+csTHQ+e5wh8OvKib9@mail.gmail.com> (raw)
In-Reply-To: <20101201195858.GD12001@dcvr.yhbt.net>

Hello,

On Wed, Dec 1, 2010 at 8:58 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Pierre <oct@fotopedia.com> wrote:
>> On Wed, Dec 1, 2010 at 5:52 PM, Eric Wong <normalperson@yhbt.net> wrote:
>> > Hi Pierre, HAProxy should be configured to send proper HTTP checks and
>> > not just TCP connection checks, the problem will go away then.
>>
>> I understood this could be fixed this way and we will probably do that
>> soon. However, I think this is also the responsibility of Unicorn not
>> to reply anything when there's no request or at least log the error
>> somewhere :)
>
> I'm not sure how Unicorn is actually replying to anything, does HAProxy
> write *anything* to the socket?
>

It does. To reproduce the behavior, start a unicorn, start a sniffing
tool (ngrep for example), connect to the unicorn and disconnect
immediately:

Connecting endpoint:

[08:22][virtual] root@infrabox:~# telnet localhost 2002
Trying 127.0.0.1...
Connected to infrabox.virtual.ftnz.net.
Escape character is '^]'.
^]
telnet> quit
Connection closed.

Sniffing Console:

[08:22][virtual] root@infrabox:~# ngrep "" port 2002 -d lo
interface: lo (127.0.0.0/255.0.0.0)
filter: (ip or ip6) and ( port 2002 )
#####
T 127.0.0.1:2002 -> 127.0.0.1:42874 [AFP]
  HTTP/1.1 500 Internal Server Error....
#


Further packet inspection reveals that the 500 Internal Server Error
is sent after the telnet has sent a FIN packet which means it will
just be discarded. I understand logging it could be a DoS vector, but
you must admit that seing 500's from the server without any obvious
reason can be frightening at first.

Re HAProxy, and to complete Laurence reply from my point of view,

> If you serve large responses that can't fit in kernel socket buffers,
> then Unicorn will get stuck writing out to a client that isn't reading
> fast enough.

That can definitely be a problem in that case. To work around that we
use a cache in the stack above the lower level of HAProxy (we have two
other levels of HAProxy in the stack).

Cheers,
-- 
Pierre <oct@fotopedia.com>
Server Shepherd at http://www.fotopedia.com/
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying


  parent reply	other threads:[~2010-12-02  9:05 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-12-01 11:59 Unicorn and HAProxy, 500 Internal errors after checks Pierre
2010-12-01 16:12 ` Clifton King
2010-12-01 17:02   ` Pierre
2010-12-01 16:52 ` Eric Wong
2010-12-01 17:14   ` Pierre
2010-12-01 19:58     ` Eric Wong
2010-12-02  0:42       ` Lawrence Pit
2010-12-02  4:59         ` Eric Wong
2010-12-02  8:38       ` Pierre [this message]
2010-12-02 17:39         ` Eric Wong
2010-12-03  8:41           ` Pierre
2010-12-04 23:38             ` Eric Wong
2010-12-05  1:04               ` russell muetzelfeldt
2010-12-06 18:39                 ` Eric Wong
2010-12-02 17:41     ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://yhbt.net/unicorn/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=AANLkTimHCSGv9c-a9y+zN42_Gf+csTHQ+e5wh8OvKib9@mail.gmail.com \
    --to=oct@fotopedia.com \
    --cc=mongrel-unicorn@rubyforge.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhbt.net/unicorn.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).