Re: nginx reverse proxy getting ECONNRESET

[Michael Fischer <mfischer@zendesk.com>]

On Tue, Mar 24, 2015 at 10:54 PM, Eric Wong <e@80x24.org> wrote:
> Michael Fischer <mfischer@zendesk.com> wrote:
>> We have an nginx 1.6.2 proxy in front of a Unicorn 4.8.3 server that
>> is frequently reporting the following error:
>>
>> 2015/03/24 01:46:01 [error] 11217#0: *872231 readv() failed (104:
>> Connection reset by peer) while reading upstream
>>
>> The interesting things are:
>>
>> 1) The upstream is a Unix domain socket (to which Unicorn is bound)
>> 2) Unicorn isn't reporting that a child died in the error log (and I
>> verified their lifetimes with ps(1))
>>
>> Any hints as to what we should look for?
>
> What changed recently with your setup?

We upgraded nginx from 1.4.7 to 1.6.2.  The frequency of the error has
increased significantly since.  But I hesitate to point the finger at
nginx without more evidence, since its developers are very skilled.

> Which OS/kernel version + vendor version?

uname -3.13.0-40-generic #69~precise1-Ubuntu

Ruby 2.1.1

> Can you setup a test instance on a different nginx port/unicorn socket
> and with a config.ru such as:
>
> ------------------------- 8< ----------------------
> run(lambda do |env|
>   $stderr.write("#$$ starting at #{Time.now}\n")
>   # be sure to configure your unicorn timeout
>   sleep
>   # should not return, wait for unicorn to kill
> end)
> ----------------------------------------------------
>
> And hitting nginx with a single test request to reproduce the issue.

I'll take that step later if I have to, but I'm not sure what evidence
that would provide, since we're not having timeout issues -- when this
happens, the response time reported by nginx is usually just a few
seconds (Unicorn timeout is 90 seconds),

Thanks,

--Michael