* "Read error: #<IOError: closed stream>" and uploads @ 2011-01-23 21:28 Alexey Bondar 2011-01-24 0:59 ` Eric Wong 0 siblings, 1 reply; 8+ messages in thread From: Alexey Bondar @ 2011-01-23 21:28 UTC (permalink / raw) To: mongrel-unicorn Hi. Unicorn 3.3.1 fails with "Read error: #<IOError: closed stream>" on request with attachments bigger than ~150kb. Ruby: 1.9.2-p0 Rails: 3.0.1 Full stack trace: http://pastie.org/private/w1vwhbhdm9xtwwnpqog6wa Configuration: http://pastie.org/private/hdpiqs0d3xj3qmsjwg I use nginx 0.8.x as frontend server. Nginx client_body_buffer_size is 128mb. Any ides why this happens? _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: "Read error: #<IOError: closed stream>" and uploads 2011-01-23 21:28 "Read error: #<IOError: closed stream>" and uploads Alexey Bondar @ 2011-01-24 0:59 ` Eric Wong [not found] ` <20110124181919.GA24783@dcvr.yhbt.net> 0 siblings, 1 reply; 8+ messages in thread From: Eric Wong @ 2011-01-24 0:59 UTC (permalink / raw) To: unicorn list Alexey Bondar <alexey.bondar@gmail.com> wrote: > Hi. > > Unicorn 3.3.1 fails with "Read error: #<IOError: closed stream>" on request with attachments bigger than ~150kb. > > Ruby: 1.9.2-p0 > Rails: 3.0.1 > > Full stack trace: http://pastie.org/private/w1vwhbhdm9xtwwnpqog6wa > Configuration: http://pastie.org/private/hdpiqs0d3xj3qmsjwg > > I use nginx 0.8.x as frontend server. Nginx client_body_buffer_size is 128mb. Anything in the nginx error log? Did this happen with Unicorn 1.1.x? The input code changed in the 3.x series drastically to support non-rewindable input, so some bugs could've crept in. -- Eric Wong _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <20110124181919.GA24783@dcvr.yhbt.net>]
[parent not found: <BE4CE2A9-B804-4EFE-9F68-8E8253B4B82D@gmail.com>]
* Re: "Read error: #<IOError: closed stream>" and uploads [not found] ` <BE4CE2A9-B804-4EFE-9F68-8E8253B4B82D@gmail.com> @ 2011-01-25 0:51 ` Eric Wong [not found] ` <C83FE4C4-F4FB-48BC-8FC6-4E1B1B26F2D8@gmail.com> 0 siblings, 1 reply; 8+ messages in thread From: Eric Wong @ 2011-01-25 0:51 UTC (permalink / raw) To: mongrel-unicorn; +Cc: Alexey Bondar Alexey Bondar <alexey.bondar@gmail.com> wrote: (back to mailing list) > On Jan 24, 2011, at 21:19 , Eric Wong wrote: > > Eric Wong <normalperson@yhbt.net> wrote: > >> Alexey Bondar <alexey.bondar@gmail.com> wrote: > >>> Hi. > >>> > >>> Unicorn 3.3.1 fails with "Read error: #<IOError: closed stream>" on request with attachments bigger than ~150kb. > >>> > >>> Ruby: 1.9.2-p0 > >>> Rails: 3.0.1 > >>> > >>> Full stack trace: http://pastie.org/private/w1vwhbhdm9xtwwnpqog6wa > >>> Configuration: http://pastie.org/private/hdpiqs0d3xj3qmsjwg > >>> > >>> I use nginx 0.8.x as frontend server. Nginx client_body_buffer_size is 128mb. > >> > >> Anything in the nginx error log? Did this happen with Unicorn 1.1.x? > >> The input code changed in the 3.x series drastically to support > >> non-rewindable input, so some bugs could've crept in. > > In nginx log only two messages: body buffered to disk and backend timeout. Wait, nginx buffered the body to disk when client_body_buffer_size is 128mb and you have trouble with a 150kb request? > Same issue with 1.1.x OK. Which timeout exactly are you hitting in nginx? How slow is your client? Is your request queue very huge and you don't have enough Unicorn workers to handle the load? -- Eric Wong _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <C83FE4C4-F4FB-48BC-8FC6-4E1B1B26F2D8@gmail.com>]
* Re: "Read error: #<IOError: closed stream>" and uploads [not found] ` <C83FE4C4-F4FB-48BC-8FC6-4E1B1B26F2D8@gmail.com> @ 2011-01-25 3:17 ` Eric Wong [not found] ` <25C1DE1D-59D2-4E80-92E6-FDC0897814BA@gmail.com> 0 siblings, 1 reply; 8+ messages in thread From: Eric Wong @ 2011-01-25 3:17 UTC (permalink / raw) To: mongrel-unicorn; +Cc: Alexey Bondar Alexey Bondar <alexey.bondar@gmail.com> wrote: > On Jan 25, 2011, at 03:51 , Eric Wong wrote: > > Wait, nginx buffered the body to disk when client_body_buffer_size is > > 128mb and you have trouble with a 150kb request? > > Oops, my fault: not body_buffer_size, but client_max_body_size. client_body_buffer_size is 128k. > > As I understand, in this case nginx should buffer whole request body, before sending to backend? Yes, nginx always buffers the whole request body > >> Same issue with 1.1.x > > > > OK. Which timeout exactly are you hitting in nginx? > > Logs: > 2011/01/23 21:09:05 [warn] 79095#0: *261 a client request body is buffered to a temporary file /var/spool/nginx-client-body/3/00/0000000003 <snip> > 2011/01/23 21:09:35 [error] 79095#0: *261 upstream timed out (60: Operation timed out) while sending request to upstream <snip> Very strange. Do you process env["rack.input"] directly from your app or do you let Rails/Rack handle everything? This probably won't help, but can you try increasing the proxy_send_timeout in nginx? What's your Unicorn timeout? _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <25C1DE1D-59D2-4E80-92E6-FDC0897814BA@gmail.com>]
* Re: "Read error: #<IOError: closed stream>" and uploads [not found] ` <25C1DE1D-59D2-4E80-92E6-FDC0897814BA@gmail.com> @ 2011-01-25 3:53 ` Eric Wong [not found] ` <41BA1BA1-3DB7-45C2-8503-17026080D04A@gmail.com> 0 siblings, 1 reply; 8+ messages in thread From: Eric Wong @ 2011-01-25 3:53 UTC (permalink / raw) To: mongrel-unicorn; +Cc: Alexey Bondar Alexey Bondar <alexey.bondar@gmail.com> wrote: > Unicorn timeout is 75s. I just tried with 75s proxy_send_timeout: same > issue. What client are you using to send? I wonder if Content-Length is somehow screwed up because a client is trying to compress the data. What happens when you try hitting Unicorn directly without nginx? -- Eric Wong _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <41BA1BA1-3DB7-45C2-8503-17026080D04A@gmail.com>]
* Re: "Read error: #<IOError: closed stream>" and uploads [not found] ` <41BA1BA1-3DB7-45C2-8503-17026080D04A@gmail.com> @ 2011-01-25 4:10 ` Eric Wong [not found] ` <C5FE59F4-1083-4E89-BAD3-970CF0BCB745@gmail.com> 0 siblings, 1 reply; 8+ messages in thread From: Eric Wong @ 2011-01-25 4:10 UTC (permalink / raw) To: mongrel-unicorn; +Cc: Alexey Bondar Alexey Bondar <alexey.bondar@gmail.com> wrote: > On Jan 25, 2011, at 06:53 , Eric Wong wrote: > > What client are you using to send? I wonder if Content-Length is > > somehow screwed up because a client is trying to compress the data. > > Different: Webkit based browsers, FF, Opera. OK. I'm fairly certain those clients don't make compressed requests, but I'm out of ideas as to what could be wrong... > > What happens when you try hitting Unicorn directly without nginx? > > Hm... How I can directly hit unicorn via unix socket? Just have it listen on a TCP port, too (it can handle multiple listen statements in the config). Maybe we can narrow it down to something in the nginx config... -- Eric Wong _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <C5FE59F4-1083-4E89-BAD3-970CF0BCB745@gmail.com>]
* Re: "Read error: #<IOError: closed stream>" and uploads [not found] ` <C5FE59F4-1083-4E89-BAD3-970CF0BCB745@gmail.com> @ 2011-01-25 4:54 ` Eric Wong [not found] ` <5F765353-9348-4264-84FE-B639FB198868@gmail.com> 0 siblings, 1 reply; 8+ messages in thread From: Eric Wong @ 2011-01-25 4:54 UTC (permalink / raw) To: mongrel-unicorn; +Cc: Alexey Bondar Alexey Bondar <alexey.bondar@gmail.com> wrote: > On Jan 25, 2011, at 07:10 , Eric Wong wrote: > > OK. I'm fairly certain those clients don't make compressed requests, > > but I'm out of ideas as to what could be wrong... > > > > Just have it listen on a TCP port, too (it can handle multiple > > listen statements in the config). Maybe we can narrow it down > > to something in the nginx config... > > Hm. Yeah, direct access works well. > Uh, what can be wrong with nginx config in this case? :( I would trim your config down the the bare minimum and use the defaults wherever you can. Then start adding things back in and see what broke it. -- Eric Wong _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <5F765353-9348-4264-84FE-B639FB198868@gmail.com>]
* [PATCH] KNOWN_ISSUES: FreeBSD 8 and sendfile can be buggy [not found] ` <5F765353-9348-4264-84FE-B639FB198868@gmail.com> @ 2011-01-25 21:33 ` Eric Wong 0 siblings, 0 replies; 8+ messages in thread From: Eric Wong @ 2011-01-25 21:33 UTC (permalink / raw) To: mongrel-unicorn; +Cc: Alexey Bondar Alexey Bondar <alexey.bondar@gmail.com> wrote: > On Jan 25, 2011, at 07:54 , Eric Wong wrote: > > Alexey Bondar <alexey.bondar@gmail.com> wrote: > >> Hm. Yeah, direct access works well. > >> Uh, what can be wrong with nginx config in this case? :( > > > > I would trim your config down the the bare minimum and use the defaults > > wherever you can. Then start adding things back in and see what broke > > it. > > Maxim Dounin from nginx mail list says that this issue caused by some issues with sendfile in > FreeBSD 8. > > After setting sendfile to off uploads works well. > > You can add this info to known 3rd party issues. :) Thanks Alexey! >From 8ac0ae45a04f5f121f323c182403ef6eb0d8aa18 Mon Sep 17 00:00:00 2001 From: Eric Wong <normalperson@yhbt.net> Date: Tue, 25 Jan 2011 13:30:21 -0800 Subject: [PATCH] KNOWN_ISSUES: FreeBSD 8 and sendfile can be buggy Reported by Alexey Bondar. --- KNOWN_ISSUES | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/KNOWN_ISSUES b/KNOWN_ISSUES index bc3dac5..95f4e43 100644 --- a/KNOWN_ISSUES +++ b/KNOWN_ISSUES @@ -6,6 +6,10 @@ acceptable solution. Those issues are documented here. * For notes on sandboxing tools such as Bundler or Isolate, see the {Sandbox}[link:Sandbox.html] page. +* nginx with "sendfile on" under FreeBSD 8 is broken when + uploads are buffered to disk. Disabling sendfile is required to + work around this bug which should be fixed in newer versions of FreeBSD. + * Under Ruby 1.9.1, methods like Array#shuffle and Array#sample will segfault if called after forking. This is fixed in trunk (r26936) and should be backported to the next 1.9.1 stable release (after p378). -- Eric Wong _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying ^ permalink raw reply related [flat|nested] 8+ messages in thread
end of thread, other threads:[~2011-01-25 21:33 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-01-23 21:28 "Read error: #<IOError: closed stream>" and uploads Alexey Bondar
2011-01-24 0:59 ` Eric Wong
[not found] ` <20110124181919.GA24783@dcvr.yhbt.net>
[not found] ` <BE4CE2A9-B804-4EFE-9F68-8E8253B4B82D@gmail.com>
2011-01-25 0:51 ` Eric Wong
[not found] ` <C83FE4C4-F4FB-48BC-8FC6-4E1B1B26F2D8@gmail.com>
2011-01-25 3:17 ` Eric Wong
[not found] ` <25C1DE1D-59D2-4E80-92E6-FDC0897814BA@gmail.com>
2011-01-25 3:53 ` Eric Wong
[not found] ` <41BA1BA1-3DB7-45C2-8503-17026080D04A@gmail.com>
2011-01-25 4:10 ` Eric Wong
[not found] ` <C5FE59F4-1083-4E89-BAD3-970CF0BCB745@gmail.com>
2011-01-25 4:54 ` Eric Wong
[not found] ` <5F765353-9348-4264-84FE-B639FB198868@gmail.com>
2011-01-25 21:33 ` [PATCH] KNOWN_ISSUES: FreeBSD 8 and sendfile can be buggy Eric Wong
Code repositories for project(s) associated with this public inbox https://yhbt.net/unicorn.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).