"upstream timed out" after upgrades

[John-Paul Bader <hukl@berlin.ccc.de>]

Hey guys,


i recently updated several components of my website setup: 

varnish   2.0.5 -> 2.0.6
nginx      7.4.2 -> 7.4.5
unicorn  0.95.3 -> 0.96
ruby18  to the latest patch level 
and rails from 2.3.2

It all seemed to work great but recently I noticed that the website hangs loading. The weird thing is that it hangs because 2 or 3 javascript files take about a minute to finish and when they do they are incomplete as in cut of in the middle of the content. The nginx error log gives the following output every few seconds:

2010/02/04 08:58:41 [error] 29389#0: *471 upstream timed out (60: Operation timed out) while reading upstream, client: 213.73.89.122, server: www.ccc.de, request: "GET /javascripts/jquery-1.3.2.min.js?1246657683 HTTP/1.1", upstream: "http://unix:/var/run/unicorn.sock:/javascripts/jquery-1.3.2.min.js?1246657683", host: "www.ccc.de"

I don't think its varnishs fault because i get Internal Server Errors (500) if I run curl on the machine against nginx, not at first though - several requests for that jquery file run just fine and then suddenly i get a 500 on a request with the content being cut of in the middle.

Next thing I did was configuring unicorn to listen on 213.73.89.122:9090 rather than on a socket so i could run curl directly against it and even after a fresh start of unicorn gives me:


curl http://213.73.89.122:9090/javascripts/jquery-1.3.2.min.js
(lots of javascript)…,rowspan:"rowSpan",tabindex:"tabIndHTTP/1.1 500 Internal Server Error

curl: (18) transfer closed with 16256 bytes remaining to read

I also downgraded back to 0.95.3 but that didn't change anything. So now after 8h of try and error I'm out of ideas and would be happy to hear some suggestions.

Kind regards, John
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying