* [ANN] unicorn 0.9.0 (experimental release)
@ 2009-07-01 22:58 1% Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2009-07-01 22:58 UTC (permalink / raw)
To: mongrel-unicorn, ruby-talk, rack-devel; +Cc: mongrel-development
Unicorn is a Rack HTTP server for Unix, fast clients and nothing else[1]
We now have support for "Transfer-Encoding: chunked" bodies in
requests. Not only that, Rack applications reading input bodies
get that data streamed off to the client socket on an as-needed
basis. This allows the application to do things like upload
progress notification and even tunneling of arbitrary stream
protocols via bidirectional chunked encoding.
See Unicorn::App::Inetd and examples/git.ru (including the
comments) for an example of tunneling the git:// protocol over
HTTP.
This release also gives applications the ability to respond
positively to "Expect: 100-continue" headers before being rerun
without closing the socket connection. See Unicorn::App::Inetd
for an example of how this is used.
This release is NOT recommended for production use.
Eric Wong (43):
http_request: no need to reset the request
http_request: StringIO is binary for empty bodies (1.9)
http_request: fix typo for 1.9
Transfer-Encoding: chunked streaming input support
Unicorn::App::Inetd: reinventing Unix, poorly :)
README: update with mailing list info
local.mk.sample: publish_doc gzips all html, js, css
Put copyright text in new files, include GPL2 text
examples/cat-chunk-proxy: link to proposed curl(1) patch
Update TODO
Avoid duplicating the "Z" constant
Optimize body-less GET/HEAD requests (again)
tee_input: Don't expose the @rd object as a return value
exec_cgi: small cleanups
README: another note about older Sinatra
tee_input: avoid defining a @rd.size method
Make TeeInput easier to use
test_upload: add tests for chunked encoding
GNUmakefile: more stringent error checking in tests
test_upload: fix ECONNRESET with 1.9
GNUmakefile: allow TRACER= to be specified for tests
test_rails: workaround long-standing 1.9 bug
tee_input: avoid rereading fresh data
"Fix" tests that break with stream_input=false
inetd: fix broken constant references
configurator: provide stream_input (true|false) option
chunked_reader: simpler interface
http_request: force BUFFER to be Encoding::BINARY
ACK clients on "Expect: 100-continue" header
Only send "100 Continue" when no body has been sent
http_request: tighter Transfer-Encoding: "chunked" check
Add trailer_parser for parsing trailers
chunked_reader: Add test for chunk parse failure
TeeInput: use only one IO for tempfile
trailer_parser: set keys with "HTTP_" prefix
TrailerParser integration into ChunkedReader
Unbind listeners as before stopping workers
Retry listen() on EADDRINUSE 5 times ever 500ms
Re-add support for non-portable socket options
Move "Expect: 100-continue" handling to the app
tee_input: avoid ignoring initial body blob
Force streaming input onto apps by default
unicorn 0.9.0
* site: http://unicorn.bogomips.org/
* git: git://git.bogomips.org/unicorn.git
* http+git: http://git.bogomips.org:8080/unicorn.git [1]
* cgit: http://git.bogomips.org/cgit/unicorn.git/
* list: mongrel-unicorn@rubyforge.org
[1] - Actually, most of the new features in this release should in fact
work on non-Unix OSes so they can be adapted for other servers with
wider audiences.
[2] - This is the git:// protocol tunneled inside a bidirectional
"Transfer-Encoding: chunked" request/response using this latest
release of Unicorn.
--
Eric Wong
^ permalink raw reply [relevance 1%]
* Re: Unicorn
@ 2009-07-07 20:48 2% ` Eric Wong
[not found] ` <CE7F2AC4-4088-44ED-A5FF-2A41FE2B61EF@sponagl.de>
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2009-07-07 20:48 UTC (permalink / raw)
To: Paul Sponagl; +Cc: mongrel-unicorn
Paul Sponagl <paul@sponagl.de> wrote:
> Hi Eric,
>
> unicorn does not close the connection on osx after sending the content.
> (It acts as the client would send a Connection: Keep-Alive)
> The connection will be aborted by nginx after about 60
> secs and the content will be send out.
>
> Macintosh:~> telnet 127.0.0.1 3001
> Trying 127.0.0.1...
> Connected to localhost.
> Escape character is '^]'.
> GET / HTTP/1.1
>
> HTTP/1.1 200 OK
> Date: Tue, 07 Jul 2009 08:11:17 GMT
> Status: 200 OK
> Connection: close
> X-Runtime: 293
> ETag: "2659d611050f70fc65c6c052687e5a13"
> Content-Type: text/html; charset=utf-8
> Cache-Control: private, max-age=0, must-revalidate
> Content-Length: 24328
> ...
> the rest of the content
Hi Paul,
This is strange. Can you tell that you're getting all 24328 bytes of
the content? And no exception is thrown to stderr, either? Unicorn
should've closed the socket even on an exception. Which version of OSX
and Ruby are you running? Is there anything special/weird about the
Rack response body you're sending? Does Rack::Lint complain about
anything? Unicorn is fairly strict about Rack::Lint compatibility.
Does the following patch help?
diff --git a/lib/unicorn/http_response.rb b/lib/unicorn/http_response.rb
index bfaa33d..acf4d04 100644
--- a/lib/unicorn/http_response.rb
+++ b/lib/unicorn/http_response.rb
@@ -60,6 +60,7 @@ module Unicorn
"Connection: close\r\n" \
"#{OUT.join(Z)}\r\n")
body.each { |chunk| socket.write(chunk) }
+ socket.flush
socket.close # flushes and uncorks the socket immediately
ensure
body.respond_to?(:close) and body.close rescue nil
------------------------------------------------------------------------
Or you can replace the "socket.close" with "exit!(0)" and really
force it to close :)
Thanks again for trying this!
--
Eric Wong
^ permalink raw reply related [relevance 2%]
* Re: Unicorn
@ 2009-07-09 10:10 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2009-07-09 10:10 UTC (permalink / raw)
To: Paul Sponagl; +Cc: mongrel-unicorn
Paul Sponagl <paul@sponagl.de> wrote:
> Eric - you're a genius ! This patch solved it!
Cool, but a genius would've avoided this problem entirely :)
> Indeed i am forking a sleeping cache refresh process at requests to '/'
> via a system call.
> Now i am ready to give it a try. Thank you very much for your help!
>
> All the best for you and unicorn
> Paul
No problem, thanks for reporting the issues. I've just released 0.8.2
(stable) and 0.9.1 (experimental) with the fix. Hopefully that's the
last 0.8.x release since I've bigger plans for 0.9.x (or 0.10.x).
> PS: Last time i forgot to cc to the list - i added my previous mail
> below to keep searchers- and engines happy.
Thanks again, btw can you avoid top-posting here? I'm really old
school like that :)
--
Eric Wong
^ permalink raw reply [relevance 2%]
* Re: Pidfiles and cwd?
[not found] <8b73aaca0909042020w73fb03dfpf6c77c85c1c486ad@mail.gmail.com>
@ 2009-09-05 3:21 2% ` Chris Wanstrath
2009-09-05 4:28 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Chris Wanstrath @ 2009-09-05 3:21 UTC (permalink / raw)
To: mongrel-unicorn
Yikes. Let me try that again.
Hi,
Thanks for unicorn!
Two questions:
A) Is there a reason `unicorn` allows you to specify the pidfile's
location but `unicorn_rails` does not?
B) Is there a reason `unicorn_rails` must start in the app root and
doesn't allow it as a config option?
Thanks,
Chris
On Fri, Sep 4, 2009 at 8:20 PM, Chris Wanstrath <chris@ozmm.org> wrote:
>
> Hi,
> Thanks for unicorn!
> Two questions:
> A) Is there a reason `unicorn` allows you to specify the pidfile's location but `unicorn_rails
>
> --
> Chris Wanstrath
> http://github.com/defunkt
--
Chris Wanstrath
http://github.com/defunkt
^ permalink raw reply [relevance 2%]
* Re: Pidfiles and cwd?
2009-09-05 3:21 2% ` Pidfiles and cwd? Chris Wanstrath
@ 2009-09-05 4:28 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2009-09-05 4:28 UTC (permalink / raw)
To: Chris Wanstrath; +Cc: mongrel-unicorn
Chris Wanstrath <chris@ozmm.org> wrote:
> Yikes. Let me try that again.
>
> Hi,
>
> Thanks for unicorn!
Hi Chris, no problem!
> Two questions:
>
> A) Is there a reason `unicorn` allows you to specify the pidfile's
> location but `unicorn_rails` does not?
`unicorn` was designed to mimic `rackup` in terms of command line
options and `unicorn_rails` was designed to mimic `script/server` in
Rails.
I really didn't know what I was doing with the command-line options for
this, so I decided to steal from others :)
For long-running servers, I'm not a fan of command-line options in
general because they're easy to forget, so `unicorn` only supports it
because `rackup` does it (so the embedded CLI options in config.ru can
be shared).
For `unicorn_rails`, --daemonize already sets a default PID path in
RAILS_ROOT/tmp/pids/unicorn.pid whereas `script/server` chooses
RAILS_ROOT/tmp/pids/server.pid. Since Rails values "convention over
configuration", I figured I might as well hard code it...
Additionally, the "-P" parameter used by unicorn_rails and script/server
is used to set RAILS_RELATIVE_URL_ROOT so it conflicts with the short
option used by rackup/unicorn.
> B) Is there a reason `unicorn_rails` must start in the app root and
> doesn't allow it as a config option?
Since the config file is just Ruby, you can just Dir.chdir inside it.
And since the chdir is done when the config file is evaluated, the
chdir can be done across restarts/reloads (you can point it to a
symlinked directory) to pick up new code/releases.
If you do that, I would initially start Unicorn in "/" or some other
directory that won't get deleted so you'll be safe for upgrades.
If you managed to forget that, you can set the following in your
Unicorn config:
Unicorn::HttpServer::START_CTX[:cwd] = "/"
And then HUP the process before doing the USR2+QUIT to reexec.
Subsequent Unicorns will always start in "/" and then you can
Dir.chdir to wherever you run your app.
Hopefully that makes sense, one thing I've been trying to avoid with the
configuration is having too many ways to do the same thing.
--
Eric Wong
^ permalink raw reply [relevance 0%]
* GitHub on Unicorn
@ 2009-09-19 23:07 2% Chris Wanstrath
2009-09-20 0:00 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Chris Wanstrath @ 2009-09-19 23:07 UTC (permalink / raw)
To: mongrel-unicorn
Just wanted to say that GitHub has been running on Unicorn for about
two weeks now. In that time it's successfully served millions of pages
and has survived two separate DDoS attacks.
Here's the config file we currently use (complete with a fun hack to
gracefully kill the old master when a new worker pool is ready):
http://gist.github.com/189623
(Tom's thread with the `backlog` fix concerns our new servers, which
aren't yet in production.)
I plan to do a writeup on our transition from mongrel_cluster to
Unicorn in the near future, in case others are interested. I'll post
the link here when it's available.
Also: I'm keeping a mirror of the project at
http://github.com/defunkt/unicorn for any other GH users who want to
watch updates in their generalized feed. I update it semi-regularly.
Long live fork(2)! And thanks again for the project.
Cheers,
--
Chris Wanstrath
http://github.com/defunkt
^ permalink raw reply [relevance 2%]
* Re: GitHub on Unicorn
2009-09-19 23:07 2% GitHub on Unicorn Chris Wanstrath
@ 2009-09-20 0:00 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2009-09-20 0:00 UTC (permalink / raw)
To: Chris Wanstrath; +Cc: mongrel-unicorn
Chris Wanstrath <chris@ozmm.org> wrote:
> Just wanted to say that GitHub has been running on Unicorn for about
> two weeks now. In that time it's successfully served millions of pages
> and has survived two separate DDoS attacks.
Wow, this is wonderful news!
> Here's the config file we currently use (complete with a fun hack to
> gracefully kill the old master when a new worker pool is ready):
>
> http://gist.github.com/189623
Great use of the before/after_fork hooks
One possible issue is a race condition in the before_fork hook,
so I'd put a rescue to protect the File.read in the before_fork:
old master new master
-----------------------------------------------------------------------
before_fork for worker=0
File.exist?(old_pid) => true
Process.kill :QUIT, File.read(old_pid).to_i
before_fork for worker=1
File.exist?(old_pid) => true
processes :QUIT
unlinks old_pid
# the File.read will raise Errno::ENOENT:
Process.kill :QUIT, File.read(old_pid).to_i
> (Tom's thread with the `backlog` fix concerns our new servers, which
> aren't yet in production.)
>
> I plan to do a writeup on our transition from mongrel_cluster to
> Unicorn in the near future, in case others are interested. I'll post
> the link here when it's available.
Looking forward to it!
> Also: I'm keeping a mirror of the project at
> http://github.com/defunkt/unicorn for any other GH users who want to
> watch updates in their generalized feed. I update it semi-regularly.
Cool, more exposure's always good.
Small nit: "Eric Wong's Unicorn" doesn't sound quite right...
While I am the benevolent dictator for now, I do welcome contributions.
I could not have have built it without standing on the shoulders of
Mongrel and the existence of Rack and nginx.
Personally, I try to keep a low public profile and it's always been an
weird balancing act trying to get people to use my work at the same
time...
> Long live fork(2)! And thanks again for the project.
:)
--
Eric Wong
^ permalink raw reply [relevance 0%]
* 0.93.0 gem bug report: permissions snafu
@ 2009-10-02 23:54 2% Jay Reitz
0 siblings, 0 replies; 200+ results
From: Jay Reitz @ 2009-10-02 23:54 UTC (permalink / raw)
To: mongrel-unicorn
Hey all-
I'm not embarrassed to say that I love the unicorn, no matter how
weird that sounds. When I first read the README, my hopes soared
because the philosophy was exactly in line with how I'd imagined a
ruby application server should behave. And so far I have been pleased
with our initial results on a portion of traffic in production.
I found some permissions issues with the 0.93.0 gem release. In
short, the installed permissions are excessively strict and incorrect.
The following files have the mode 700 rather than the correct 755
mode:
lib/unicorn.rb
lib/unicorn/configurator.rb
lib/unicorn/http_request.rb
lib/unicorn/http_response.rb
lib/unicorn/socket_helper.rb
This prevents unicorn from being started by a non-super user, as many
people are wont to do. There are also a smattering of document files
and the gemspec with incorrect permissions but this it less of an
issue.
Thanks again and I've subscribed to the mailing list.
>j.
^ permalink raw reply [relevance 2%]
* Re: Unicorn - "Still a couple bugs to knock out."
[not found] ` <b89b94f00910072029m5c7265ban2f98e4346265e534@mail.gmail.com>
@ 2009-10-08 4:47 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2009-10-08 4:47 UTC (permalink / raw)
To: Dan Herrera; +Cc: mongrel-unicorn, labs
Dan Herrera <dan@revelationglobal.com> wrote:
> Hi Eric,
>
> Thanks for getting in touch. We were also pleasantly surprised by the
> performance of unicorn!
>
> As to the blog post, we misspoke and will revise our language on the post to
> reflect that what we encountered weren't bugs, but issues with our own
> setup.
Thanks for the clarification, I was worried :x
> Our app has a memory leak, and we haven't yet settled on a monit recipe to
> restart unicorn when the sub processes that are started run over a certain
> memory threshold. There were also some changes we needed to make with our
> nginx configuration, but again, they didn't reflect any problems with
> unicorn.
You can probably have monit just SIGQUIT the workers that run over the
memory threshold and just let the unicorn master restart them (note: I'm
not a monit user).
If your OS supports it, you can try setrlimit in your after_fork hook.
Modern Linux 2.6 doesn't support Process::RLIMIT_RSS, but you can still
do Process::RLIMIT_AS which is still somewhat useful[1].
after_fork do |server, worker|
Process.setrlimit(Process::RLIMIT_AS, size_in_kilobytes)
end
Then have a simple Rack middleware that traps NoMemoryError:
# XXX totally untested!, I'm not sure what to do if Process.kill
# here hits NoMemoryError, either...
class ExitOnOOM < Struct.new(:app)
def call(env)
begin
app.call(env)
rescue NoMemoryError
Process.kill(:QUIT, 0) # graceful exit signal
# or maybe just: exit!
end
end
end
[1] On modern GNU/Linux systems that don't have RLIMIT_RSS, the VM size
taken by the mmap()'ed shared libraries usually doesn't change over the
lifetime of the process, so you can use that as a baseline and figure
out how far you want to go from there.... Of course, your Unicorns
shouldn't be swapping so swap usage won't have to be accounted for.
Of course the proper solution to all this is to fix your memory leaks :)
I posted some notes here a few months back on the Mongrel list:
http://thread.gmane.org/gmane.comp.lang.ruby.mongrel.general/5760/focus=5766
> Engine Yard doesn't currently support unicorn on their slices, but once we
> have a satisfactory monit recipe and no other issues with our setup, we plan
> to work with them so that they'll hopefully change their mind. I think
> this, and the monit recipe is what James meant when he said we wouldn't be
> using unicorn in the near future.
Ah, didn't know that. I guess github was a special case where they
thought: "oh well, these guys are leaving us for another host anyways,
they can be our guinea pig!" :)
> Anyway, we are all great fans, and would like to thank all of you for all of
> your hard work.
No problem :)
--
Eric Wong
^ permalink raw reply [relevance 0%]
* Our Unicorn Setup
@ 2009-10-09 19:42 2% Chris Wanstrath
2009-10-09 20:30 2% ` Eric Wong
2009-10-09 21:03 0% ` Dusty Doris
0 siblings, 2 replies; 200+ results
From: Chris Wanstrath @ 2009-10-09 19:42 UTC (permalink / raw)
To: mongrel-unicorn
Hi list,
I've just published a post detailing our Unicorn setup, migration
process, and reasons for choosing it:
http://github.com/blog/517-unicorn
Thanks again!
--
Chris Wanstrath
http://github.com/defunkt
^ permalink raw reply [relevance 2%]
* Re: Our Unicorn Setup
2009-10-09 19:42 2% Our Unicorn Setup Chris Wanstrath
@ 2009-10-09 20:30 2% ` Eric Wong
2009-10-09 21:25 0% ` Chris Wanstrath
2009-10-09 21:03 0% ` Dusty Doris
1 sibling, 1 reply; 200+ results
From: Eric Wong @ 2009-10-09 20:30 UTC (permalink / raw)
To: Chris Wanstrath; +Cc: mongrel-unicorn
Chris Wanstrath <chris@ozmm.org> wrote:
> Hi list,
>
> I've just published a post detailing our Unicorn setup, migration
> process, and reasons for choosing it:
> http://github.com/blog/517-unicorn
Cool!
"The Unicorn master manages the workers and balancing"
Actually, the master never manages balancing, just the workers. The
diagram is a little inaccurate as it looks like the master sees the
requests, it never does.
The request flow is like this:
requests
|
|
|
shared socket(s)
/|\
/ | \
| | |
| | |
worker pool
While the shared socket is opened and configured by the master, but the
master does nothing else with the sockets. You're completely right
about the pull balancing, it's one of the most distinctive differences
between Unicorn and other setups.
Also, for the 502s, do you get more 502s after the initial worker times
out? I think nginx may (still)[1] mark a backend as completely dead/down
when a backend dies. That may cause nginx to stop forwarding to that
backend entirely and throw more 502s for a few seconds until nginx
decides to actually try that backend again.
Setting fail_timeout=0 causes nginx to never mark backends as down and
always try to send a request to them:
upstream github {
server unix:/data/github/current/tmp/sockets/unicorn.sock fail_timeout=0;
}
I'll add this bit somewhere in the Unicorn docs and release 0.93.3 with
the OpenBSD fix for Jeremy in a bit.
> Thanks again!
No problem :)
[1] - I'm not 100% sure if nginx still does this, but I don't see
anything in the 0.6.x changelog that indicates otherwise.
I don't see a huge amount of harm in doing this always, we've been
using fail_timeout=0 for ~2 years now regardless of the backend.
--
Eric Wong
^ permalink raw reply [relevance 2%]
* Re: Our Unicorn Setup
2009-10-09 19:42 2% Our Unicorn Setup Chris Wanstrath
2009-10-09 20:30 2% ` Eric Wong
@ 2009-10-09 21:03 0% ` Dusty Doris
1 sibling, 0 replies; 200+ results
From: Dusty Doris @ 2009-10-09 21:03 UTC (permalink / raw)
To: Chris Wanstrath; +Cc: mongrel-unicorn
Thanks for this post Chris, it was very informative and has answered a
few questions that I've had in my head over the last couple of days.
I've been testing unicorn with a few apps for a couple days and
actually already moved one over to it.
I have a question for list.
We are currently setup with a load balancer that runs nginx and
haproxy. Nginx, simply proxies to haproxy, which then balances that
across multiple mongrel or thin instances that span several servers.
We simply include the public directory on our load balancer so nginx
can serve static files right there. We don't have nginx running on
the app servers, they are just mongrel or thin.
So, my question. How would you do a Unicorn deployment when you have
multiple app servers?
1. Simply use mongrels upstream and let it round-robin between all
the unicorn instances on the different servers? Or, perhaps use the
fair-upstream plugin?
nginx -> [unicorns]
2. Keep haproxy in the middle?
nginx -> haproxy -> [unicorns]
3. Stick haproxy in front and have it balance between the app servers
that run their own nginx?
haproxy -> [nginxs] -> unicorn # could use socket instead of tcp in this case
I would love to hear any opinions.
Thanks!
Dusty Doris
On Fri, Oct 9, 2009 at 3:42 PM, Chris Wanstrath <chris@ozmm.org> wrote:
> Hi list,
>
> I've just published a post detailing our Unicorn setup, migration
> process, and reasons for choosing it:
> http://github.com/blog/517-unicorn
>
> Thanks again!
>
> --
> Chris Wanstrath
> http://github.com/defunkt
> _______________________________________________
> mongrel-unicorn mailing list
> mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>
^ permalink raw reply [relevance 0%]
* Re: Our Unicorn Setup
2009-10-09 20:30 2% ` Eric Wong
@ 2009-10-09 21:25 0% ` Chris Wanstrath
0 siblings, 0 replies; 200+ results
From: Chris Wanstrath @ 2009-10-09 21:25 UTC (permalink / raw)
To: Eric Wong; +Cc: mongrel-unicorn
On Fri, Oct 9, 2009 at 1:30 PM, Eric Wong <normalperson@yhbt.net> wrote:
> "The Unicorn master manages the workers and balancing"
>
> Actually, the master never manages balancing, just the workers. The
> diagram is a little inaccurate as it looks like the master sees the
> requests, it never does.
>
> The request flow is like this:
>
> requests
> |
> |
> |
> shared socket(s)
> /|\
> / | \
> | | |
> | | |
> worker pool
>
> While the shared socket is opened and configured by the master, but the
> master does nothing else with the sockets. You're completely right
> about the pull balancing, it's one of the most distinctive differences
> between Unicorn and other setups.
Thanks! I've updated the diagram and some of the language to be accurate.
> Also, for the 502s, do you get more 502s after the initial worker times
> out? I think nginx may (still)[1] mark a backend as completely dead/down
> when a backend dies. That may cause nginx to stop forwarding to that
> backend entirely and throw more 502s for a few seconds until nginx
> decides to actually try that backend again.
>
> Setting fail_timeout=0 causes nginx to never mark backends as down and
> always try to send a request to them:
>
> upstream github {
> server unix:/data/github/current/tmp/sockets/unicorn.sock fail_timeout=0;
> }
We'll try this out, thanks for letting us know about it.
Cheers,
--
Chris Wanstrath
http://github.com/defunkt
^ permalink raw reply [relevance 0%]
* Re: Unicorn Nginx Issue
@ 2009-10-13 19:43 2% ` Eric Wong
2009-10-13 20:11 0% ` Matt Mongeau
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2009-10-13 19:43 UTC (permalink / raw)
To: unicorn general mailing list
Matt Mongeau <halogenandtoast@gmail.com> wrote:
> sudo gem check -t unicorn
Again, please don't top post on this mailing list (nor on other technical
mailing lists in general).
> fails
>
> Failure:
> test_rack_lint_big_put(RequestTest) [./test/unit/test_request.rb:178]:
> <nil> expected but was
> <"
>
> followed by lots whitespace
>
> ">.
Unrelated to the problem we were having, but this really should be
working, especially with 0.93.2 or later. Is anybody else out there
hitting this?
> Error:
> test_expand_addr(TestConfigurator):
> SocketError: getaddrinfo: nodename nor servname provided, or not known
> /opt/local/lib/ruby/gems/1.8/gems/unicorn-0.93.2/lib/unicorn/configurator.rb:346:in
> `pack_sockaddr_in'
> /opt/local/lib/ruby/gems/1.8/gems/unicorn-0.93.2/lib/unicorn/configurator.rb:346:in
> `expand_addr'
> ./test/unit/test_configurator.rb:35:in `call'
> ./test/unit/test_configurator.rb:35:in `test_expand_addr'
This looks like a portability issue. I'll probably rip those tests out
since a good chunk of systems don't addresses like this.
But above this test failure, the other test_expand_addr assertions
manage to pass which is strange, namely the following:
meth = Unicorn::Configurator.new.method(:expand_addr)
assert_equal "/var/run/unicorn.sock", meth.call("/var/run/unicorn.sock")
assert_equal "#{Dir.pwd}/foo/bar.sock", meth.call("unix:foo/bar.sock")
Your original paths were under 104 bytes, too[1]
> >> I had
> >> listen '/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock',
> >> :backlog => 1024
> >> I guess I needed
> >> listen 'unix:/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock',
> >> :backlog => 1024
Does using a shorter path help at all?
Shorter (and shallower) paths are even a small bit faster because the
filesystem has to do less work to resolve it for every connection :)
[1] - http://portabilityblog.com/blog/archives/4-UNIX-domain-sockets.html
--
Eric Wong
^ permalink raw reply [relevance 2%]
* Re: Unicorn Nginx Issue
2009-10-13 19:43 2% ` Eric Wong
@ 2009-10-13 20:11 0% ` Matt Mongeau
0 siblings, 0 replies; 200+ results
From: Matt Mongeau @ 2009-10-13 20:11 UTC (permalink / raw)
To: unicorn general mailing list
On Tue, Oct 13, 2009 at 3:43 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Matt Mongeau <halogenandtoast@gmail.com> wrote:
>> sudo gem check -t unicorn
>
> Again, please don't top post on this mailing list (nor on other technical
> mailing lists in general).
>
>> fails
>>
>> Failure:
>> test_rack_lint_big_put(RequestTest) [./test/unit/test_request.rb:178]:
>> <nil> expected but was
>> <"
>>
>> followed by lots whitespace
>>
>> ">.
>
> Unrelated to the problem we were having, but this really should be
> working, especially with 0.93.2 or later. Is anybody else out there
> hitting this?
>
>> Error:
>> test_expand_addr(TestConfigurator):
>> SocketError: getaddrinfo: nodename nor servname provided, or not known
>> /opt/local/lib/ruby/gems/1.8/gems/unicorn-0.93.2/lib/unicorn/configurator.rb:346:in
>> `pack_sockaddr_in'
>> /opt/local/lib/ruby/gems/1.8/gems/unicorn-0.93.2/lib/unicorn/configurator.rb:346:in
>> `expand_addr'
>> ./test/unit/test_configurator.rb:35:in `call'
>> ./test/unit/test_configurator.rb:35:in `test_expand_addr'
>
> This looks like a portability issue. I'll probably rip those tests out
> since a good chunk of systems don't addresses like this.
>
> But above this test failure, the other test_expand_addr assertions
> manage to pass which is strange, namely the following:
>
> meth = Unicorn::Configurator.new.method(:expand_addr)
>
> assert_equal "/var/run/unicorn.sock", meth.call("/var/run/unicorn.sock")
> assert_equal "#{Dir.pwd}/foo/bar.sock", meth.call("unix:foo/bar.sock")
>
> Your original paths were under 104 bytes, too[1]
>
>> >> I had
>> >> listen '/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock',
>> >> :backlog => 1024
>> >> I guess I needed
>> >> listen 'unix:/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock',
>> >> :backlog => 1024
>
> Does using a shorter path help at all?
>
> Shorter (and shallower) paths are even a small bit faster because the
> filesystem has to do less work to resolve it for every connection :)
>
> [1] - http://portabilityblog.com/blog/archives/4-UNIX-domain-sockets.html
>
> --
> Eric Wong
> _______________________________________________
> mongrel-unicorn mailing list
> mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>
Sorry for top posting, I actually didn't know what you meant by
that... had to have someone else explain it to me.
Ok my previous problem seems to be from running unicorn_rails multiple times
so if I run
unicorn_rails -c config/unicorn.rb -E development -D
it works fine (with and without the unix prefix on the socket)
then if I just run
unicorn_rails -c config/unicorn.rb -E development
without killing the current master everything seems to work
if I issue a term signal ^C to it and try
unicorn_rails -c config/unicorn.rb -E development -D
it no longer works
seems like I shouldn't issue these commands in tandem without first
killing the previous unicorn instance
^ permalink raw reply [relevance 0%]
* Re: Truncated request bodies
@ 2009-10-25 0:04 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2009-10-25 0:04 UTC (permalink / raw)
To: unicorn list
Vadim Spivak <vadim@spivak.net> wrote:
> > Hi Vadim,
> >
> > What's the output of that standalone ruby example for you? Is there a
> > bug filed with ruby-core I could look at?
>
> "Should be 5: 0"
>
> I haven't had a chance to file a bug yet and couldn't find
>
> Also, if you don't set sync to true, then you get the expected output
> 5 instead of 0.
Interesting....
I bet it doesn't fail if you skip the f.read in there.
> > Also, does using f << or f.syswrite change things?
> > f.sync = true should really make it unnecessary...
>
> I tried both and they didn't make a difference
sysread (instead of read) + syswrite() would've made a difference,
I very much hope, at least.
> > f = File.open("bar", File::RDWR|File::CREAT, 0600)
^-- btw, I would put File::TRUNC
there just in case because I
got 10 the 2nd time running it :x
> > f.sync=true
> > f.read # why is this here?
> > f << "Hello" # or f.syswrite
> >
> > # would an explicit f.flush here work? shouldn't be needed
> > # with f.sync = true
> > puts "Should be 5: #{f.pos}"
> > f.close
> >
> > > I couldn't reproduce this on 1.9.1 on OS X or 1.8.7 on Linux.
This could be a stdio bug... 1.9.1 uses only the saner unistd.h
functions so it's immune to stdio bugs, but 1.8.7 uses stdio with
explicit fflush() when sync=true instead of calling
setvbuf(..._IONBF...) to disable buffering like a normal app would.
Looking at io.c in 1.8.7-p72, it even seems to call
setvbuf(..._IOFBF...) for full buffering on FreeBSD regardless
of sync mode (!)
> > I'll definitely need help with testing this then since I only have
> > Linux.
Does the following patch fix things for you?
diff --git a/lib/unicorn/util.rb b/lib/unicorn/util.rb
index 6b35426..0f517e9 100644
--- a/lib/unicorn/util.rb
+++ b/lib/unicorn/util.rb
@@ -54,7 +54,6 @@ module Unicorn
end
File.unlink(fp.path)
fp.binmode
- fp.sync = true
fp
end
---
On the flip side, I'm once again tempted to use sysread/sysseek/syswrite
because I have a very intense and long-standing distrust of luserspace
IO buffering libraries.
--
Eric Wong
^ permalink raw reply related [relevance 2%]
* Re: workers does not seems to exit when served one client
@ 2009-11-04 5:33 2% ` HaiMing Yin
0 siblings, 0 replies; 200+ results
From: HaiMing Yin @ 2009-11-04 5:33 UTC (permalink / raw)
To: unicorn list
On Wed, Nov 4, 2009 at 12:56 AM, Eric Wong <normalperson@yhbt.net> wrote:
>
> You're doing nothing wrong except misunderstanding that phrase.
>
> "one client at a time" means it's not possible to be servicing more
> clients than there are worker_processes (the kernel will buffer them).
Thanks for your explanation and sorry for my misunderstanding.
I'm running unicorn at production about one week, it works great, tnx
again for your work!
--
Regards,
Josh.Yin/殷海明
Friendfeed: http://friendfeed.com/yinhm
_______________________________________________
mongrel-unicorn mailing list
mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
^ permalink raw reply [relevance 2%]
* Re: Nginx Conf
@ 2009-11-24 7:19 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2009-11-24 7:19 UTC (permalink / raw)
To: unicorn list
Dylan Stamat <dstamat@elctech.com> wrote:
> On Nov 22, 2009, at 2:53 PM, Eric Wong wrote:
> > One general thing about the nginx configs I've seen is that they're
> > missing the fail_timeout=0 directive in the "server" lines.
> >
> > I highly recommend setting it since it's a low cost to try an upstream
> > for nginx, and you can avoid 502 errors in case there's a bug in your
> > app that causes a Unicorn worker to not send a valid HTTP response
> > (including hitting the app timeout).
>
> Hey Eric,
>
> Yeah, an nginx.conf in examples/ would be great.
> It's probably going to be the most widely used front for
> Unicorn/Rainbows!, so a sample config with some explanations here and
> there would be awesome. It was great setting up Unicorn and being
> able to just grab the init.sh out of there!
OK, I'll push out an nginx example in a bit, need to make sure things
are adequately explained and linked.
> In terms of the fail_timeout, I haven't seen a nginx.conf with that
> entry yet! Maybe I'm looking at configuration files on the wrong
> projects ;) So, since the upstream attempts are cheap, the combination
> of a max_fail of 1 and fail_timeout of 0 is ideal? If the
> fail_timeout was at 10, and all the upstreams timed out, wouldn't it
> restart at the beginning of the round-robin, and not block... or...
> would it actually not retry on any due to the inoperable state?
max_fails doesn't seem to have any effect when fail_timeout=0. Setting
max_fails=0 means the same thing as fail_timeout=0 in nginx >=0.6.33
(it would segfault before :x) I've just been using fail_timeout=0
since what seems like forever in nginx...
Reading ngx_http_upstream_round_robin.c again, it appears seems that
fail_timeout/max_fails is supposed to get ignored if *all* upstreams are
failing. So if you have a single upstream it looks like you're safe
even if your Unicorn master nukes workers. On the other hand I remember
this not being the case at some point a while back; an entire cluster
got effectively shut down because of it. Just set fail_timeout=0
and stop worrying about it :)
--
Eric Wong
^ permalink raw reply [relevance 2%]
* Re: feature request - when_ready() hook
@ 2009-11-30 23:47 2% ` Suraj Kurapati
0 siblings, 0 replies; 200+ results
From: Suraj Kurapati @ 2009-11-30 23:47 UTC (permalink / raw)
To: unicorn list
On Thu, Nov 26, 2009 at 11:53 AM, Eric Wong <normalperson@yhbt.net> wrote:
> Suraj Kurapati <sunaku@gmail.com> wrote:
>> the XML dataset loading
>> (see above) kept increasing the master's (and the new set of workers')
>> memory footprint by 1.5x every time Unicorn was restarted via SIGUSR2.
>
> Side problem, but another thing that makes me go "Huh?"
>
> Did the new master's footprint increase?
Yes, but this seems to have been my fault. I programmed the master
(via the Unicorn configuration file) to accept a SIGPWR signal which
made it (1) reload the XML dataset (i.e. added memory bloat) and (2)
send SIGUSR2 to itself (thereby providing the new XML data to the new
Unicorn master + workers). The idea was to cut down the time required
for loading the XML dataset. Unfortunately, the extra memory bloat
added by reloading the XML dataset carried over to the new Unicorn
generation as a side-effect.
> Are you mmap()-ing the XML dataset?
Nope, nothing fancy like that.
> Is RSS increasing or just VmSize?
Hmm, I did not pay attention to these individual stats. I just saw
that the memory% statistic in `ps xv` would increase by 10% every time
Unicorn was restarted through my SIGPWR handler. Again, this was my
fault for using such a non-standard approach.
> Unicorn sets FD_CLOEXEC on
> the first 1024 (non-listener) file descriptors, so combined with exec(),
> that should give the new master (and subsequent workers) a clean memory
> footprint.
Thanks. This is good to know, now that I'm using the standard
SIGUSR2/QUIT method.
>> > At this stage, maybe even implementing something as middleware and
>> > making it hook into request processing (that way you really know the
>> > worker is really responding to requests) is the way to go...
>>
>> Hmm, but that would incur a penalty on each request (check if I've
>> already killed the old master and do it if necessary).
>
> I don't think a runtime condition would be any more expensive than all
> the routing/filters/checks that any Rails app already does and you can
> cache the result into a global variable.
Okay.
> As you may have noticed, I'm quite hesitant to add new features,
> especially for uncommon/rare cases. Things like supporting the
> "working_directory" directive and user-switching took *months* of
> debating with myself before they were finally added.
No problem. I ended up using a simple workaround for this whole
problem: from Capistrano, I send SIGUSR2 to the existing Unicorn
master (which will become the old Unicorn master), wait 90 seconds,
and then send SIGQUIT to the old Unicorn master. There's nothing
fancy in my Unicorn configuration file anymore --- no before/after
hooks at all; just a number of workers + listen directive.
This configuration is working out pretty well, and I have finally
achieved zero downtime deploys. (Yay! :-) The only thing I'm worried
about is that I'll have to keep adjusting this timeout as the
infrastructure my app depends upon becomes slower/faster. A
when_ready() hook would really do wonders for me, and I will implement
and try it as planned when I get some time.
> let us know if it's the DB doing reverse DNS because
> I've seen that to be a problem in a lot of cases.
I'll ask about this and let you know.
Thanks for your consideration.
_______________________________________________
mongrel-unicorn mailing list
mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
^ permalink raw reply [relevance 2%]
* [ANN] unicorn 0.95.2 - small HTTP parser fixes
@ 2009-12-07 10:01 2% Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2009-12-07 10:01 UTC (permalink / raw)
To: ruby-talk ML, mongrel-unicorn
Unicorn is an HTTP server for Rack applications designed to only serve
fast clients on low-latency, high-bandwidth connections and take
advantage of features in Unix/Unix-like kernels. Slow clients should
only be served by placing a reverse proxy capable of fully buffering
both the the request and response in between Unicorn and slow clients.
* http://unicorn.bogomips.org/
* email: mongrel-unicorn@rubyforge.org
* git://git.bogomips.org/unicorn.git
* finger: unicorn@bogomips.org
Changes:
Small fixes to our HTTP parser to allows semicolons in PATH_INFO
as allowed by RFC 2396, section 3.3. This is low impact for
existing apps as semicolons are rarely seen in URIs. Our HTTP
parser runs properly under Rubinius 0.13.0 and 1.0.0-rc1 again
(though not yet the rest of the server since we rely heavily on
signals).
Another round of small documentation tweaks and minor cleanups.
--
Eric Wong
^ permalink raw reply [relevance 2%]
* Re: Expect 100-continue errors
@ 2009-12-09 22:16 2% ` Eric Wong
2009-12-09 23:46 0% ` Jan Dvorak
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2009-12-09 22:16 UTC (permalink / raw)
To: unicorn list
Jan Dvorak <jan.dvorak@kraxnet.cz> wrote:
> Hello,
>
> i'm trying to implement handling of http Expect: 100-continue with
> unicorn in my rails app, but when i return [100] as a response (which
> should force unicorn to return 100 status and rerun my app with the same
> connection) i get this error:
>
> Read error: #<ThreadError: stopping only thread
> note: use sleep to stop forever>
> /var/lib/gems/1.8/gems/actionpack-2.3.5/lib/action_controller/reloader.rb:31:in
> `lock'
Hi Jan,
Try disabling automatic code reloading under Rails development mode.
Unicorn (and Rainbows!) are the only Ruby servers I know of that allow
Rack applications to properly return and continue with 100 responses.
Unicorn calls your Rack app twice in this case: it sees a 100 response,
writes the 100-continue response and then does app.call(env) again after
deleting the "Expect: 100-continue" header from env.
I haven't tested this under Rails so I don't know how well it works with
what Rails does under the covers, but I've done this heavily with bare
Rack applications.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Expect 100-continue errors
2009-12-09 22:16 2% ` Eric Wong
@ 2009-12-09 23:46 0% ` Jan Dvorak
0 siblings, 0 replies; 200+ results
From: Jan Dvorak @ 2009-12-09 23:46 UTC (permalink / raw)
To: mongrel-unicorn
Eric Wong wrote:
> Hi Jan,
>
> Try disabling automatic code reloading under Rails development mode.
>
>
Thanks, that worked.
> Unicorn (and Rainbows!) are the only Ruby servers I know of that allow
> Rack applications to properly return and continue with 100 responses.
>
> Unicorn calls your Rack app twice in this case: it sees a 100 response,
> writes the 100-continue response and then does app.call(env) again after
> deleting the "Expect: 100-continue" header from env.
>
> I haven't tested this under Rails so I don't know how well it works with
> what Rails does under the covers, but I've done this heavily with bare
> Rack applications.
>
>
After googling, it seems like rails error, it creates locks but never
properly releases them, and deadlocks upon class reloading.
Thanks,
Jan
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* PATH_INFO spec (with regard to ";")
@ 2009-12-10 22:30 2% Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2009-12-10 22:30 UTC (permalink / raw)
To: rack-devel; +Cc: mongrel-unicorn, Marc-Andre Cournoyer
Hi all,
I've been notified privately that my changes for PATH_INFO in Unicorn
0.95.2 (which also got into Thin) may not be completely kosher, but I'm
also asking for the Rack team to clarify PATH_INFO for HTTP parser
implementers.
Upon further reading (and also of the
related-but-not-necessarily-true-for-Rack RFC 3875 section 4.1.5),
I came across this:
Unlike a URI path, the PATH_INFO is not URL-encoded, and cannot
contain path-segment parameters.
First off, Rack already directly contradicts the "the PATH_INFO is not
URL-encoded" part, so Unicorn conforms to Rack specs over RFC 3875.
*But* Rack does not address the "cannot contain path-segment parameters"
part at all. So I (and probably a few other people) would like
clarification on how to handle PATH_INFO when it comes to ";"
Things to keep in mind:
* URI.parse keeps ";" in URI::HTTP#path
This point may not be relevant to us, as PATH_INFO and
URI::HTTP#path should not necessarily be treated as equals
* WEBrick keeps ";" in PATH_INFO
* PEP333 (which Rack is based on) does not go into this level of
detail regarding PATH_INFO and path segments
* PATH_INFO in Rack appears to be based on CGI/1.1 (RFC 3875)
* Again, Rack already contradicts the URL encoding rules of RFC 3875
for PATH_INFO, so there is precedence for Rack contradicting more
of RFC 3875...
* Rack::Request#full_path only looks at PATH_INFO + QUERY_STRING,
this means many Rack applications may never see the ";" parts
if Thin and Unicorn revert to old behavior.
* Rack does not require REQUEST_URI, this is an extension Unicorn
and Thin both carried over from Mongrel.
* None of the official rack/rack-contrib middleware use REQUEST_URI
Of course, in the grand scheme of things, hardly anybody uses ";" in
paths. Yay for rare corner cases making our lives difficult.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* "unicorn -D" always returns 0 "success" (even when failed to load)
@ 2009-12-23 2:20 3% Iñaki Baz Castillo
2009-12-23 7:26 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Iñaki Baz Castillo @ 2009-12-23 2:20 UTC (permalink / raw)
To: mongrel-unicorn
Hi, I'm writing a Debian init script for unicorn and realized that when
starting unicorn with daemonize option, the command always returns 0, even if
the start action failed (due for example Errno::EADDRINUSE).
Returning 0 in such case is not good as it breaks service init scripts or
service controllers (as HeartBeat) that fully rely on the appropriate exit
code.
Is there some way to determine if unicorn failed to start when using "-D"?
Another related issue: When the Rack config.ru file contains some error (as a
typo) the worker(s) returns 1 (at the moment usually). Then unicorn master
process reapes the terminated worker process and restarts it. Of course it
would fail again and again. Anyhow "unicorn -D" returns 0 again (success).
Usually if a worker (all the workers) fail to start at the moment of running
it, it obviously means that there is some error in the application with
prevents it to run. It could be great if Unicorn could detect it.
For that I suggest something as a new option "--validation-time TIME". Let's
suppose TIME is 5 seconds. In case *all* the workers fail within 5 seconds
after starting unicorn, then unicorn understands that the Rack application is
wrong (or any other error as Errno::EADDRINUSE) so terminates all the workers
and itself (and hopefully returns 1 or any other non-zero exit status).
Of course, all the above means that Unicorn should wait TIME seconds before
being daemonized (so after TIME seconds it can decide which code to return).
Does it make sense? Thanks a lot.
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 3%]
* Re: "unicorn -D" always returns 0 "success" (even when failed to load)
2009-12-23 2:20 3% "unicorn -D" always returns 0 "success" (even when failed to load) Iñaki Baz Castillo
@ 2009-12-23 7:26 0% ` Eric Wong
2009-12-25 23:58 2% ` Iñaki Baz Castillo
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2009-12-23 7:26 UTC (permalink / raw)
To: unicorn list
Iñaki Baz Castillo <ibc@aliax.net> wrote:
> Hi, I'm writing a Debian init script for unicorn and realized that when
> starting unicorn with daemonize option, the command always returns 0, even if
> the start action failed (due for example Errno::EADDRINUSE).
>
> Returning 0 in such case is not good as it breaks service init scripts or
> service controllers (as HeartBeat) that fully rely on the appropriate exit
> code.
>
> Is there some way to determine if unicorn failed to start when using "-D"?
Hi Iñaki,
No way to determine that currently, as I've never encountered the need.
For validating startups, most folks I know have specific endpoint(s) in
application dedicated to checks. That way they can get way more info
and all the way down into stack including things like database/memcached
connections.
Anything less is superficial because they can fail to detect other
misconfigurations (including stuff like wrong RAILS_ENV); not just
startup errors.
> Another related issue: When the Rack config.ru file contains some error (as a
> typo) the worker(s) returns 1 (at the moment usually). Then unicorn master
> process reapes the terminated worker process and restarts it. Of course it
> would fail again and again. Anyhow "unicorn -D" returns 0 again (success).
>
> Usually if a worker (all the workers) fail to start at the moment of running
> it, it obviously means that there is some error in the application with
> prevents it to run. It could be great if Unicorn could detect it.
I'm generally hesitant to introduce code/features/bloat that aren't
helpful to people who properly test configurations before deploying :)
Adding this code doesn't solve problems. Either way the site can become
inaccessible/unusable and problems are noticeable very quickly.
If there's sufficient interest, I'll consider accepting a small patch
for this. Right now I'm unconvinced...
> For that I suggest something as a new option "--validation-time TIME". Let's
> suppose TIME is 5 seconds. In case *all* the workers fail within 5 seconds
> after starting unicorn, then unicorn understands that the Rack application is
> wrong (or any other error as Errno::EADDRINUSE) so terminates all the workers
> and itself (and hopefully returns 1 or any other non-zero exit status).
>
> Of course, all the above means that Unicorn should wait TIME seconds before
> being daemonized (so after TIME seconds it can decide which code to return).
>
> Does it make sense? Thanks a lot.
We avoid introducing command-line options since they're unlikely to be
compatible with "rackup" parsing of the "#\" lines in .ru files.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* How to trap USR2 for custom ussage ?
@ 2009-12-23 14:29 2% Iñaki Baz Castillo
2009-12-23 20:47 2% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Iñaki Baz Castillo @ 2009-12-23 14:29 UTC (permalink / raw)
To: mongrel-unicorn
Hi, I've modified bin/unicorn to start a DRb server before last lines:
MyApp::DRbServer.start(Process.pid)
Unicorn::Launcher.daemonize! if daemonize
Unicorn.run(app, options)
DRb server listens in port 5555. When I send a USR2 signal to unicorn master
process I get an error because DRb is started again so:
/usr/local/lib/ruby1.9/1.9.1/drb/drb.rb:861:in `initialize': Address already
in use - bind(2) (Errno::EADDRINUSE)
The only solution I can imagine is to trap USR2 signal:
MyApp::DRbServer.start(Process.pid)
trap("USR2") { ::DRb.stop_service }
Unicorn::Launcher.daemonize! if daemonize
Unicorn.run(app, options)
Unfortunatelly it doesn't work because Unicorn overrides the 'trap' call so I
don't get the USR2 signal.
Is there any way to achive this?
I could imagine a new config option "on_usr2" (or "before_reexec") so the
passsed block would be executed upon receipt of USR2 (before the real "reexec"
method).
Does it make sense?
Thanks a lot.
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: How to trap USR2 for custom ussage ?
2009-12-23 14:29 2% How to trap USR2 for custom ussage ? Iñaki Baz Castillo
@ 2009-12-23 20:47 2% ` Eric Wong
2009-12-24 9:40 2% ` Iñaki Baz Castillo
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2009-12-23 20:47 UTC (permalink / raw)
To: unicorn list
Iñaki Baz Castillo <ibc@aliax.net> wrote:
> Hi, I've modified bin/unicorn to start a DRb server before last lines:
>
> MyApp::DRbServer.start(Process.pid)
> Unicorn::Launcher.daemonize! if daemonize
> Unicorn.run(app, options)
>
> DRb server listens in port 5555. When I send a USR2 signal to unicorn master
> process I get an error because DRb is started again so:
>
> /usr/local/lib/ruby1.9/1.9.1/drb/drb.rb:861:in `initialize': Address already
> in use - bind(2) (Errno::EADDRINUSE)
First off I really don't think you need to be using DRb for this or
(generally) sticking things into the master process that don't belong
there.
> Is there any way to achive this?
> I could imagine a new config option "on_usr2" (or "before_reexec") so the
> passsed block would be executed upon receipt of USR2 (before the real "reexec"
> method).
>
> Does it make sense?
There's already a similar before_exec hook documented in
http://unicorn.bogomips.org/Unicorn/Configurator.html
But it is called after forking, so the listener will be shared.
But again, you don't need to use DRb for this.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: How to trap USR2 for custom ussage ?
2009-12-23 20:47 2% ` Eric Wong
@ 2009-12-24 9:40 2% ` Iñaki Baz Castillo
2009-12-24 19:41 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Iñaki Baz Castillo @ 2009-12-24 9:40 UTC (permalink / raw)
To: mongrel-unicorn
El Miércoles, 23 de Diciembre de 2009, Eric Wong escribió:
> First off I really don't think you need to be using DRb for this or
> (generally) sticking things into the master process that don't belong
> there.
The problem is: how to start a server (which binds in a TCP port) in just one
worker? humm, would the following make sense?:
after_fork do |server, worker|
# Start DRb server just in worker[0]
if worker.nr == 0
... start DRb server ...
end
end
But it would also fail upon receipt of USR2 as there would be two instances of
the DRb server trying to bind on same port...
> > Is there any way to achive this?
> > I could imagine a new config option "on_usr2" (or "before_reexec") so the
> > passsed block would be executed upon receipt of USR2 (before the real
> > "reexec" method).
> >
> > Does it make sense?
>
> There's already a similar before_exec hook documented in
> http://unicorn.bogomips.org/Unicorn/Configurator.html
> But it is called after forking, so the listener will be shared.
Hummm, perhaps I could use the above code plus:
before_exec do |server|
.... stop DRb server ...
end
In this way the DRb server is stopped before exec and started in the new
worker[0].
Just a problem, how to access to existing/old worker[0] from "before_exec" in
order to stop the DRb server?
> But again, you don't need to use DRb for this.
As I've said in other thread I want DRb to get some info related to the
application. A DRb client console could be used to check application
database(s) connection and so.
What would you suggest for this rather than DRb server?
Again thanks a lot for your help.
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: How to trap USR2 for custom ussage ?
2009-12-24 9:40 2% ` Iñaki Baz Castillo
@ 2009-12-24 19:41 0% ` Eric Wong
2009-12-25 22:16 2% ` Iñaki Baz Castillo
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2009-12-24 19:41 UTC (permalink / raw)
To: unicorn list
Iñaki Baz Castillo <ibc@aliax.net> wrote:
> El Miércoles, 23 de Diciembre de 2009, Eric Wong escribió:
> > But again, you don't need to use DRb for this.
>
> As I've said in other thread I want DRb to get some info related to the
> application. A DRb client console could be used to check application
> database(s) connection and so.
> What would you suggest for this rather than DRb server?
As I said before, write an endpoint in your HTTP application that
calls and displays all the needed info you need. Something that
hits the database(s) and/or whatever else you're using.
There's no need to introduce another socket protocol into that process.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: How to trap USR2 for custom ussage ?
2009-12-24 19:41 0% ` Eric Wong
@ 2009-12-25 22:16 2% ` Iñaki Baz Castillo
0 siblings, 0 replies; 200+ results
From: Iñaki Baz Castillo @ 2009-12-25 22:16 UTC (permalink / raw)
To: mongrel-unicorn
El Jueves, 24 de Diciembre de 2009, Eric Wong escribió:
> > As I've said in other thread I want DRb to get some info related to the
> > application. A DRb client console could be used to check application
> > database(s) connection and so.
> > What would you suggest for this rather than DRb server?
>
> As I said before, write an endpoint in your HTTP application that
> calls and displays all the needed info you need. Something that
> hits the database(s) and/or whatever else you're using.
Can Rack/Unicorn behave different depending on the request received port?
This is: I could run Unicorn in port 80 for clients access and port 5000 for
admin access (status and so).
However I couldn't find in Rack specification, neither in Unicorn, a way to
inspect the port in which the request was received by the http server. Is it
possible?
> There's no need to introduce another socket protocol into that process.
Yes, I'm realizing that openging another socet protocol gets in conflict with
Unicorn signals usage. For example USR2 + QUIT is really great as it allows to
reload/restart a server and go back to the previos running instance if the new
one fails in some way.
So I'll use HTTP instead of DRb protocol to display the server status.
Thanks a lot again.
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: "unicorn -D" always returns 0 "success" (even when failed to load)
2009-12-23 7:26 0% ` Eric Wong
@ 2009-12-25 23:58 2% ` Iñaki Baz Castillo
0 siblings, 0 replies; 200+ results
From: Iñaki Baz Castillo @ 2009-12-25 23:58 UTC (permalink / raw)
To: mongrel-unicorn
El Miércoles, 23 de Diciembre de 2009, Eric Wong escribió:
> > Usually if a worker (all the workers) fail to start at the moment of
> > running it, it obviously means that there is some error in the
> > application with prevents it to run. It could be great if Unicorn could
> > detect it.
>
> I'm generally hesitant to introduce code/features/bloat that aren't
> helpful to people who properly test configurations before deploying :)
>
> Adding this code doesn't solve problems. Either way the site can become
> inaccessible/unusable and problems are noticeable very quickly.
Ok, I should explain better why I need this feature:
I'm not coding a HTTP app on top of Unicorn to host in a server. Instead I'm
building a XCAP server (RFC 4825) using Rack and Unicorn. I'll also publish a
Ruby gem containing all the server (Unicorn + Rack application).
So, any user could install the gem, set the database(s), fill the
configuration file and star the server in background by running the provided
init script ("/etc/init.d/xcapserver start") which runs the server daemonized.
Imagine that the user wrote a typo, i.e:
after_fork do |server, worker|
# Note the space typo:
worker.u ser("www-data", "www-data") if Process.euid == 0
end
Then the init script would return 0 (success) but the server wouldn't work.
The user would check "ps aux | grep xcapserver" and would see the xcapserver
running. It would be complex to understand why the server it's failing.
So it would be great the following:
- The user start the init script (which calls "unicorn -D ...") with a
hypotethical new option ("--no-error-time 2").
- The command "unicorn -D" remains in foreground for 2 seconds before master
process going to background.
- Unicorn fails to create the workers at the moment (so within 2 seconds).
- Instead of re-spawning them, the command (still in foreground) exits with
return code 1 (error).
But there is other case which is much wrose IMHO. Imagine the user writes a
space typo:
stde rr_path "/var/log/xcapserver.err.log"
or imagine it uses a path that doesn't exist (/var/log/xcapserver/ doesn't
exist):
stderr_path "/var/log/xcapserver/err.log"
In both cases "unicorn -D" returns 0 but the server is not running (no unicorn
process running). So, why did it return 0? It's not an error when creating the
workers, but when running also the master process. IMHO in this case Unicorn
should return non zero (even if it called with "-D").
I'm playing right now with unicorn/launcher.rb (daemonize! method)but I get
nothing. Unfortunatelly I think that the usage of fork makes very difficult
the main command to behave as I desire (exit status).
If you could give me some tips I'd try to implement it.
Reall thanks a lot again and again :)
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: "unicorn -D" always returns 0 "success" (even when failed to load)
@ 2009-12-26 15:47 1% ` Iñaki Baz Castillo
0 siblings, 0 replies; 200+ results
From: Iñaki Baz Castillo @ 2009-12-26 15:47 UTC (permalink / raw)
To: mongrel-unicorn
El Sábado, 26 de Diciembre de 2009, Eric Wong escribió:
> > http://oversip.net/public/min_time_running.rb
> >
> > It contains a modification for bin/unicorn and a rewritten
> > lib/unicorn/launcher.rb.
>
> Interesting, I could be tempted to accept this with a few changes...
>
> Process.detach() spawns a background Thread. Having running threads
> before forking won't fly with certain OS threading libraries (some
> versions of FreeBSD, I believe, check MRI Redmine for some open bugs).
Yes, but there is a problem:
Imagine I don't use Process.detach and the master process ends when loads the
rack application (due to a typo or due to an error in unicorn conf file).
Then the master process would remain visible/alive but as zombie.
If so, when the controller process sends signal 0 to check the master process
it will always receive 'true' (a zombie process is a process which can receive
signals, it's does exist).
If I could check if the master process is zombie then I wouldn't need
Process.detach, but I don't know how to check if a process is zombie, no way
in:
http://ruby-doc.org/core/classes/Process.html
http://ruby-doc.org/core/classes/Process/Status.html
Any suggestion here?
> Use trap(Symbol), trap(Integer) is harder to read and has a likelyhood
> of being non-portable for certain signals. Likewise with Process.kill
> (0 being the only exception, of course).
Sure, I'll change it.
> I would trap() in the parent before any forking, in case the
> sleep time is ever so low there's a race condition.
Ok.
> Probably no need to handle USR1, either, just let the parent die on its own,
Note that the parent sleeps for 'min_running_time' + 1 (to avoid exiting
before receiving USR1 or USR2 from the controller process). But if the
controller process sends USR1 then the parent can already exit without waiting
one second more.
Well, this is not very interesting, but I think it's good in order to get a
faster exit code (it's not a good idea that a daemon takes long time to start
and return ans exit status).
> or alternatively, have the parent sleep forever in case something
> unforseen happens in the children...
But then the parent process doesn't end and continues in foreground. This is
not valid for running as daemon. The parent process must exit with the
appropriate status code after a few seconds to behave as any service running
in background.
> > The code works for me for the two cases I explained in my previous mail.
> > Of course it's an optional feature (I've implemented it with "-M --min-
> > running-time SECONDS" in the options parser).
>
> I believe the nginx grace period is hard-coded at 5 seconds, which should
> be enough.
Sorry, I don't know ngix. I need to use other reverse proxy (Pound) but let me
open a new thread about it :)
Anyhow I don't understand what you mean with "nginx grace period". How does
affect to Unicorn? Do you mean that, in case of implementing the above,
Unicorn wouldn't need a --min-running-time option? (I expect it would be
required since Unicorn is 100% independent of ngix)
Really thanks a lot. I'll do the changes you suggest and show you again (but
still don't know how to avoid Process.detach for now...).
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 1%]
* Re: "unicorn -D" always returns 0 "success" (even when failed to load)
@ 2009-12-27 3:06 4% ` Iñaki Baz Castillo
2009-12-28 3:29 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Iñaki Baz Castillo @ 2009-12-27 3:06 UTC (permalink / raw)
To: mongrel-unicorn
El Domingo, 27 de Diciembre de 2009, Eric Wong escribió:
> > Hi, I've improved it a bit with your suggestions and also simplified
> > it a lot. Now there is no a "controller process". Instead, if
> > "Unicorn.run" raises then master process rescues the exception and
> > sends USR1 to parent process so it exists with 1.
>
> Hi, I realized Unicorn.run inside the daemonize method doesn't work.
Strange, it works for me...
> However, I've reimplemented much of your idea here so it does not
> involve sleeping at all, but rather shares a pipe with the grandparent
> (started by the user) and Unicorn master process. The added benefit of
> using a pipe is that there's no fuzzy sleeping involved at or grace
> periods to worry about, too.
>
> Also pushed out to the "ready_pipe" branch of
> git://git.bogomips.org/unicorn.git
>
> Let me know how it goes.
>
> If everything looks good, I'll consider merging for 0.96.0.
It's really awesome! I've tested it and it works great, and avoids the
sleeping stuff and a new commandlline option. Congratulations :)
However there is still a not solved case. Let me please explain it with two
examples:
1) In "after_fork" I set:
worker.user("non_existing_user","non_existing_group")
The master process would start properly and would notify "success" to
grandparent. (so the init script returns 0). But the fact is that all the
workers fail to start and are respawned again and again.
2) I use "Clogger" in my Rack config.ru. But I set a log file in a path that
doesn't exist. Clogger raises due to this (it doesn't try to create the full
path).
Again the master process has notified "success" to grandparent (exis status 0
then) but the workers are respawned again and again.
In both cases, if "preload_app" is false then doing a "ps" I see new master
processes being created by first master process (with new workers) and all of
them die while new ones born.
If "preload_app" is false then there are no borning master processes as just
the workers fail to start when loading the Rack app.
It would be great if the same mechanims (master notifies to grandparent) would
be implemented between first process loading the Rack app (a worker if
"preload_app") and master process, so if the first worker fails when loading
the Rack app then master process doesn't notify "success" to grandparent and
exists (instead of respawning again and again the workers).
If "preload_app" is true then I don't understand why your new code doesn't
work right now. I expect that master process notifies "success" to grandparent
before loading the Rack app, am I right? would make sense master process to
wait until Rack app is loaded and if it fails then notify "error" and exit
completely?
Kind regards and thanks a lot for your great work.
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 4%]
* Re: "unicorn -D" always returns 0 "success" (even when failed to load)
2009-12-27 3:06 4% ` Iñaki Baz Castillo
@ 2009-12-28 3:29 0% ` Eric Wong
2009-12-28 10:39 0% ` Iñaki Baz Castillo
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2009-12-28 3:29 UTC (permalink / raw)
To: unicorn list
Iñaki Baz Castillo <ibc@aliax.net> wrote:
> El Domingo, 27 de Diciembre de 2009, Eric Wong escribió:
> > > Hi, I've improved it a bit with your suggestions and also simplified
> > > it a lot. Now there is no a "controller process". Instead, if
> > > "Unicorn.run" raises then master process rescues the exception and
> > > sends USR1 to parent process so it exists with 1.
> >
> > Hi, I realized Unicorn.run inside the daemonize method doesn't work.
>
> Strange, it works for me...
"Doesn't work" as in it's not friendly to servers based on Unicorn (like
Rainbows!).
> > However, I've reimplemented much of your idea here so it does not
> > involve sleeping at all, but rather shares a pipe with the grandparent
> > (started by the user) and Unicorn master process. The added benefit of
> > using a pipe is that there's no fuzzy sleeping involved at or grace
> > periods to worry about, too.
> >
> > Also pushed out to the "ready_pipe" branch of
> > git://git.bogomips.org/unicorn.git
> >
> > Let me know how it goes.
> >
> > If everything looks good, I'll consider merging for 0.96.0.
>
> It's really awesome! I've tested it and it works great, and avoids the
> sleeping stuff and a new commandlline option. Congratulations :)
The current version is actually slightly buggy in that it leaks
the pipe descriptor...
> However there is still a not solved case. Let me please explain it with two
> examples:
>
> 1) In "after_fork" I set:
> worker.user("non_existing_user","non_existing_group")
>
> The master process would start properly and would notify "success" to
> grandparent. (so the init script returns 0). But the fact is that all the
> workers fail to start and are respawned again and again.
For that particular case there'll be a Unicorn::Configurator#user
directive.
But really, there's absolutely no good reason to use user switching in a
backend application server like Unicorn.
I only added that feature to support derivative servers like Rainbows!,
and even then it's debatable since using things like iptables or load
balancers can be used to redirect port 80 to arbitrary ports anyways.
> 2) I use "Clogger" in my Rack config.ru. But I set a log file in a path that
> doesn't exist. Clogger raises due to this (it doesn't try to create the full
> path).
> Again the master process has notified "success" to grandparent (exis status 0
> then) but the workers are respawned again and again.
There's really an infinite number of ways to screw things up badly in
workers and cause them to flap. It's not possible to protect careless
users from all of them, so attempting to do so would be a waste of
effort.
Avoiding user-switching in an app server is a great first step, as it
eliminates an entire class of problems :)
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: "unicorn -D" always returns 0 "success" (even when failed to load)
2009-12-28 3:29 0% ` Eric Wong
@ 2009-12-28 10:39 0% ` Iñaki Baz Castillo
0 siblings, 0 replies; 200+ results
From: Iñaki Baz Castillo @ 2009-12-28 10:39 UTC (permalink / raw)
To: mongrel-unicorn
El Lunes, 28 de Diciembre de 2009, Eric Wong escribió:
> > The master process would start properly and would notify "success" to
> > grandparent. (so the init script returns 0). But the fact is that all the
> > workers fail to start and are respawned again and again.
>
> For that particular case there'll be a Unicorn::Configurator#user
> directive.
>
> But really, there's absolutely no good reason to use user switching in a
> backend application server like Unicorn.
>
> I only added that feature to support derivative servers like Rainbows!,
> and even then it's debatable since using things like iptables or load
> balancers can be used to redirect port 80 to arbitrary ports anyways.
Well, chaning the running user it's common in most of servers. I've already
found lots of cases of attacks to Apache servers running some "cool" PHP
application (so we get exploits in /tmp or/var/tmp as they are the only
writable paths for "www-data" user running apache).
However it's true that Unicorn approach (worker.user) is different as the
master process remains as root (but since the master process doesn't listen it
shouldn't matter).
So, do you mean that there will be a new configuration option called "user"
(and "group") so also themaster process would run as such user?
Thanks.
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Raised exceptions are not logged when USR2
@ 2010-01-02 21:33 2% ` Iñaki Baz Castillo
2010-01-03 0:04 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Iñaki Baz Castillo @ 2010-01-02 21:33 UTC (permalink / raw)
To: mongrel-unicorn
El Sábado, 2 de Enero de 2010, Iñaki Baz Castillo escribió:
> A simpler approach would be removing this line at the end of launcher.rb:
>
> Unicorn::Configurator::DEFAULTS[:stderr_path] = "/dev/null"
>
> With this, errors would be displayed in stderr. Of course for workers
> stderr should be reopened to point to /dev/null, but IMHO not for master
> process.
>
> Perhaps errors preventing the master process to start should be displayed
> to stderr instead of to the logger (as it could be not set yet).
And the other already available workaround is setting the "stderr_path".
Unfortunatelly $stderr.reopen requires a file as parameter so is not possible
to use Syslog or any other logger, is it?
BTW, how to reset the $stderr so it points again to the default one (the
terminal)?
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* unicorn/reexec/bundler/cap deploy issue
@ 2010-01-02 22:59 2% skaar
2010-01-03 0:25 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: skaar @ 2010-01-02 22:59 UTC (permalink / raw)
To: mongrel-unicorn
Hi Eric,
we've run into an issue with Unicorn's reexec, when the rails app is
using Bundler and we deploy into dated release directories. Still not
sure what the correct fix is for this, but what happens is that bundler
modifies (acutally prefixes) PATH/RUBYOPT and when you reexec unicorn
these are passed along to the new master process - and Bundler will
again append to the PATH/RUBYOBT.
This becomes more of a problem when you deploy with capistrano and
include SIGUSR2 restart of your unicorn. And effectively it will attempt
to load all Bundler vendor/bundler_gems/environment.rb for all deploys
since unicorn was first started - and if you have, say, a keep only 5
releases strategy, you will soon get failures when a bundler
environment.rb file in RUBYOPT has been shifted out.
I'm still not sure what the right solution is (to modify unicorn, the
unicorn.rb config file or to address it in bundler), but the following
patch does it brute force in unicorn and introduce the assumption that
a reexec should take the same PATH/RUBYOPT variables as the initial
invokation of the unicorn process (using a similar strategy as with
PWD):
>From 3d6ca163723148668d69115ebaf00cb814db8f49 Mon Sep 17 00:00:00 2001
From: skaar <skaar@waste.org>
Date: Sat, 2 Jan 2010 12:29:38 -0500
Subject: [PATCH] filter PATH/RUBYOPT when rexec, and give new master same values as the
original master process
---
lib/unicorn.rb | 4 ++++
1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 225e00a..6b3e334 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -111,6 +111,8 @@ module Unicorn
Dir.pwd
end
}.call,
+ :path => ENV['PATH'],
+ :rybyopt => ENV['RUBYOPT'],
0 => $0.dup,
}
@@ -488,6 +490,8 @@ module Unicorn
self.reexec_pid = fork do
listener_fds = LISTENERS.map { |sock| sock.fileno }
ENV['UNICORN_FD'] = listener_fds.join(',')
+ ENV['PATH'] = START_CTX[:path]
+ ENV['RUBYOPT'] = START_CTX[:rubyopt]
Dir.chdir(START_CTX[:cwd])
cmd = [ START_CTX[0] ].concat(START_CTX[:argv])
--
1.6.6.rc3
--
/skaar
skaar@waste.org
where in the W.A.S.T.E is the wisdom
s_u_b_s_t_r_u_c_t_i_o_n
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 2%]
* unicorn/reexec/bundler/cap deploy issue
@ 2010-01-02 18:29 2% skaar
0 siblings, 0 replies; 200+ results
From: skaar @ 2010-01-02 18:29 UTC (permalink / raw)
To: mongrel-unicorn
Hi Eric,
we've run into an issue with Unicorn's reexec, when the rails app is
using Bundler and we deploy into dated release directories. Still not
sure what the correct fix is for this, but what happens is that bundler
modifies (acutally prefixes) PATH/RUBYOPT and when you reexec unicorn
these are passed along to the new master process - and Bundler will
again append to the PATH/RUBYOBT.
This becomes more of a problem when you deploy with capistrano and
include SIGUSR2 restart of your unicorn. And effectively it will attempt
to load all Bundler vendor/bundler_gems/environment.rb for all deploys
since unicorn was first started - and if you have, say, a keep only 5
releases strategy, you will soon get failures when a bundler
environment.rb file in RUBYOPT has been shifted out.
I'm still not sure what the right solution is (to modify unicorn, the
unicorn.rb config file or to address it in bundler), but the following
patch does it brute force in unicorn and introduce the assumption that
a reexec should take the same PATH/RUBYOPT variables as the initial
invokation of the unicorn process (using a similar strategy as with
PWD):
>From 3d6ca163723148668d69115ebaf00cb814db8f49 Mon Sep 17 00:00:00 2001
From: skaar <skaar@waste.org>
Date: Sat, 2 Jan 2010 12:29:38 -0500
Subject: [PATCH] filter PATH/RUBYOPT when rexec, and give new master same values as the
original master process
---
lib/unicorn.rb | 4 ++++
1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 225e00a..6b3e334 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -111,6 +111,8 @@ module Unicorn
Dir.pwd
end
}.call,
+ :path => ENV['PATH'],
+ :rybyopt => ENV['RUBYOPT'],
0 => $0.dup,
}
@@ -488,6 +490,8 @@ module Unicorn
self.reexec_pid = fork do
listener_fds = LISTENERS.map { |sock| sock.fileno }
ENV['UNICORN_FD'] = listener_fds.join(',')
+ ENV['PATH'] = START_CTX[:path]
+ ENV['RUBYOPT'] = START_CTX[:rubyopt]
Dir.chdir(START_CTX[:cwd])
cmd = [ START_CTX[0] ].concat(START_CTX[:argv])
--
1.6.6.rc3
--
/skaar
skaar@waste.org
where in the W.A.S.T.E is the wisdom
s_u_b_s_t_r_u_c_t_i_o_n
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 2%]
* Re: Raised exceptions are not logged when USR2
2010-01-02 21:33 2% ` Iñaki Baz Castillo
@ 2010-01-03 0:04 0% ` Eric Wong
2010-01-03 16:22 0% ` Iñaki Baz Castillo
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2010-01-03 0:04 UTC (permalink / raw)
To: unicorn list
Iñaki Baz Castillo <ibc@aliax.net> wrote:
> El Sábado, 2 de Enero de 2010, Iñaki Baz Castillo escribió:
> > A simpler approach would be removing this line at the end of launcher.rb:
> >
> > Unicorn::Configurator::DEFAULTS[:stderr_path] = "/dev/null"
> >
> > With this, errors would be displayed in stderr. Of course for workers
> > stderr should be reopened to point to /dev/null, but IMHO not for master
> > process.
> >
> > Perhaps errors preventing the master process to start should be displayed
> > to stderr instead of to the logger (as it could be not set yet).
>
> And the other already available workaround is setting the "stderr_path".
> Unfortunatelly $stderr.reopen requires a file as parameter so is not possible
> to use Syslog or any other logger, is it?
Hi Iñaki,
Unfortunately, the stock Syslog module isn't compatible with the Ruby
Logger implementation. Try grabbing the "SyslogLogger" gem and using
that instead. I haven't had any experience with SyslogLogger, but a
cursory look says it's OK.
I would always keep stderr_path set somewhere in your config file since
by default "rack.errors" points to it, and it's the default place for
Kernel#warn to output to.
If you really want to, you may be able to subclass SyslogLogger and give
it IO-like methods (write/puts/flush) are require for Rack::Lint
compatibility and then set $stderr:
# Totally untested, stick this in your Unicorn config file and let
# us know if it works or blows up badly:
require 'syslog_logger'
class MySyslogLogger < SyslogLogger
alias puts error
alias write error
def flush; self; end
end
$stderr = MySyslogLogger.new('foo')
> BTW, how to reset the $stderr so it points again to the default one (the
> terminal)?
You would've had to dup stderr in the config file and keep it referenced
before it got redirected. But a daemonized process shouldn't be holding
onto terminals...
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: unicorn/reexec/bundler/cap deploy issue
2010-01-02 22:59 2% skaar
@ 2010-01-03 0:25 0% ` Eric Wong
2010-01-03 17:08 2% ` skaar
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2010-01-03 0:25 UTC (permalink / raw)
To: unicorn list
skaar <skaar@waste.org> wrote:
> Hi Eric,
>
> we've run into an issue with Unicorn's reexec, when the rails app is
> using Bundler and we deploy into dated release directories. Still not
> sure what the correct fix is for this, but what happens is that bundler
> modifies (acutally prefixes) PATH/RUBYOPT and when you reexec unicorn
> these are passed along to the new master process - and Bundler will
> again append to the PATH/RUBYOBT.
>
> This becomes more of a problem when you deploy with capistrano and
> include SIGUSR2 restart of your unicorn. And effectively it will attempt
> to load all Bundler vendor/bundler_gems/environment.rb for all deploys
> since unicorn was first started - and if you have, say, a keep only 5
> releases strategy, you will soon get failures when a bundler
> environment.rb file in RUBYOPT has been shifted out.
>
> I'm still not sure what the right solution is (to modify unicorn, the
> unicorn.rb config file or to address it in bundler),
Hi skaar,
Does having this in the Unicorn config file work for you?
stash_env = %w(PATH RUBYOPT).map { |x| [ x, ENV[x] ] }
before_exec do |_|
stash_env.each { |(k,v)| ENV[k] = v }
end
It's more generic and less surprising for people that (may) want
to change environment variables on upgrades:
> but the following
> patch does it brute force in unicorn and introduce the assumption that
> a reexec should take the same PATH/RUBYOPT variables as the initial
> invokation of the unicorn process (using a similar strategy as with
> PWD):
Early versions of Unicorn actually captured the entire ENV at startup
and restored it before exec. I felt it was too heavy-handed since I
figured any process that changed its environment variables has good
reason to do so, so I've been trying to leave as much alone as
possible...
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Raised exceptions are not logged when USR2
2010-01-03 0:04 0% ` Eric Wong
@ 2010-01-03 16:22 0% ` Iñaki Baz Castillo
0 siblings, 0 replies; 200+ results
From: Iñaki Baz Castillo @ 2010-01-03 16:22 UTC (permalink / raw)
To: mongrel-unicorn
El Domingo, 3 de Enero de 2010, Eric Wong escribió:
> Unfortunately, the stock Syslog module isn't compatible with the Ruby
> Logger implementation. Try grabbing the "SyslogLogger" gem and using
> that instead. I haven't had any experience with SyslogLogger, but a
> cursory look says it's OK.
Yes, in fact I already use a "bit" customized SyslogLogger :)
> If you really want to, you may be able to subclass SyslogLogger and give
> it IO-like methods (write/puts/flush) are require for Rack::Lint
> compatibility and then set $stderr:
>
> # Totally untested, stick this in your Unicorn config file and let
> # us know if it works or blows up badly:
>
> require 'syslog_logger'
>
> class MySyslogLogger < SyslogLogger
> alias puts error
> alias write error
> def flush; self; end
> end
>
> $stderr = MySyslogLogger.new('foo')
Really great! It works fine :)
> > BTW, how to reset the $stderr so it points again to the default one (the
> > terminal)?
>
> You would've had to dup stderr in the config file and keep it referenced
> before it got redirected. But a daemonized process shouldn't be holding
> onto terminals...
Really thanks a lot.
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: unicorn/reexec/bundler/cap deploy issue
2010-01-03 0:25 0% ` Eric Wong
@ 2010-01-03 17:08 2% ` skaar
0 siblings, 0 replies; 200+ results
From: skaar @ 2010-01-03 17:08 UTC (permalink / raw)
To: unicorn list
> Hi skaar,
>
> Does having this in the Unicorn config file work for you?
>
> stash_env = %w(PATH RUBYOPT).map { |x| [ x, ENV[x] ] }
> before_exec do |_|
> stash_env.each { |(k,v)| ENV[k] = v }
> end
>
> It's more generic and less surprising for people that (may) want
> to change environment variables on upgrades:
yep, that does the job just fine - thanks.
> Early versions of Unicorn actually captured the entire ENV at startup
> and restored it before exec. I felt it was too heavy-handed since I
> figured any process that changed its environment variables has good
> reason to do so, so I've been trying to leave as much alone as
> possible...
the approach of just stashing what you need from ENV as above is by far
a better approach than just forcing in the reexec code.
thanks again
--
/skaar
skaar@waste.org
where in the W.A.S.T.E is the wisdom
s_u_b_s_t_r_u_c_t_i_o_n
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Strange Thread related errors
@ 2010-01-07 16:05 1% Michael Guterl
2010-01-07 20:13 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Michael Guterl @ 2010-01-07 16:05 UTC (permalink / raw)
To: unicorn list
Our Rails app has started raising exceptions (caught by hoptoad
thankfully) and I can only imagine they're related to unicorn. I only
*think* the errors are occurring on the request after we deploy, which
upgrades the Unicorn process. I say this because the errors are
coming from many different actions, but I haven't been able to
reproduce.
After the most recent batch of errors, I upgraded from 0.95.1 to
0.95.3, but we have not deployed again.
ThreadError: stopping only thread note: use sleep to stop forever
/usr/local/lib/ruby/1.8/monitor.rb:285:in `stop'
/usr/local/lib/ruby/1.8/monitor.rb:285:in `mon_acquire'
/usr/local/lib/ruby/1.8/monitor.rb:214:in `mon_enter'
/usr/local/lib/ruby/1.8/monitor.rb:240:in `synchronize'
/usr/local/lib/ruby/1.8/logger.rb:496:in `write'
/usr/local/lib/ruby/1.8/logger.rb:326:in `add'
/usr/local/lib/ruby/1.8/logger.rb:374:in `info'
/home/deploy/public_html/rm/releases/20100107153636/vendor/plugins/newrelic_rpm/lib/new_relic/agent/agent.rb:46:in
`ensure_worker_thread_started'
/home/deploy/public_html/rm/releases/20100107153636/vendor/plugins/newrelic_rpm/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:248:in
`perform_action'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/base.rb:532:in
`send'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/base.rb:532:in
`process_without_filters'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/filters.rb:606:in
`process'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/base.rb:391:in
`process'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/base.rb:386:in
`call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/routing/route_set.rb:437:in
`call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:87:in
`dispatch'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:121:in
`_call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:130:in
`build_middleware_stack'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/query_cache.rb:29:in
`call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/query_cache.rb:29:in
`call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb:34:in
`cache'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/query_cache.rb:9:in
`cache'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/query_cache.rb:28:in
`call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb:361:in
`call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/string_coercion.rb:25:in
`call'
/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/head.rb:9:in `call'
/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/methodoverride.rb:24:in
`call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/params_parser.rb:15:in
`call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/session/cookie_store.rb:93:in
`call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/activesupport/lib/active_support/cache/strategy/local_cache.rb:24:in
`call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/failsafe.rb:26:in
`call'
/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `call'
/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in
`synchronize'
/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/actionpack/lib/action_controller/dispatcher.rb:106:in
`call'
/home/deploy/public_html/rm/releases/20100107153636/vendor/rails/railties/lib/rails/rack/static.rb:31:in
`call'
/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:46:in `call'
/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:40:in `each'
/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:40:in `call'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:571:in
`process_client'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:643:in
`worker_loop'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:641:in `each'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:641:in
`worker_loop'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:534:in
`spawn_missing_workers'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:534:in `fork'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:534:in
`spawn_missing_workers'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:530:in `each'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:530:in
`spawn_missing_workers'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:540:in
`maintain_worker_count'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:215:in `start'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/lib/unicorn.rb:28:in `run'
/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.1/bin/unicorn_rails:207
/usr/local/bin/unicorn_rails:19:in `load'
/usr/local/bin/unicorn_rails:19
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 1%]
* Re: Strange Thread related errors
2010-01-07 16:05 1% Strange Thread related errors Michael Guterl
@ 2010-01-07 20:13 0% ` Eric Wong
2010-01-07 21:32 0% ` Michael Guterl
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2010-01-07 20:13 UTC (permalink / raw)
To: unicorn list
Michael Guterl <mguterl@gmail.com> wrote:
> Our Rails app has started raising exceptions (caught by hoptoad
> thankfully) and I can only imagine they're related to unicorn. I only
> *think* the errors are occurring on the request after we deploy, which
> upgrades the Unicorn process. I say this because the errors are
> coming from many different actions, but I haven't been able to
> reproduce.
>
> After the most recent batch of errors, I upgraded from 0.95.1 to
> 0.95.3, but we have not deployed again.
>
> ThreadError: stopping only thread note: use sleep to stop forever
>
> /usr/local/lib/ruby/1.8/monitor.rb:285:in `stop'
>
> /usr/local/lib/ruby/1.8/monitor.rb:285:in `mon_acquire'
>
> /usr/local/lib/ruby/1.8/monitor.rb:214:in `mon_enter'
>
> /usr/local/lib/ruby/1.8/monitor.rb:240:in `synchronize'
>
> /usr/local/lib/ruby/1.8/logger.rb:496:in `write'
>
> /usr/local/lib/ruby/1.8/logger.rb:326:in `add'
>
> /usr/local/lib/ruby/1.8/logger.rb:374:in `info'
>
> /home/deploy/public_html/rm/releases/20100107153636/vendor/plugins/newrelic_rpm/lib/new_relic/agent/agent.rb:46:in
> `ensure_worker_thread_started'
Hi Michael,
It looks like there's a background thread with the NewRelic plugin...
With "preload_app true", then any threads spawned in the master will die
in workers. This is true for Ruby 1.8 green threads by design. With
1.9, it's not even to share native POSIX threads between processes.
So in your after_fork hook, you need to restart threads inside
each worker.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Using a worker for a different purpose
@ 2010-01-07 20:44 3% ` Iñaki Baz Castillo
2010-01-07 20:53 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Iñaki Baz Castillo @ 2010-01-07 20:44 UTC (permalink / raw)
To: mongrel-unicorn
El Jueves, 7 de Enero de 2010, Eric Wong escribió:
> Iñaki Baz Castillo <ibc@aliax.net> wrote:
> > Hi, I'm thinking in using a worker (i.e: "worker.nr == 0") to accomplish
> > a diferent task than binding in the Unicorn socket.
> >
> > It would behave as a different process which binds in a different socket
> > as daemon, so the other workers would notify it after processing data.
> >
> > Of course I could have a separate process but why not using an Unicorn
> > worker for this? in this way it's automatically reaped by master process
> > if it crashes and I don't need to manage two different services.
> >
> > Is is suitable? The main question is: how to tell a worker not to bind in
> > the Unicorn configured socket(s)? is it possible?
>
> Hi Iñaki,
>
> You could _try_ something like:
>
> after_fork do |server, worker|
> if worker.nr == 0
> # new app
> server.app = Rack::Builder.new { ... }
>
> # clear the local listener set
> server.listeners = []
>
> # new listeners
> server.listen another_socket, socket_options
> end
> end
>
> I make no guarantees that it'll work, though, and I'm hesitant
> to support/encourage it even if it does.
It seems interesting. Just a doubt: would it work with "preload_app true"?
I use preload_app since in case he config.ru is wrong then it raises in the
master (instead of raising each worker and being reaped again and again).
Thanks, I'll try it.
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 3%]
* Re: Using a worker for a different purpose
2010-01-07 20:44 3% ` Iñaki Baz Castillo
@ 2010-01-07 20:53 0% ` Eric Wong
2010-01-07 20:58 0% ` Iñaki Baz Castillo
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2010-01-07 20:53 UTC (permalink / raw)
To: unicorn list
Iñaki Baz Castillo <ibc@aliax.net> wrote:
> El Jueves, 7 de Enero de 2010, Eric Wong escribió:
> > Iñaki Baz Castillo <ibc@aliax.net> wrote:
> > > Hi, I'm thinking in using a worker (i.e: "worker.nr == 0") to accomplish
> > > a diferent task than binding in the Unicorn socket.
> > >
> > > It would behave as a different process which binds in a different socket
> > > as daemon, so the other workers would notify it after processing data.
> > >
> > > Of course I could have a separate process but why not using an Unicorn
> > > worker for this? in this way it's automatically reaped by master process
> > > if it crashes and I don't need to manage two different services.
> > >
> > > Is is suitable? The main question is: how to tell a worker not to bind in
> > > the Unicorn configured socket(s)? is it possible?
> >
> > Hi Iñaki,
> >
> > You could _try_ something like:
> >
> > after_fork do |server, worker|
> > if worker.nr == 0
> > # new app
> > server.app = Rack::Builder.new { ... }
> >
> > # clear the local listener set
> > server.listeners = []
> >
> > # new listeners
> > server.listen another_socket, socket_options
> > end
> > end
> >
> > I make no guarantees that it'll work, though, and I'm hesitant
> > to support/encourage it even if it does.
>
> It seems interesting. Just a doubt: would it work with "preload_app true"?
> I use preload_app since in case he config.ru is wrong then it raises in the
> master (instead of raising each worker and being reaped again and again).
It should work, but you won't be saving memory/spawning time with
"preload_app true" since you're replacing the app with a new one.
It's really dirty and smelly. Processes are a great OS-level
abstraction to separate distinct services from each other. IMHO it's
saner to just run another Unicorn instance.
Unicorn is not a VM, after all :)
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Using a worker for a different purpose
2010-01-07 20:53 0% ` Eric Wong
@ 2010-01-07 20:58 0% ` Iñaki Baz Castillo
0 siblings, 0 replies; 200+ results
From: Iñaki Baz Castillo @ 2010-01-07 20:58 UTC (permalink / raw)
To: mongrel-unicorn
El Jueves, 7 de Enero de 2010, Eric Wong escribió:
> > It seems interesting. Just a doubt: would it work with "preload_app
> > true"? I use preload_app since in case he config.ru is wrong then it
> > raises in the master (instead of raising each worker and being reaped
> > again and again).
>
> It should work, but you won't be saving memory/spawning time with
> "preload_app true" since you're replacing the app with a new one.
>
> It's really dirty and smelly. Processes are a great OS-level
> abstraction to separate distinct services from each other. IMHO it's
> saner to just run another Unicorn instance.
>
> Unicorn is not a VM, after all :)
Yes, you are right. In fact the purpose of such worker would not be a Rack
handler (http server) but a different one.
Humm, I think I'll keep it simple and run an external process :)
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Strange Thread related errors
2010-01-07 20:13 0% ` Eric Wong
@ 2010-01-07 21:32 0% ` Michael Guterl
0 siblings, 0 replies; 200+ results
From: Michael Guterl @ 2010-01-07 21:32 UTC (permalink / raw)
To: unicorn list
On Thu, Jan 7, 2010 at 3:13 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Michael Guterl <mguterl@gmail.com> wrote:
>> Our Rails app has started raising exceptions (caught by hoptoad
>> thankfully) and I can only imagine they're related to unicorn. I only
>> *think* the errors are occurring on the request after we deploy, which
>> upgrades the Unicorn process. I say this because the errors are
>> coming from many different actions, but I haven't been able to
>> reproduce.
>>
>> After the most recent batch of errors, I upgraded from 0.95.1 to
>> 0.95.3, but we have not deployed again.
>>
>> ThreadError: stopping only thread note: use sleep to stop forever
>>
>> /usr/local/lib/ruby/1.8/monitor.rb:285:in `stop'
>>
>> /usr/local/lib/ruby/1.8/monitor.rb:285:in `mon_acquire'
>>
>> /usr/local/lib/ruby/1.8/monitor.rb:214:in `mon_enter'
>>
>> /usr/local/lib/ruby/1.8/monitor.rb:240:in `synchronize'
>>
>> /usr/local/lib/ruby/1.8/logger.rb:496:in `write'
>>
>> /usr/local/lib/ruby/1.8/logger.rb:326:in `add'
>>
>> /usr/local/lib/ruby/1.8/logger.rb:374:in `info'
>>
>> /home/deploy/public_html/rm/releases/20100107153636/vendor/plugins/newrelic_rpm/lib/new_relic/agent/agent.rb:46:in
>> `ensure_worker_thread_started'
>
> Hi Michael,
>
> It looks like there's a background thread with the NewRelic plugin...
>
> With "preload_app true", then any threads spawned in the master will die
> in workers. This is true for Ruby 1.8 green threads by design. With
> 1.9, it's not even to share native POSIX threads between processes.
>
> So in your after_fork hook, you need to restart threads inside
> each worker.
Thanks for this Eric, much appreciated. It turns out that New Relic
doesn't officially support Unicorn yet, but I upgraded their plugin
and I have deployed a few times and haven't seen anything since.
Michael Guterl
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: X-Forwarded-Proto / X_FORWARDED_PROTO
@ 2010-01-10 11:43 2% ` John-Paul Bader
0 siblings, 0 replies; 200+ results
From: John-Paul Bader @ 2010-01-10 11:43 UTC (permalink / raw)
To: unicorn list
Hey Guys,
I posted my setup before but for this thread I'll do it again.
For http I have: varnish(80) -> nginx(8080) -> unicorn
And for https: nginx(443) -> varnish(80) -> nginx(8080) -> unicorn
Nginx has these proxy settings for the 443 vhost:
location / {
…
access_log off;
proxy_redirect off;
proxy_pass http://www.ccc.de;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-FORWARDED-PROTO https;
}
…
So I explicitly set this header. This way varnish(80), nginx(8080), unicorn and ultimately rails will get this Header and know that this was a https request. This works for me perfectly and maybe that is helpful for you ? If not never mind ;)
After all this header is a non standard extension introduced by squid. Theres a nice wikipedia page about it: http://en.wikipedia.org/wiki/X-Forwarded-For
Kind regards, John
On 10.01.2010, at 07:32, Eric Wong wrote:
> Eric Wong <normalperson@yhbt.net> wrote:
>> Iñaki Baz Castillo <ibc@aliax.net> wrote:
>>> In your case it seems valid for me (just an opinnion) as
>>> "HTTP_X_FORWARDED_PROTO: http,https" could mean that the request has been sent
>>> using HTTPS and an intermediary proxy has forwarded it using HTTP. Of course
>>> the final destination (Unicorn application) must be ready to support such
>>> syntax.
>>
>> Is it safe to say that if there's an "https" *anywhere* in the
>> X-Forwarded-Proto chain, that "rack.url_scheme" should be set to
>> "https"? Because I suppose most of the time there's only one
>> (client-facing) proxy using https.
>
> Maybe this will work...
>
> diff --git a/ext/unicorn_http/global_variables.h b/ext/unicorn_http/global_variables.h
> index e593cf6..6705851 100644
> --- a/ext/unicorn_http/global_variables.h
> +++ b/ext/unicorn_http/global_variables.h
> @@ -74,7 +74,6 @@ void init_globals(void)
> DEF_GLOBAL(server_name, "SERVER_NAME");
> DEF_GLOBAL(server_port, "SERVER_PORT");
> DEF_GLOBAL(server_protocol, "SERVER_PROTOCOL");
> - DEF_GLOBAL(http_x_forwarded_proto, "HTTP_X_FORWARDED_PROTO");
> DEF_GLOBAL(port_80, "80");
> DEF_GLOBAL(port_443, "443");
> DEF_GLOBAL(localhost, "localhost");
> diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
> index 6232e2c..f3945b2 100644
> --- a/ext/unicorn_http/unicorn_http.rl
> +++ b/ext/unicorn_http/unicorn_http.rl
> @@ -197,6 +197,14 @@ static void write_value(VALUE req, struct http_parser *hp,
> assert_frozen(f);
> }
>
> + /*
> + * any X-Forwarded-Proto: https means there's an https server in the
> + * proxy chain, and that server is most likely the one that actually
> + * sees the client, so help Rack apps generate URLs with "https"
> + */
> + if (f == g_http_x_forwarded_proto && STR_CSTR_EQ(v, "https"))
> + rb_hash_aset(req, g_rack_url_scheme, v);
> +
> e = rb_hash_aref(req, f);
> if (NIL_P(e)) {
> hp->cont = rb_hash_aset(req, f, v);
> @@ -393,12 +401,7 @@ static void finalize_header(struct http_parser *hp, VALUE req)
>
> /* set rack.url_scheme to "https" or "http", no others are allowed by Rack */
> if (NIL_P(temp)) {
> - temp = rb_hash_aref(req, g_http_x_forwarded_proto);
> - if (!NIL_P(temp) && STR_CSTR_EQ(temp, "https"))
> - server_port = g_port_443;
> - else
> - temp = g_http;
> - rb_hash_aset(req, g_rack_url_scheme, temp);
> + rb_hash_aset(req, g_rack_url_scheme, g_http);
> } else if (STR_CSTR_EQ(temp, "https")) {
> server_port = g_port_443;
> } else {
> @@ -712,5 +715,6 @@ void Init_unicorn_http(void)
> SET_GLOBAL(g_http_transfer_encoding, "TRANSFER_ENCODING");
> SET_GLOBAL(g_content_length, "CONTENT_LENGTH");
> SET_GLOBAL(g_http_connection, "CONNECTION");
> + SET_GLOBAL(g_http_x_forwarded_proto, "X_FORWARDED_PROTO");
> }
> #undef SET_GLOBAL
> --
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Unicorn 0.96 doesn't play nice with Rails 2.3.5
@ 2010-01-11 22:27 1% Peter Kieltyka
2010-01-12 3:42 0% ` Eric Wong
2010-01-19 21:50 0% ` Iñaki Baz Castillo
0 siblings, 2 replies; 200+ results
From: Peter Kieltyka @ 2010-01-11 22:27 UTC (permalink / raw)
To: mongrel-unicorn
Hello,
I've experienced a bug while trying to setup Unicorn 0.96.0 with my Rails 2.3.5 app. Some details on my system: I'm running Freebsd 8.0-p2 (amd64) with Ruby 1.9.1 and Rubygems 1.3.5.
Pretty much, I tried running "unicorn_rails" in the root of my rails application, but it never loads. What happens is the master process load then it keeps trying to start the worker app but fails. I kept getting the message that I need to install Rails 2.3.5. At fist I thought I had configured something incorrectly and there was a bad reference to my gems, but no. The error message came from my Rails config/boot.rb file which is called after the gem list has been refreshed (via app.call in Unicorn). I traced through unicorn until I found that in the load_rails_gem method of the GemBoot class was raising the load error. When checking the exception message variable it told me:
$ unicorn_rails
I, [2010-01-11T16:20:11.330566 #23391] INFO -- : listening on addr=0.0.0.0:8080 fd=3
I, [2010-01-11T16:20:11.337793 #23391] INFO -- : worker=0 spawning...
I, [2010-01-11T16:20:11.339090 #23391] INFO -- : master process ready
I, [2010-01-11T16:20:11.339225 #23393] INFO -- : worker=0 spawned pid=23393
I, [2010-01-11T16:20:11.379570 #23393] INFO -- : Refreshing Gem list
Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
I, [2010-01-11T16:20:11.446208 #23391] INFO -- : reaped #<Process::Status: pid 23393 exit 1> worker=0
I, [2010-01-11T16:20:11.446461 #23391] INFO -- : worker=0 spawning...
I, [2010-01-11T16:20:11.447539 #23394] INFO -- : worker=0 spawned pid=23394
I, [2010-01-11T16:20:11.488722 #23394] INFO -- : Refreshing Gem list
Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
... etc. ... keeps going on and on and on.
The exception message in GemBoot tells me:
can't activate rack (~> 1.0.0, runtime) for ["actionpack-2.3.5", "rails-2.3.5"], already activated rack-1.1.0 for ["unicorn-0.96.0"]
So, I uninstalled rack 1.1.0, leaving behind just rack 1.0.1. Started up the unicorn_rails again and voila. The issue seemed to be a conflict with the rack 1.1 already being loaded by Unicorn, and when Rails begins to load and checks its dependencies, the rack version is too new for 2.3.5.
I'm not sure what the fix is here, but uninstalling rack 1.1 probably is not the answer.
Cheers,
Peter
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 1%]
* Re: Unicorn 0.96 doesn't play nice with Rails 2.3.5
2010-01-11 22:27 1% Unicorn 0.96 doesn't play nice with Rails 2.3.5 Peter Kieltyka
@ 2010-01-12 3:42 0% ` Eric Wong
2010-01-12 14:36 0% ` Peter Kieltyka
2010-01-19 21:50 0% ` Iñaki Baz Castillo
1 sibling, 1 reply; 200+ results
From: Eric Wong @ 2010-01-12 3:42 UTC (permalink / raw)
To: Peter Kieltyka; +Cc: unicorn list
Peter Kieltyka <peter.kieltyka@gmail.com> wrote:
> Hello,
>
> I've experienced a bug while trying to setup Unicorn 0.96.0 with my
> Rails 2.3.5 app. Some details on my system: I'm running Freebsd 8.0-p2
> (amd64) with Ruby 1.9.1 and Rubygems 1.3.5.
>
> Pretty much, I tried running "unicorn_rails" in the root of my rails
> application, but it never loads. What happens is the master process
> load then it keeps trying to start the worker app but fails. I kept
> getting the message that I need to install Rails 2.3.5. At fist I
> thought I had configured something incorrectly and there was a bad
> reference to my gems, but no. The error message came from my Rails
> config/boot.rb file which is called after the gem list has been
> refreshed (via app.call in Unicorn). I traced through unicorn until I
> found that in the load_rails_gem method of the GemBoot class was
> raising the load error. When checking the exception message variable
> it told me:
>
> $ unicorn_rails
> I, [2010-01-11T16:20:11.330566 #23391] INFO -- : listening on addr=0.0.0.0:8080 fd=3
> I, [2010-01-11T16:20:11.337793 #23391] INFO -- : worker=0 spawning...
> I, [2010-01-11T16:20:11.339090 #23391] INFO -- : master process ready
> I, [2010-01-11T16:20:11.339225 #23393] INFO -- : worker=0 spawned pid=23393
> I, [2010-01-11T16:20:11.379570 #23393] INFO -- : Refreshing Gem list
> Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
> I, [2010-01-11T16:20:11.446208 #23391] INFO -- : reaped #<Process::Status: pid 23393 exit 1> worker=0
> I, [2010-01-11T16:20:11.446461 #23391] INFO -- : worker=0 spawning...
> I, [2010-01-11T16:20:11.447539 #23394] INFO -- : worker=0 spawned pid=23394
> I, [2010-01-11T16:20:11.488722 #23394] INFO -- : Refreshing Gem list
> Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
> ... etc. ... keeps going on and on and on.
>
> The exception message in GemBoot tells me:
>
> can't activate rack (~> 1.0.0, runtime) for ["actionpack-2.3.5",
> "rails-2.3.5"], already activated rack-1.1.0 for ["unicorn-0.96.0"]
>
> So, I uninstalled rack 1.1.0, leaving behind just rack 1.0.1. Started
> up the unicorn_rails again and voila. The issue seemed to be a
> conflict with the rack 1.1 already being loaded by Unicorn, and when
> Rails begins to load and checks its dependencies, the rack version is
> too new for 2.3.5.
>
> I'm not sure what the fix is here, but uninstalling rack 1.1 probably
> is not the answer.
Hi Peter,
Uninstalling Rack 1.1 may be the best answer, especially
if you don't need Rack for anything else.
There are several problems at hand:
1. Rails 2.3.5 declares it is only compatible with Rack 1.0.x
2. Unicorn does not require any particular Rack version
3. RubyGems defaults to the latest version of any Gem if
no version is explicitly specified.
Since Unicorn is loaded before Rails, RubyGems will default to
loading the latest version of Rack based on the Unicorn gemspec
You have several other options here to work around this,
they all suck (in no particular order of suckiness):
1. edit the installed Rails/active* gemspecs[1] so they're not
pinned on on 1.0.x anymore (they *should* be compatible enough)
1a. vendorize Rails/active* and edit the gemspecs there
2. edit your copy of /path/to/unicorn_rails and
activate Rack 1.0.x in there before Unicorn is activated
(this is the "unicorn_rails" wrapper RubyGems creates for you,
not the actual "unicorn_rails" source).
3. create a "vendored" copy of unicorn_rails for your
application (script/my_unicorn) and activate Rack 1.0.x
in there first.
4. install Unicorn using setup.rb without RubyGems
(uninstalls/upgrades won't be as nice to deal with, though).
5. edit the installed Unicorn gemspec[1] to depend on Rack ~> 1.0
[1] - RubyGems gemspecs are here:
$prefix/lib/ruby/gems/$RUBY_VERSION/specifications/*.gemspec
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Unicorn 0.96 doesn't play nice with Rails 2.3.5
2010-01-12 3:42 0% ` Eric Wong
@ 2010-01-12 14:36 0% ` Peter Kieltyka
0 siblings, 0 replies; 200+ results
From: Peter Kieltyka @ 2010-01-12 14:36 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn list
On 2010-01-11, at 10:42 PM, Eric Wong wrote:
> Peter Kieltyka <peter.kieltyka@gmail.com> wrote:
>> Hello,
>>
>> I've experienced a bug while trying to setup Unicorn 0.96.0 with my
>> Rails 2.3.5 app. Some details on my system: I'm running Freebsd 8.0-p2
>> (amd64) with Ruby 1.9.1 and Rubygems 1.3.5.
>>
>> Pretty much, I tried running "unicorn_rails" in the root of my rails
>> application, but it never loads. What happens is the master process
>> load then it keeps trying to start the worker app but fails. I kept
>> getting the message that I need to install Rails 2.3.5. At fist I
>> thought I had configured something incorrectly and there was a bad
>> reference to my gems, but no. The error message came from my Rails
>> config/boot.rb file which is called after the gem list has been
>> refreshed (via app.call in Unicorn). I traced through unicorn until I
>> found that in the load_rails_gem method of the GemBoot class was
>> raising the load error. When checking the exception message variable
>> it told me:
>>
>> $ unicorn_rails
>> I, [2010-01-11T16:20:11.330566 #23391] INFO -- : listening on addr=0.0.0.0:8080 fd=3
>> I, [2010-01-11T16:20:11.337793 #23391] INFO -- : worker=0 spawning...
>> I, [2010-01-11T16:20:11.339090 #23391] INFO -- : master process ready
>> I, [2010-01-11T16:20:11.339225 #23393] INFO -- : worker=0 spawned pid=23393
>> I, [2010-01-11T16:20:11.379570 #23393] INFO -- : Refreshing Gem list
>> Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
>> I, [2010-01-11T16:20:11.446208 #23391] INFO -- : reaped #<Process::Status: pid 23393 exit 1> worker=0
>> I, [2010-01-11T16:20:11.446461 #23391] INFO -- : worker=0 spawning...
>> I, [2010-01-11T16:20:11.447539 #23394] INFO -- : worker=0 spawned pid=23394
>> I, [2010-01-11T16:20:11.488722 #23394] INFO -- : Refreshing Gem list
>> Missing the Rails 2.3.5 gem. Please `gem install -v=2.3.5 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.
>> ... etc. ... keeps going on and on and on.
>>
>> The exception message in GemBoot tells me:
>>
>> can't activate rack (~> 1.0.0, runtime) for ["actionpack-2.3.5",
>> "rails-2.3.5"], already activated rack-1.1.0 for ["unicorn-0.96.0"]
>>
>> So, I uninstalled rack 1.1.0, leaving behind just rack 1.0.1. Started
>> up the unicorn_rails again and voila. The issue seemed to be a
>> conflict with the rack 1.1 already being loaded by Unicorn, and when
>> Rails begins to load and checks its dependencies, the rack version is
>> too new for 2.3.5.
>>
>> I'm not sure what the fix is here, but uninstalling rack 1.1 probably
>> is not the answer.
>
> Hi Peter,
>
> Uninstalling Rack 1.1 may be the best answer, especially
> if you don't need Rack for anything else.
>
> There are several problems at hand:
>
> 1. Rails 2.3.5 declares it is only compatible with Rack 1.0.x
> 2. Unicorn does not require any particular Rack version
> 3. RubyGems defaults to the latest version of any Gem if
> no version is explicitly specified.
>
> Since Unicorn is loaded before Rails, RubyGems will default to
> loading the latest version of Rack based on the Unicorn gemspec
>
> You have several other options here to work around this,
> they all suck (in no particular order of suckiness):
>
> 1. edit the installed Rails/active* gemspecs[1] so they're not
> pinned on on 1.0.x anymore (they *should* be compatible enough)
>
> 1a. vendorize Rails/active* and edit the gemspecs there
>
> 2. edit your copy of /path/to/unicorn_rails and
> activate Rack 1.0.x in there before Unicorn is activated
> (this is the "unicorn_rails" wrapper RubyGems creates for you,
> not the actual "unicorn_rails" source).
>
> 3. create a "vendored" copy of unicorn_rails for your
> application (script/my_unicorn) and activate Rack 1.0.x
> in there first.
>
> 4. install Unicorn using setup.rb without RubyGems
> (uninstalls/upgrades won't be as nice to deal with, though).
>
> 5. edit the installed Unicorn gemspec[1] to depend on Rack ~> 1.0
>
>
> [1] - RubyGems gemspecs are here:
> $prefix/lib/ruby/gems/$RUBY_VERSION/specifications/*.gemspec
>
> --
> Eric Wong
Hey Eric,
Thanks for the quick reply.
That is pretty much what I expected. I plan to begin experimenting with Rails 3 for this project in any regards but I thought I'd bring it to your attention if others have the same issue (which some already did acknowledge it on #rubyonrails@irc.freenode.net).
Strange though, on my OSX 10.6 system with the same setup (Ruby 1.9.1, Gems 1.3.5, Rack 1.1.0 & 1.0.1, and Unicorn 0.96.0) the issue doesn't come up. Ohh well..
Peter
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Unicorn 0.96 doesn't play nice with Rails 2.3.5
2010-01-11 22:27 1% Unicorn 0.96 doesn't play nice with Rails 2.3.5 Peter Kieltyka
2010-01-12 3:42 0% ` Eric Wong
@ 2010-01-19 21:50 0% ` Iñaki Baz Castillo
1 sibling, 0 replies; 200+ results
From: Iñaki Baz Castillo @ 2010-01-19 21:50 UTC (permalink / raw)
To: mongrel-unicorn
El Lunes, 11 de Enero de 2010, Peter Kieltyka escribió:
> The exception message in GemBoot tells me:
> can't activate rack (~> 1.0.0, runtime) for ["actionpack-2.3.5",
> "rails-2.3.5"], already activated rack-1.1.0 for ["unicorn-0.96.0"]
>
> So, I uninstalled rack 1.1.0, leaving behind just rack 1.0.1. Started up
> the unicorn_rails again and voila. The issue seemed to be a conflict with
> the rack 1.1 already being loaded by Unicorn, and when Rails begins to
> load and checks its dependencies, the rack version is too new for 2.3.5.
>
> I'm not sure what the fix is here, but uninstalling rack 1.1 probably is
> not the answer.
I think this bug is related to a recent thread:
http://www.mail-archive.com/mongrel-unicorn@rubyforge.org/msg00329.html
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Suggestion for reload action (USR2)
@ 2010-01-20 8:54 2% Iñaki Baz Castillo
0 siblings, 0 replies; 200+ results
From: Iñaki Baz Castillo @ 2010-01-20 8:54 UTC (permalink / raw)
To: mongrel-unicorn
Hi, when Unicorn receives a USR2 it invokes the original script and arguments
and creates a new master as a child.
However when running this new master there could be something to treat
different as when we run Unicorn for first time.
For example:
- In the initial unicorn master I want it to create a Posix_mq, and in case it
already exists then I want it to delete and create again.
- But in the new master (when USR2) I just want it to reuse the existing
Posiq_mq.
Note this is just an example, perhaps not the best however.
So I suggest the following:
- When unicorn master receives a USR2 it executes the original executable
withthe original arguments plus "--reload". This just would mean that the new
instance is a reloaded process. The executable could store such option
somewhere so the application could know it and react.
This would involve just three changes:
- Add an option "--reload" in OptionParser in bin/unicorn.
- Store such option somewhere so it can be readed later by the Rack
application or other library.
- Add "--reload" to the list of original arguments when receiving USR2.
Opinnions?
--
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: "upstream timed out" after upgrades
@ 2010-02-04 8:48 1% ` John-Paul Bader
0 siblings, 1 reply; 200+ results
From: John-Paul Bader @ 2010-02-04 8:48 UTC (permalink / raw)
To: unicorn list
Hrm its getting weirder and weirder.
I started unicorn_rails now manually in the app root which is now running on rails 2.3.3 again:
sudo unicorn_rails -E production -d -c /usr/local/etc/unicorn.rb
Now first of all it throws lots of Exceptions it couldn't find several (important) gems and then it produces this every second or so:
Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)
When I request the javascript in question I get:
Exception `Errno::EINVAL' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn/http_response.rb:67 - Invalid argument
Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)
This is the same no matter if I run it with -c or not.
I am puzzled.
To give you the entire output of unicorn starting:
[hukl@cccms /usr/local/www/cccms]$ sudo unicorn_rails -E production -d -p 9090
{:daemonize=>false,
:unicorn_options=>{:listeners=>["0.0.0.0:9090"]},
:app=>
#<Proc:0x0000000801782388@/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/bin/unicorn_rails:124>}
Exception `Errno::EEXIST' at /usr/local/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/cache
Exception `Errno::EEXIST' at /usr/local/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/pids
Exception `Errno::EEXIST' at /usr/local/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/sessions
Exception `Errno::EEXIST' at /usr/local/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/sockets
I, [2010-02-04T09:42:08.141034 #38499] INFO -- : listening on addr=213.73.89.122:9090 fd=3
Exception `LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- Win32API
Exception `LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:38 - no such file to load -- Win32API
I, [2010-02-04T09:42:08.158275 #38499] INFO -- : worker=0 spawning...
I, [2010-02-04T09:42:08.159352 #38499] INFO -- : master process ready
I, [2010-02-04T09:42:08.159471 #38501] INFO -- : worker=0 spawned pid=38501
I, [2010-02-04T09:42:08.197082 #38501] INFO -- : Refreshing Gem list
Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem builder (~> 2.1.2)
Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem memcache-client (>= 1.7.4)
Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem tzinfo (~> 0.3.12)
Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem i18n (~> 0.1.3)
Exception `TypeError' at (eval):4 - can't modify frozen object
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- fast_xs
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- json
Using c extension for JSON.
Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem tmail (~> 1.2.3)
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- tmail/tmailscanner
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:38 - no such file to load -- tmail/tmailscanner
Exception `MissingSourceFile' at /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:162 - no such file to load -- tmail/tmailscanner
Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem activerecord-postgresql-adapter (>= 0)
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- pg
WARNING: Nokogiri was built against LibXML version 2.7.3, but has dynamically loaded 2.7.6
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- ruby-debug
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- classtree
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:38 - no such file to load -- classtree
Exception `MissingSourceFile' at /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:162 - no such file to load -- classtree
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- methodsig
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:38 - no such file to load -- methodsig
Exception `MissingSourceFile' at /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:162 - no such file to load -- methodsig
=> Debugger enabled
worker=0 ready
Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)
Exception `Errno::EINVAL' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn/http_response.rb:67 - Invalid argument
Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)
Exception `Errno::EINVAL' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn/http_response.rb:67 - Invalid argument
Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)
Exception `Errno::EINVAL' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn/http_response.rb:67 - Invalid argument
Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)
I seriously have no idea went wrong - It ran all so smooth couple of days ago. What am I missing - there must be something plain wrong. I left the js files out of the public site for now - it isn't needed that much and its more important to serve the content for now. In case you are trying to get it.
Kind regards, John
On 04.02.2010, at 09:16, John-Paul Bader wrote:
> Hey guys,
>
>
> i recently updated several components of my website setup:
>
> varnish 2.0.5 -> 2.0.6
> nginx 7.4.2 -> 7.4.5
> unicorn 0.95.3 -> 0.96
> ruby18 to the latest patch level
> and rails from 2.3.2
>
> It all seemed to work great but recently I noticed that the website hangs loading. The weird thing is that it hangs because 2 or 3 javascript files take about a minute to finish and when they do they are incomplete as in cut of in the middle of the content. The nginx error log gives the following output every few seconds:
>
> 2010/02/04 08:58:41 [error] 29389#0: *471 upstream timed out (60: Operation timed out) while reading upstream, client: 213.73.89.122, server: www.ccc.de, request: "GET /javascripts/jquery-1.3.2.min.js?1246657683 HTTP/1.1", upstream: "http://unix:/var/run/unicorn.sock:/javascripts/jquery-1.3.2.min.js?1246657683", host: "www.ccc.de"
>
> I don't think its varnishs fault because i get Internal Server Errors (500) if I run curl on the machine against nginx, not at first though - several requests for that jquery file run just fine and then suddenly i get a 500 on a request with the content being cut of in the middle.
>
> Next thing I did was configuring unicorn to listen on 213.73.89.122:9090 rather than on a socket so i could run curl directly against it and even after a fresh start of unicorn gives me:
>
>
> curl http://213.73.89.122:9090/javascripts/jquery-1.3.2.min.js
> (lots of javascript)…,rowspan:"rowSpan",tabindex:"tabIndHTTP/1.1 500 Internal Server Error
>
> curl: (18) transfer closed with 16256 bytes remaining to read
>
> I also downgraded back to 0.95.3 but that didn't change anything. So now after 8h of try and error I'm out of ideas and would be happy to hear some suggestions.
>
> Kind regards, John
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 1%]
* Re: "upstream timed out" after upgrades
@ 2010-02-04 12:26 2% ` John-Paul Bader
2010-02-04 14:22 2% ` John-Paul Bader
0 siblings, 1 reply; 200+ results
From: John-Paul Bader @ 2010-02-04 12:26 UTC (permalink / raw)
To: unicorn list
Hey again,
I will try to compile build the svn revision and let you know. I have to do some other work before that so I will report back in the evening (CEST / BERLIN)
Thank you for helping out, kind regards,
John
On 04.02.2010, at 11:11, Eric Wong wrote:
> John-Paul Bader <hukl@h3q.com> wrote:
>> One more,
>
> Hi,
>
> About the exceptions, looks like you had '-d' (debug) instead of '-D'
> (daemonize) so it spewed every exception (even if trapped) to stderr.
> EAGAIN is common when dealing with non-blocking sockets. Most of that
> is noise...
>
> However the EINVAL in unicorn/http_response.rb is suspect.
>
>> I reproduced the upgrading on my staging server which was still at the
>> old state in terms of software. I started by upgrading ruby from
>>
>> ruby+nopthreads-1.8.7.160_5,1 -> ruby+nopthreads-1.8.7.248,1
>> ruby18-iconv-1.8.7.160,1 -> ruby18-iconv-1.8.7.248,1
>>
>> And voilá - the very same problems. So now I'm like what?
>>
>> Is this rather a ruby than a unicorn issue ? Couldn't find any clues
>> in the changelog so far:
>> http://svn.ruby-lang.org/cgi-bin/viewvc.cgi/tags/v1_8_7_248/ChangeLog
>
> That should narrow it down, so I start reading diffs from
> v1_8_7_160..v1_8_7_248 ...
>
> Does backporting the following change in ruby_1_8
> (but not yet in the ruby_1_8_7 branch) fix things for you?
>
> commit 841a57341ed43f5fa86489c12aceb25a232be820
> Author: nobu <nobu@b2dd03c8-39d4-4d8f-98ff-823fe69b080e>
> Date: Fri Jan 8 09:51:23 2010 +0000
>
> * io.c (io_fwrite): preserve errno. [ruby-core:27425]
>
>
> git-svn-id: http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8@26253
>
> (snipped)
>
> diff --git a/io.c b/io.c
> index 375cbc8..d4d28e5 100644
> --- a/io.c
> +++ b/io.c
> @@ -122,6 +122,9 @@ extern void Init_File _((void));
> # endif
> #endif
>
> +#define preserving_errno(stmts) \
> + do {int saved_errno = errno; stmts; errno = saved_errno;} while (0)
> +
> VALUE rb_cIO;
> VALUE rb_eEOFError;
> VALUE rb_eIOError;
> @@ -490,7 +493,7 @@ io_fwrite(str, fptr)
> r = write(fileno(f), RSTRING(str)->ptr+offset, l);
> TRAP_END;
> #if BSD_STDIO
> - fseeko(f, lseek(fileno(f), (off_t)0, SEEK_CUR), SEEK_SET);
> + preserving_errno(fseeko(f, lseek(fileno(f), (off_t)0, SEEK_CUR), SEEK_SET));
> #endif
> if (r == n) return len;
> if (0 <= r) {
> ---
> Of course, the original reason for this fseeko() was to fix another
> problem Unicorn exposed when mixing stdio + unistd calls...
>
> http://redmine.ruby-lang.org/issues/show/2267
>
> --
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: "upstream timed out" after upgrades
2010-02-04 12:26 2% ` John-Paul Bader
@ 2010-02-04 14:22 2% ` John-Paul Bader
2010-02-04 15:11 0% ` John-Paul Bader
0 siblings, 1 reply; 200+ results
From: John-Paul Bader @ 2010-02-04 14:22 UTC (permalink / raw)
To: unicorn list
Hey,
i checked out the revision 26253 built it and ran it and it seems to work again but: I didn't build it via FreeBSD ports so there might be a difference in configure / build options and I checked out the revision which included the change you suspect for causing the issue so I'm not sure if thats really helping us. Curling directly unicorn works without truncation and issues though.
Kind regards, John
On 04.02.2010, at 13:26, John-Paul Bader wrote:
> Hey again,
>
> I will try to compile build the svn revision and let you know. I have to do some other work before that so I will report back in the evening (CEST / BERLIN)
>
> Thank you for helping out, kind regards,
>
> John
>
> On 04.02.2010, at 11:11, Eric Wong wrote:
>
>> John-Paul Bader <hukl@h3q.com> wrote:
>>> One more,
>>
>> Hi,
>>
>> About the exceptions, looks like you had '-d' (debug) instead of '-D'
>> (daemonize) so it spewed every exception (even if trapped) to stderr.
>> EAGAIN is common when dealing with non-blocking sockets. Most of that
>> is noise...
>>
>> However the EINVAL in unicorn/http_response.rb is suspect.
>>
>>> I reproduced the upgrading on my staging server which was still at the
>>> old state in terms of software. I started by upgrading ruby from
>>>
>>> ruby+nopthreads-1.8.7.160_5,1 -> ruby+nopthreads-1.8.7.248,1
>>> ruby18-iconv-1.8.7.160,1 -> ruby18-iconv-1.8.7.248,1
>>>
>>> And voilá - the very same problems. So now I'm like what?
>>>
>>> Is this rather a ruby than a unicorn issue ? Couldn't find any clues
>>> in the changelog so far:
>>> http://svn.ruby-lang.org/cgi-bin/viewvc.cgi/tags/v1_8_7_248/ChangeLog
>>
>> That should narrow it down, so I start reading diffs from
>> v1_8_7_160..v1_8_7_248 ...
>>
>> Does backporting the following change in ruby_1_8
>> (but not yet in the ruby_1_8_7 branch) fix things for you?
>>
>> commit 841a57341ed43f5fa86489c12aceb25a232be820
>> Author: nobu <nobu@b2dd03c8-39d4-4d8f-98ff-823fe69b080e>
>> Date: Fri Jan 8 09:51:23 2010 +0000
>>
>> * io.c (io_fwrite): preserve errno. [ruby-core:27425]
>>
>>
>> git-svn-id: http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8@26253
>>
>> (snipped)
>>
>> diff --git a/io.c b/io.c
>> index 375cbc8..d4d28e5 100644
>> --- a/io.c
>> +++ b/io.c
>> @@ -122,6 +122,9 @@ extern void Init_File _((void));
>> # endif
>> #endif
>>
>> +#define preserving_errno(stmts) \
>> + do {int saved_errno = errno; stmts; errno = saved_errno;} while (0)
>> +
>> VALUE rb_cIO;
>> VALUE rb_eEOFError;
>> VALUE rb_eIOError;
>> @@ -490,7 +493,7 @@ io_fwrite(str, fptr)
>> r = write(fileno(f), RSTRING(str)->ptr+offset, l);
>> TRAP_END;
>> #if BSD_STDIO
>> - fseeko(f, lseek(fileno(f), (off_t)0, SEEK_CUR), SEEK_SET);
>> + preserving_errno(fseeko(f, lseek(fileno(f), (off_t)0, SEEK_CUR), SEEK_SET));
>> #endif
>> if (r == n) return len;
>> if (0 <= r) {
>> ---
>> Of course, the original reason for this fseeko() was to fix another
>> problem Unicorn exposed when mixing stdio + unistd calls...
>>
>> http://redmine.ruby-lang.org/issues/show/2267
>>
>> --
>> Eric Wong
>> _______________________________________________
>> Unicorn mailing list - mongrel-unicorn@rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>> Do not quote signatures (like this one) or top post when replying
>>
>
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: "upstream timed out" after upgrades
2010-02-04 14:22 2% ` John-Paul Bader
@ 2010-02-04 15:11 0% ` John-Paul Bader
0 siblings, 1 reply; 200+ results
From: John-Paul Bader @ 2010-02-04 15:11 UTC (permalink / raw)
To: unicorn list
To counter check I just built the latest ruby from source as well and the issue reappeared - so it happened between 26253 and tag/248
Kind regards, John
On 04.02.2010, at 15:22, John-Paul Bader wrote:
> Hey,
>
> i checked out the revision 26253 built it and ran it and it seems to work again but: I didn't build it via FreeBSD ports so there might be a difference in configure / build options and I checked out the revision which included the change you suspect for causing the issue so I'm not sure if thats really helping us. Curling directly unicorn works without truncation and issues though.
>
> Kind regards, John
>
> On 04.02.2010, at 13:26, John-Paul Bader wrote:
>
>> Hey again,
>>
>> I will try to compile build the svn revision and let you know. I have to do some other work before that so I will report back in the evening (CEST / BERLIN)
>>
>> Thank you for helping out, kind regards,
>>
>> John
>>
>> On 04.02.2010, at 11:11, Eric Wong wrote:
>>
>>> John-Paul Bader <hukl@h3q.com> wrote:
>>>> One more,
>>>
>>> Hi,
>>>
>>> About the exceptions, looks like you had '-d' (debug) instead of '-D'
>>> (daemonize) so it spewed every exception (even if trapped) to stderr.
>>> EAGAIN is common when dealing with non-blocking sockets. Most of that
>>> is noise...
>>>
>>> However the EINVAL in unicorn/http_response.rb is suspect.
>>>
>>>> I reproduced the upgrading on my staging server which was still at the
>>>> old state in terms of software. I started by upgrading ruby from
>>>>
>>>> ruby+nopthreads-1.8.7.160_5,1 -> ruby+nopthreads-1.8.7.248,1
>>>> ruby18-iconv-1.8.7.160,1 -> ruby18-iconv-1.8.7.248,1
>>>>
>>>> And voilá - the very same problems. So now I'm like what?
>>>>
>>>> Is this rather a ruby than a unicorn issue ? Couldn't find any clues
>>>> in the changelog so far:
>>>> http://svn.ruby-lang.org/cgi-bin/viewvc.cgi/tags/v1_8_7_248/ChangeLog
>>>
>>> That should narrow it down, so I start reading diffs from
>>> v1_8_7_160..v1_8_7_248 ...
>>>
>>> Does backporting the following change in ruby_1_8
>>> (but not yet in the ruby_1_8_7 branch) fix things for you?
>>>
>>> commit 841a57341ed43f5fa86489c12aceb25a232be820
>>> Author: nobu <nobu@b2dd03c8-39d4-4d8f-98ff-823fe69b080e>
>>> Date: Fri Jan 8 09:51:23 2010 +0000
>>>
>>> * io.c (io_fwrite): preserve errno. [ruby-core:27425]
>>>
>>>
>>> git-svn-id: http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8@26253
>>>
>>> (snipped)
>>>
>>> diff --git a/io.c b/io.c
>>> index 375cbc8..d4d28e5 100644
>>> --- a/io.c
>>> +++ b/io.c
>>> @@ -122,6 +122,9 @@ extern void Init_File _((void));
>>> # endif
>>> #endif
>>>
>>> +#define preserving_errno(stmts) \
>>> + do {int saved_errno = errno; stmts; errno = saved_errno;} while (0)
>>> +
>>> VALUE rb_cIO;
>>> VALUE rb_eEOFError;
>>> VALUE rb_eIOError;
>>> @@ -490,7 +493,7 @@ io_fwrite(str, fptr)
>>> r = write(fileno(f), RSTRING(str)->ptr+offset, l);
>>> TRAP_END;
>>> #if BSD_STDIO
>>> - fseeko(f, lseek(fileno(f), (off_t)0, SEEK_CUR), SEEK_SET);
>>> + preserving_errno(fseeko(f, lseek(fileno(f), (off_t)0, SEEK_CUR), SEEK_SET));
>>> #endif
>>> if (r == n) return len;
>>> if (0 <= r) {
>>> ---
>>> Of course, the original reason for this fseeko() was to fix another
>>> problem Unicorn exposed when mixing stdio + unistd calls...
>>>
>>> http://redmine.ruby-lang.org/issues/show/2267
>>>
>>> --
>>> Eric Wong
>>> _______________________________________________
>>> Unicorn mailing list - mongrel-unicorn@rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>>> Do not quote signatures (like this one) or top post when replying
>>>
>>
>> _______________________________________________
>> Unicorn mailing list - mongrel-unicorn@rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>> Do not quote signatures (like this one) or top post when replying
>>
>
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: "upstream timed out" after upgrades
@ 2010-02-04 21:52 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-02-04 21:52 UTC (permalink / raw)
To: unicorn list
John-Paul Bader <hukl@h3q.com> wrote:
> Hmm,
>
> my C skills are really weak but this patch suppresses any IO errors
> right? At least on BSD? Can you explain what exactly it is doing? Just
> want to understand entirely.
>From the beginning (a bit long, feel free to ask me for more
clarification, too much of this is second nature to me by now
and I may glance over important details...):
There are two standard ways of doing IO in C: stdio.h (fwrite(3)/
fread(3)/fseek(2) ...) and the lower-level Unix system calls found in
unistd.h (write(2)/read(2)/lseek(2)...)
stdio.h functions are wrappers that do buffering in userspace and wrap
the underlying unistd.h syscalls. They should not be used
interchangeably on the same file descriptors.
Unfortunately, Ruby 1.8 makes this mistake and uses them interchangeably
in some places: bad.
So when working with regular files, file offsets maintained in userspace
via stdio did not get properly synchronized to the underlying
kernel-level file offsets. That's why the fseeko(lseek()) was added to
fix an issue exposed by Unicorn. lseek() was used to read the offset
from the kernel, and then fseeko() is then used to synchronize the
userspace offset to that of the kernel. All of this works well for
seekable regular files.
Now, sockets and pipes aren't seekable, so you'll get an error from the
kernel if you try to seek on them. Errors from system calls are stored
in "errno", a global variable that stores the error of the last system
call executed. So since attempting to seek on an unseekable file sets
errno, it clobbers the previous clean (or the "safe" value of EAGAIN[1])
errno.
Eventually, this caused rb_sys_fail() function to be called, which
raises a Ruby exception matching the current value of errno.
[1] - EAGAIN (and EWOULDBLOCK on some systems) basically means "try
calling this same function again, later". It gets returned when kernel
buffers (not the userspace ones) are full if attempting to write, or
empty if attempting to read. Since Ruby 1.8 relies on non-blocking I/O
for sockets/pipes, the "blocking" write methods are coded to
(eventually) retry on EAGAIN.
> I thought your diff was the ruby diff of r26253 and didn't realize it
> was yours ;)
Actually, it's not mine, I just submitted the ticket that fixed one bug
and introduced the one you hit :)
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Nginx Sock And Rails Envinroment Error
@ 2010-02-15 16:19 2% ` Eric Wong
2010-02-15 22:14 0% ` Alex Barlow
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2010-02-15 16:19 UTC (permalink / raw)
To: Alex Barlow; +Cc: unicorn list
Alex Barlow <alexbarlowis@me.com> wrote:
> Hi
>
> It goes
>
> client <- TCP -> Unicorn => quiet errors (good)
> client <- TCP -> nginx <- TCP -> Unicorn => quiet errors (good)
> client <- TCP -> nginx <- Unix socket-> Unicorn => verbose errors (bad)
>
> Strange i know!
Hi Alex,
My gut feeling is that somehow nginx is hitting a different instance of
your app when using the Unix socket. Other than that, I'm confused.
More info/questions below.
> Ill try the socket in /tmp/ see what that does.
>
> The Nginx error logs show nothing error wise really. My Nginx config is...
>
> user nginx;
> worker_processes 1;
>
> events {
> worker_connections 1024;
> }
>
> http {
> include mime.types;
> default_type application/octet-stream;
>
> sendfile on;
> keepalive_timeout 65;
> gzip on;
>
> upstream unicorn_sock {
> server unix:/root/pbr/unicorn.sock;
> }
>
> server {
> listen 80;
> server_name localhost;
> proxy_set_header Host $host;
>
> location / {
> proxy_pass http://unicorn_sock;
> }
> }
> }
Nothing strange there, what's the verbosity of the nginx error_log?
Also, anything enlightening in the Rails production.log or Unicorn
stderr?
Which OS are you running? Maybe there's a platform-specific bug
somewhere, too...
> This is currently throwing out verbose errors, in production (must be,
> its using asset host and the production database)
>
> Put the sock in /tmp, no difference
Is there another Unicorn instance on the same box that it might be
somehow hitting?
What happens when you try have Unicorn listening on both TCP and a Unix
socket? Just put both "listen" directives in your config file and point
nginx to the Unix socket. Then try hitting the Unicorn TCP port
directly, and then also the Unix socket via nginx.
You can also try hitting the Unix socket directly by crafting your
own HTTP request using socat from th shell:
req='GET / HTTP/1.1\r\nHost: example.com\r\n\r\n'
printf "$req" | socat - UNIX:/root/pbr/unicorn.sock
P.S.: I might not have a chance to respond again for the next day
or so due to personal matters. Maybe somebody else on this
list can help....
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Nginx Sock And Rails Envinroment Error
2010-02-15 16:19 2% ` Eric Wong
@ 2010-02-15 22:14 0% ` Alex Barlow
0 siblings, 0 replies; 200+ results
From: Alex Barlow @ 2010-02-15 22:14 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn list
Fixed it!
Rails was considering all requests to be local. As the request was coming from the local ip (from nginx i assume)
i put these headers in nginx
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect false
Works fine now
Alex
On 15 Feb 2010, at 16:19, Eric Wong wrote:
> Alex Barlow <alexbarlowis@me.com> wrote:
>> Hi
>>
>> It goes
>>
>> client <- TCP -> Unicorn => quiet errors (good)
>> client <- TCP -> nginx <- TCP -> Unicorn => quiet errors (good)
>> client <- TCP -> nginx <- Unix socket-> Unicorn => verbose errors (bad)
>>
>> Strange i know!
>
> Hi Alex,
>
> My gut feeling is that somehow nginx is hitting a different instance of
> your app when using the Unix socket. Other than that, I'm confused.
> More info/questions below.
>
>> Ill try the socket in /tmp/ see what that does.
>>
>> The Nginx error logs show nothing error wise really. My Nginx config is...
>>
>> user nginx;
>> worker_processes 1;
>>
>> events {
>> worker_connections 1024;
>> }
>>
>> http {
>> include mime.types;
>> default_type application/octet-stream;
>>
>> sendfile on;
>> keepalive_timeout 65;
>> gzip on;
>>
>> upstream unicorn_sock {
>> server unix:/root/pbr/unicorn.sock;
>> }
>>
>> server {
>> listen 80;
>> server_name localhost;
>> proxy_set_header Host $host;
>>
>> location / {
>> proxy_pass http://unicorn_sock;
>> }
>> }
>> }
>
> Nothing strange there, what's the verbosity of the nginx error_log?
>
> Also, anything enlightening in the Rails production.log or Unicorn
> stderr?
>
> Which OS are you running? Maybe there's a platform-specific bug
> somewhere, too...
>
>> This is currently throwing out verbose errors, in production (must be,
>> its using asset host and the production database)
>>
>> Put the sock in /tmp, no difference
>
> Is there another Unicorn instance on the same box that it might be
> somehow hitting?
>
> What happens when you try have Unicorn listening on both TCP and a Unix
> socket? Just put both "listen" directives in your config file and point
> nginx to the Unix socket. Then try hitting the Unicorn TCP port
> directly, and then also the Unix socket via nginx.
>
> You can also try hitting the Unix socket directly by crafting your
> own HTTP request using socat from th shell:
>
> req='GET / HTTP/1.1\r\nHost: example.com\r\n\r\n'
> printf "$req" | socat - UNIX:/root/pbr/unicorn.sock
>
> P.S.: I might not have a chance to respond again for the next day
> or so due to personal matters. Maybe somebody else on this
> list can help....
>
> --
> Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Issue with unicorn not starting via bluepill?
@ 2010-03-24 19:14 2% ` James Cox
0 siblings, 0 replies; 200+ results
From: James Cox @ 2010-03-24 19:14 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn list, ghazel
Not yet- i've been fixing other things first. Looking at this again shortly
On Tue, Mar 23, 2010 at 2:54 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Eric Wong <normalperson@yhbt.net> wrote:
>> Eric Wong <normalperson@yhbt.net> wrote:
>> > James Cox <james@imaj.es> wrote:
>> > > trying to start my unicorn via bluepill, and running into this :
>>
>> > > master failed to start, check stderr log for details
>>
>> Both of you:
>
> Hi, did you guys ever figure this out?
>
>> Was there anything useful in stderr_path? The daemomized
>> process can never output to the terminal, so it had to
>> log somewhere... There's also strace, but looking at the
>> Unicorn code I haven't been able to find much (I'm very
>> sleep deprived right now, though).
>>
>> @Greg: I got through to /v6lUsuzD finally, can you put some
>> debug statements around your before_fork hook and print out
>> the pid path name and whether it exists or not?
>
--
James Cox,
Consultant, Raconteur, Photographer, Entrepreneur
t: 07968 349990 e: james@imaj.es w: http://imaj.es/
talk: http://twitter.com/imajes photos: http://flickr.com/imajes
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Shared memory between workers
@ 2010-04-26 8:18 2% Iñaki Baz Castillo
2010-04-26 19:03 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Iñaki Baz Castillo @ 2010-04-26 8:18 UTC (permalink / raw)
To: unicorn list
Hi, I plan to build a SIP TCP server (no UDP) based on
Unicorn/Rainbows! HTTP server. The main different between a SIP server
and HTTP server are:
- SIP uses persistent TCP connections, so I should use Rainbows!.
- For a SIP server it's not valid a simple request-response model.
Different workers could handle SIP messages (requests and responses)
belonging to the same SIP session so I need a shared memory between
all the workers.
Another option is using EventMachine, perhaps more suitable for this
purpose by design as it uses a single Ruby process so sharing memory
is not a problem. In the other side using a single process in a
multicore server is a pain.
I would like to use Unicorn/Rainbows as I love its design: by far it's
the more reliable and efficient Ruby HTTP server and it takes
advantages of Unix's features.
I don't want to use a DB server neither MemCache as "shared memory" as
it would be too slow. Is there any way to share RAM memory between
different Unicorn/Rainbows! workers in a *safe* way? I could create a
Hash or Array of SIP sessions into the master process so all the
workers reuse it, but I don't think it would be safe to access/write
into it from different Ruby processes. For that I would also need a
semaphore system (perhaps again a shared variable between all workers
in order to lock the shared Array/Hash when a worker writes into it).
Any tip about it? suggstions?
Thanks a lot.
--
Iñaki Baz Castillo
<ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Shared memory between workers
2010-04-26 8:18 2% Shared memory between workers Iñaki Baz Castillo
@ 2010-04-26 19:03 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-04-26 19:03 UTC (permalink / raw)
To: unicorn list
Iñaki Baz Castillo <ibc@aliax.net> wrote:
> Hi, I plan to build a SIP TCP server (no UDP) based on
> Unicorn/Rainbows! HTTP server. The main different between a SIP server
> and HTTP server are:
>
> - SIP uses persistent TCP connections, so I should use Rainbows!.
> - For a SIP server it's not valid a simple request-response model.
> Different workers could handle SIP messages (requests and responses)
> belonging to the same SIP session so I need a shared memory between
> all the workers.
>
> Another option is using EventMachine, perhaps more suitable for this
> purpose by design as it uses a single Ruby process so sharing memory
> is not a problem. In the other side using a single process in a
> multicore server is a pain.
> I would like to use Unicorn/Rainbows as I love its design: by far it's
> the more reliable and efficient Ruby HTTP server and it takes
> advantages of Unix's features.
>
> I don't want to use a DB server neither MemCache as "shared memory" as
> it would be too slow. Is there any way to share RAM memory between
> different Unicorn/Rainbows! workers in a *safe* way? I could create a
> Hash or Array of SIP sessions into the master process so all the
> workers reuse it, but I don't think it would be safe to access/write
> into it from different Ruby processes. For that I would also need a
> semaphore system (perhaps again a shared variable between all workers
> in order to lock the shared Array/Hash when a worker writes into it).
>
> Any tip about it? suggstions?
Hi Iñaki,
First off I wouldn't call Memcached slow... How many requests do you
need out of it and what kind of access patterns are you doing?
If you're on a modern Linux 2.6 box, then anything on the local
filesystem is basically shared memory if you have enough RAM to store
your working set (and it sounds like you do).
If you're willing to be confined to a single box, I've had great
experiences with TokyoCabinet the past few years. One system I built
shares 10G of read-mostly data (on 16G boxes) across 8 Unicorn
(previously Mongrel) workers. TokyoCabinet uses mmap and pread/pwrite,
so you can safely share DB handles across any number of native
threads/processes.
Writes in TC need filesystem locking and you can only have one active
writer, but that was still plenty fast enough in my experience.
If fsync() becomes a bottleneck, then for non-critical data either:
1) disable it (TC lets you)
2) or use libeatmydata if you choose something that doesn't let
you disable fsync()
<soapbox>
My personal opinion is that fsync()/O_SYNC are completely overrated.
All important machines these days have redundant storage, redundant
power supplies, ECC memory, hardware health monitoring/alerting and
backup power. Any remaining hardware/kernel failure possibilities
are far less likely than application bugs that corrupt data :)
But, if you really have critical data, /then/ get an SSD or
a battery-backed write cache.
</soapbox>
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Garbage collection outside of request cycle?
@ 2010-05-13 20:29 3% ` Luke Melia
2010-05-14 19:02 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Luke Melia @ 2010-05-13 20:29 UTC (permalink / raw)
To: Luke Melia; +Cc: unicorn list
> On May 6, 2010, at 4:57 PM, Eric Wong wrote:
>
>> ==> big_app_gc.rb <==
>> # This shouldn't hurt overall performance as long as the server cluster
>> # is at <=50% CPU capacity, and improves the performance of most memory
>> # intensive requests. This serves to improve _client-visible_
>> # performance (possibly at the cost of overall performance).
I thought the list might be interested in how this worked for us. I applied the patch to execute GC between each request. I'm using NewRelic to measure the app. Prior to the patch our, we spent about 25% of our aggregate time serving a request in GC and our application was running at around 20-30% CPU load. Our running app shows up as using ~330MB of memory.
Applying the patch cut the time spent in GC time to nearly zero and as predicted CPU spiked. Client-perceived responsiveness increased as well. Unfortunately, during our busiest time of the day, CPU load got so high that nginx locked up, so we rolled back the patch.
I made a simple change to execute GC once every 5 requests and applied it again. Aggregate time spent in GC reduced to out 10% of total request time. This resulted in a bout a 25% overall improvement in client response time. Big win! CPU maxes out at about 80% with this configuration..
One other thing I did was force GC to execute before_fork, on the theory that with COW, we would want to fork in the tidiest state possible. I have not measured this on it's own to evaluate it's impact.
Thanks again for the help and code on this, Eric.
Considering how useful this is, perhaps unicorn should have an after_request hook, to avoid the need to monkey-patch?
Cheers,
Luke
--
http://www.lukemelia.com/
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 3%]
* Re: Garbage collection outside of request cycle?
2010-05-13 20:29 3% ` Luke Melia
@ 2010-05-14 19:02 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-05-14 19:02 UTC (permalink / raw)
To: Luke Melia; +Cc: unicorn list
Luke Melia <luke@lukemelia.com> wrote:
> > On May 6, 2010, at 4:57 PM, Eric Wong wrote:
> >
> >> ==> big_app_gc.rb <== # This shouldn't hurt overall performance as
> >> long as the server cluster # is at <=50% CPU capacity, and improves
> >> the performance of most memory # intensive requests. This serves
> >> to improve _client-visible_ # performance (possibly at the cost of
> >> overall performance).
>
> I thought the list might be interested in how this worked for us. I
> applied the patch to execute GC between each request. I'm using
> NewRelic to measure the app. Prior to the patch our, we spent about
> 25% of our aggregate time serving a request in GC and our application
> was running at around 20-30% CPU load. Our running app shows up as
> using ~330MB of memory.
Thanks for the feedback, Luke.
Was the original 30% CPU load during peak traffic or normal traffic?
> Applying the patch cut the time spent in GC time to nearly zero and as
> predicted CPU spiked. Client-perceived responsiveness increased as
> well. Unfortunately, during our busiest time of the day, CPU load got
> so high that nginx locked up, so we rolled back the patch.
Yikes, nginx locking up is rare. It's worth investigating and fixing
that from the nginx side if you can reproduce it.
> I made a simple change to execute GC once every 5 requests and applied
> it again. Aggregate time spent in GC reduced to out 10% of total
> request time. This resulted in a bout a 25% overall improvement in
> client response time. Big win! CPU maxes out at about 80% with this
> configuration..
How many Unicorn workers do you have per-core? I forgot to mention that
you might want to run more workers to "hide" GC costs, something like a
poor man's concurrent GC.
> One other thing I did was force GC to execute before_fork, on the
> theory that with COW, we would want to fork in the tidiest state
> possible. I have not measured this on it's own to evaluate it's
> impact.
I doubt it'd help for anything other than the first <=5 requests that
hit the worker process.
> Thanks again for the help and code on this, Eric.
Thank _you_ for actually being willing to run and report back on
crazy experiments I come up with :)
> Considering how useful this is, perhaps unicorn should have an
> after_request hook, to avoid the need to monkey-patch?
Given configurability requirements and the ability of such a feature to
penalize some apps, I think making the monkey patch into middleware is a
nice compromise.
The GC.start in the below middleware runs at a slightly deeper stack
level, meaning GC will scan more and reap less than the original monkey
patch, but I doubt there'll be real difference if your app is already
spending enough time in GC to be a problem.
I've also pushed the following up to git://git.bogomips.org/unicorn.git
Let me know if you get a chance to test it on your app in place of
the monkey patch.
>From 95b75a5043b34f39ece4f52befb4b3f884dfdd20 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Fri, 14 May 2010 18:27:35 +0000
Subject: [PATCH] add Unicorn::OobGC middleware
This middleware allows configurable out-of-band garbage
collection outside of the normal request/response cycle.
It offers configurable paths (to only GC on expensive actions)
and intervals to limit GC frequency.
It is only expected to work well with Unicorn, as it would
hurt performance on single-threaded servers if they
have keepalive enabled. Obviously this does not work well
for multi-threaded or evented servers that serve multiple
clients at once.
---
lib/unicorn/oob_gc.rb | 58 +++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 58 insertions(+), 0 deletions(-)
create mode 100644 lib/unicorn/oob_gc.rb
diff --git a/lib/unicorn/oob_gc.rb b/lib/unicorn/oob_gc.rb
new file mode 100644
index 0000000..8dc4dcf
--- /dev/null
+++ b/lib/unicorn/oob_gc.rb
@@ -0,0 +1,58 @@
+# -*- encoding: binary -*-
+module Unicorn
+
+ # Run GC after every request, after closing the client socket and
+ # before attempting to accept more connections.
+ #
+ # This shouldn't hurt overall performance as long as the server cluster
+ # is at <50% CPU capacity, and improves the performance of most memory
+ # intensive requests. This serves to improve _client-visible_
+ # performance (possibly at the cost of overall performance).
+ #
+ # We'll call GC after each request is been written out to the socket, so
+ # the client never sees the extra GC hit it.
+ #
+ # This middleware is _only_ effective for applications that use a lot
+ # of memory, and will hurt simpler apps/endpoints that can process
+ # multiple requests before incurring GC.
+ #
+ # This middleware is only designed to work with Unicorn, as it harms
+ # keepalive performance.
+ #
+ # Example (in config.ru):
+ #
+ # require 'unicorn/oob_gc'
+ #
+ # # GC ever two requests that hit /expensive/foo or /more_expensive/foo
+ # # in your app. By default, this will GC once every 5 requests
+ # # for all endpoints in your app
+ # use Unicorn::OobGC, 2, %r{\A/(?:expensive/foo|more_expensive/foo)}
+ class OobGC < Struct.new(:app, :interval, :path, :nr, :env, :body)
+
+ def initialize(app, interval = 5, path = %r{\A/})
+ super(app, interval, path, interval)
+ end
+
+ def call(env)
+ status, headers, self.body = app.call(self.env = env)
+ [ status, headers, self ]
+ end
+
+ def each(&block)
+ body.each(&block)
+ end
+
+ # in Unicorn, this is closed _after_ the client socket
+ def close
+ body.close if body.respond_to?(:close)
+
+ if path =~ env['PATH_INFO'] && ((self.nr -= 1) <= 0)
+ self.nr = interval
+ self.body = nil
+ env.clear
+ GC.start
+ end
+ end
+
+ end
+end
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 0%]
* unicorn_rails cleanup (possible fix for Rails3) pushed
@ 2010-06-04 1:58 1% Eric Wong
2010-06-04 2:13 0% ` Michael Guterl
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2010-06-04 1:58 UTC (permalink / raw)
To: mongrel-unicorn
Hi all,
I've pushed the following patch out go git://git.bogomips.org/unicorn
along with a few other Rails-related test updates. This is more of a
shotgun fix (but less code is better :) since I haven't been able to
reproduce the brokeness people have been seeing with "unicorn_rails"
and Rails 3 betas.
Even though "unicorn" works perfectly well for Rails3, some folks will
inevitably run "unicorn_rails" because of the "rails" in its name.
If I were to do it all over again, I would've just made everybody write
config.ru files for Rails. But yes, programming is like sex, make one
mistake and support it for life :)
You can grab a git gem here, too:
http://unicorn.bogomips.org/files/unicorn-0.99.0.16.g59a625.gem
>From 4b44e21957e4cb8ec6ace5604fbe096dfd8959d2 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Thu, 3 Jun 2010 22:52:11 +0000
Subject: [PATCH] unicorn_rails: avoid duplicating config.ru logic
This should allow "unicorn_rails" to be used seamlessly
with Rails 3 projects which package config.ru for you.
---
bin/unicorn_rails | 50 ++++++++++++++------------------------------------
1 files changed, 14 insertions(+), 36 deletions(-)
diff --git a/bin/unicorn_rails b/bin/unicorn_rails
index 72ab288..45a9b11 100755
--- a/bin/unicorn_rails
+++ b/bin/unicorn_rails
@@ -109,53 +109,30 @@ end
ru = ARGV[0] || (File.exist?('config.ru') ? 'config.ru' : nil)
-if ru && ru =~ /\.ru\z/
- # parse embedded command-line options in config.ru comments
- /^#\\(.*)/ =~ File.read(ru) and opts.parse!($1.split(/\s+/))
-end
-
-def rails_builder(ru, daemonize)
+def rails_builder(daemonize)
# this lambda won't run until after forking if preload_app is false
lambda do ||
# Load Rails and (possibly) the private version of Rack it bundles.
begin
require 'config/boot'
+ require 'config/environment'
rescue LoadError => err
abort "#$0 must be run inside RAILS_ROOT: #{err.inspect}"
end
- inner_app = case ru
- when nil
- require 'config/environment'
-
- defined?(::Rails::VERSION::STRING) or
- abort "Rails::VERSION::STRING not defined by config/{boot,environment}"
- # it seems Rails >=2.2 support Rack, but only >=2.3 requires it
- old_rails = case ::Rails::VERSION::MAJOR
- when 0, 1 then true
- when 2 then Rails::VERSION::MINOR < 3 ? true : false
- else
- false
- end
-
- if old_rails
- require 'unicorn/app/old_rails'
- Unicorn::App::OldRails.new
- else
- ActionController::Dispatcher.new
- end
- when /\.ru$/
- raw = File.read(ru)
- raw.sub!(/^__END__\n.*/, '')
- eval("Rack::Builder.new {(#{raw}\n)}.to_app", TOPLEVEL_BINDING, ru)
+ defined?(::Rails::VERSION::STRING) or
+ abort "Rails::VERSION::STRING not defined by config/{boot,environment}"
+ # it seems Rails >=2.2 support Rack, but only >=2.3 requires it
+ old_rails = case ::Rails::VERSION::MAJOR
+ when 0, 1 then true
+ when 2 then Rails::VERSION::MINOR < 3 ? true : false
else
- require ru
- Object.const_get(File.basename(ru, '.rb').capitalize)
+ false
end
Rack::Builder.new do
map_path = ENV['RAILS_RELATIVE_URL_ROOT'] || '/'
- if inner_app.class.to_s == "Unicorn::App::OldRails"
+ if old_rails
if map_path != '/'
# patches + tests welcome, but I really cbf to deal with this
# since all apps I've ever dealt with just use "/" ...
@@ -163,23 +140,24 @@ def rails_builder(ru, daemonize)
end
$stderr.puts "LogTailer not available for Rails < 2.3" unless daemonize
$stderr.puts "Debugger not available" if $DEBUG
+ require 'unicorn/app/old_rails'
map(map_path) do
use Unicorn::App::OldRails::Static
- run inner_app
+ run Unicorn::App::OldRails.new
end
else
use Rails::Rack::LogTailer unless daemonize
use Rails::Rack::Debugger if $DEBUG
map(map_path) do
use Rails::Rack::Static
- run inner_app
+ run ActionController::Dispatcher.new
end
end
end.to_app
end
end
-app = rails_builder(ru, daemonize)
+app = ru ? Unicorn.builder(ru, opts) : rails_builder(daemonize)
options[:listeners] << "#{host}:#{port}" if set_listener
if $DEBUG
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 1%]
* Re: unicorn_rails cleanup (possible fix for Rails3) pushed
2010-06-04 1:58 1% unicorn_rails cleanup (possible fix for Rails3) pushed Eric Wong
@ 2010-06-04 2:13 0% ` Michael Guterl
2010-06-04 2:48 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Michael Guterl @ 2010-06-04 2:13 UTC (permalink / raw)
To: unicorn list
On Thu, Jun 3, 2010 at 9:58 PM, Eric Wong <normalperson@yhbt.net> wrote:
<snip>
> If I were to do it all over again, I would've just made everybody write
> config.ru files for Rails. But yes, programming is like sex, make one
> mistake and support it for life :)
That is probably one of the funniest things I've read in awhile.
Thanks for the unexpected source of humor! :)
Best,
Michael Guterl
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: unicorn_rails cleanup (possible fix for Rails3) pushed
2010-06-04 2:13 0% ` Michael Guterl
@ 2010-06-04 2:48 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-06-04 2:48 UTC (permalink / raw)
To: unicorn list
Michael Guterl <mguterl@gmail.com> wrote:
> On Thu, Jun 3, 2010 at 9:58 PM, Eric Wong <normalperson@yhbt.net> wrote:
> <snip>
> > If I were to do it all over again, I would've just made everybody write
> > config.ru files for Rails. But yes, programming is like sex, make one
> > mistake and support it for life :)
>
> That is probably one of the funniest things I've read in awhile.
> Thanks for the unexpected source of humor! :)
You're welcome :> It's not my line, but I heard it many years ago.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: unicorn_rails cleanup (possible fix for Rails3) pushed
@ 2010-06-08 20:55 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-06-08 20:55 UTC (permalink / raw)
To: unicorn list; +Cc: Hongli Lai
Hongli Lai <hongli@phusion.nl> wrote:
> On Tue, Jun 8, 2010 at 9:20 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Thanks Hongli, so this only affects people who remove the
> > config.ru that Rails 3 creates for them? Yikes...
>
> No. The problem even occurs if you already have config.ru. But the
> thing is, Rails 3 has deprecated ActionController::Dispatcher a few
> weeks ago and replaced it with a stub. Rails::Rack::Static changed its
> interface and must be constructed differently. The only way to obtain
> a valid Rack endpoint for the app seems to be parsing
> config/application.rb
Actually, I made "unicorn_rails" completely bypass the "rails_builder"
method if there's a config.ru, so it should never hit the
ActionController::Dispatcher stuff.
> > Let me know if the edited patch below looks alright to you.
>
> Yes it looks fine. A bit overcomplicated regexp compared to using
> 'strip' but whatever works. :)
I just kept the regexp as-is, works for me.
I just managed to push this to git://git.bogomips.org/unicorn.git before
my Internet connection died on me earlier today. I've beefed up the
tests a bit but will probably do more later.
Eric Wong (4):
t0300: Rails 3 test actually uses unicorn_rails
tests: libify common rails3 setup code
unicorn_rails: fix requires for Ruby 1.9.2
tests: add Rails 3 test for the missing config.ru case
Hongli Lai (Phusion) (1):
Fix unicorn_rails compatibility with the latest Rails 3 code
Thanks again!
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: [ANN] unicorn 0.990.0 - inching towards 1.0
@ 2010-06-10 7:38 2% ` Eric Wong
2010-06-10 8:46 0% ` Alexander Simonov
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2010-06-10 7:38 UTC (permalink / raw)
To: unicorn list
Alexander Simonov <alex@simonov.me> wrote:
> On Jun 9, 2010, at 9:22 PM, Eric Wong wrote:
> > Alexander Simonov <alex@simonov.me> wrote:
> >>
> >> Hello!
> >>
> >> One question: it's a normal state if i start rails app without preload
> >> and after Gem.refresh all works go down by exception from Ge.refresh
> >> and master process all time trying reup they. And it's go to
> >> recursion. I know it's issue of people who start app, but in any case
> >> it's must be exception for stop of master process. Is i check code
> >> when starts workers and if we have preload_app false method run
> >> build_app!. If we have preload_app true - then we check it before
> >> start worker.
> >
> > Hi Alexander,
> >
> > I hope I'm understanding you correctly, so you want the master process
> > to die if you've misdeployed or misconfigured your app?
>
> yes, something like this.
>
> >
> > This is one of those sharp edges of Unicorn that might be pointless
> > to protect against with preload_app=false...
> >
> > The general rule is that you shouldn't be mucking around with Gem (or
> > any other library) installations on a production server unless you're
> > deploying. And whenever you're deploying, you would always be checking
> > to see you've deployed correctly anyways, so you can detect any screwups
> > in the installation/deployment.
> >
> > Let me know if that makes sense.
>
> Ok. But the main thing is the master process goes to recursion restart
> of workers if they return exit or exception.May be add the counter?
> For example, 10 times we trying to respawn working process and after
> stop master. I think it makes sense.
It'd have to be a time-aware counter, because sometimes slightly broken
_will_ exit/die 10 times over the period of an hour. But the site can
still be making enough money with 99.8% successful service and not
care about spending time/resources fixing the last 0.2% :)
But even with a proper time-aware counter, the app is still broken at
deploy time. So I don't believe there is much point in adding more code
just to exit the process when the proper solution is for the user to
_fix_ the app.
And again (I seem to remember saying this months ago), any time you're
deploying, you should be paying extra attention to the app anyways.
That means testing with actual HTTP requests, not just seeing of the
process is up. Just having a running process serving error pages is
equally worthless as not running the server at all.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: [ANN] unicorn 0.990.0 - inching towards 1.0
2010-06-10 7:38 2% ` Eric Wong
@ 2010-06-10 8:46 0% ` Alexander Simonov
2010-06-11 1:27 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Alexander Simonov @ 2010-06-10 8:46 UTC (permalink / raw)
To: unicorn list
On Jun 10, 2010, at 10:38 AM, Eric Wong wrote:
> It'd have to be a time-aware counter, because sometimes slightly broken
> _will_ exit/die 10 times over the period of an hour. But the site can
> still be making enough money with 99.8% successful service and not
> care about spending time/resources fixing the last 0.2% :)
>
> But even with a proper time-aware counter, the app is still broken at
> deploy time. So I don't believe there is much point in adding more code
> just to exit the process when the proper solution is for the user to
> _fix_ the app.
>
> And again (I seem to remember saying this months ago), any time you're
> deploying, you should be paying extra attention to the app anyways.
> That means testing with actual HTTP requests, not just seeing of the
> process is up. Just having a running process serving error pages is
> equally worthless as not running the server at all.
Hm.. It's makes sense.
But much better if it will be writing in the documentation for the future.
Thank you!
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: [ANN] unicorn 0.990.0 - inching towards 1.0
2010-06-10 8:46 0% ` Alexander Simonov
@ 2010-06-11 1:27 0% ` Eric Wong
2010-06-11 18:00 0% ` Alexander Simonov
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2010-06-11 1:27 UTC (permalink / raw)
To: unicorn list
Alexander Simonov <alex@simonov.me> wrote:
> On Jun 10, 2010, at 10:38 AM, Eric Wong wrote:
> > And again (I seem to remember saying this months ago), any time you're
> > deploying, you should be paying extra attention to the app anyways.
> > That means testing with actual HTTP requests, not just seeing of the
> > process is up. Just having a running process serving error pages is
> > equally worthless as not running the server at all.
>
> Hm.. It's makes sense.
> But much better if it will be writing in the documentation for the future.
Here's what I've pushed out, let me know if everything makes
sense. I think I'll release 0.991.0 in a little bit.
>From f9baab18705218dae4c37c2c92d1ceea151bba8e Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Thu, 10 Jun 2010 17:57:08 -0700
Subject: [PATCH] docs: hopefully clarify preload_app=false behavior
While we're at it, inform people of why they might use
a symlink
---
SIGNALS | 7 ++++++-
lib/unicorn/configurator.rb | 21 ++++++++++++++++++---
2 files changed, 24 insertions(+), 4 deletions(-)
diff --git a/SIGNALS b/SIGNALS
index be96892..8851775 100644
--- a/SIGNALS
+++ b/SIGNALS
@@ -14,7 +14,12 @@ Unicorn and nginx.
will also pick up any application code changes when restarted. If
"preload_app" is true, then application code changes will have no
effect; USR2 + QUIT (see below) must be used to load newer code in
- this case.
+ this case. When reloading the application, +Gem.refresh+ will
+ be called so updated code for your application can pick up newly
+ installed RubyGems. It is not recommended that you uninstall
+ libraries your application depends on while Unicorn is running,
+ as respawned workers may enter a spawn loop when they fail to
+ load an uninstalled dependency.
* INT/TERM - quick shutdown, kills all workers immediately
diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb
index 71c7a8a..533e0ed 100644
--- a/lib/unicorn/configurator.rb
+++ b/lib/unicorn/configurator.rb
@@ -311,7 +311,22 @@ module Unicorn
#
# In addition to reloading the unicorn-specific config settings,
# SIGHUP will reload application code in the working
- # directory/symlink when workers are gracefully restarted.
+ # directory/symlink when workers are gracefully restarted when
+ # preload_app=false (the default). As reloading the application
+ # sometimes requires RubyGems updates, +Gem.refresh+ is always
+ # called before the application is loaded (for RubyGems users).
+ #
+ # During deployments, care should _always_ be taken to ensure your
+ # applications are properly deployed and running. Using
+ # preload_app=false (the default) means you _must_ check if
+ # your application is responding properly after a deployment.
+ # Improperly deployed applications can go into a spawn loop
+ # if the application fails to load. While your children are
+ # in a spawn loop, it is is possible to fix an application
+ # by properly deploying all required code and dependencies.
+ # Using preload_app=true means any application load error will
+ # cause the master process to exit with an error.
+
def preload_app(bool)
case bool
when TrueClass, FalseClass
@@ -344,9 +359,9 @@ module Unicorn
set_path(:stdout_path, path)
end
- # sets the working directory for Unicorn. This ensures USR2 will
+ # sets the working directory for Unicorn. This ensures SIGUSR2 will
# start a new instance of Unicorn in this directory. This may be
- # a symlink.
+ # a symlink, a common scenario for Capistrano users.
def working_directory(path)
# just let chdir raise errors
path = File.expand_path(path)
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 0%]
* Re: Fwd: Support for Soft Timeout in Unicorn
@ 2010-06-11 1:56 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-06-11 1:56 UTC (permalink / raw)
To: unicorn list
Pierre Baillet <oct@fotonauts.com> wrote:
> Hello Unicorns,
>
> I've manage to create a simple middleware that replaces the soft
> timeout feature. You can have a look at it at
> http://gist.github.com/431451
>
> Note that some weird Ruby interpreter behavior breaks at least the
> first level of the generated stacktrace (it indicates the actual
> method where the raise happened but the wrong line number and file).
You should be able to avoid spawning a new thread for every request, as
that can get very expensive on some systems. Maybe a global hash
guarded by a mutex that tells the worker Thread which thread to raise
on. The worker thread could sleep until the next timeout registered.
But then again, take care to only spawn new threads in workers (with
preload_app=true), as threads never get carried across fork. And
threads may leave mutexes and such in a bad state when they vanish
in the child.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: [ANN] unicorn 0.990.0 - inching towards 1.0
2010-06-11 1:27 0% ` Eric Wong
@ 2010-06-11 18:00 0% ` Alexander Simonov
0 siblings, 0 replies; 200+ results
From: Alexander Simonov @ 2010-06-11 18:00 UTC (permalink / raw)
To: unicorn list
On Jun 11, 2010, at 4:27 AM, Eric Wong wrote:
> Alexander Simonov <alex@simonov.me> wrote:
>> On Jun 10, 2010, at 10:38 AM, Eric Wong wrote:
>>> And again (I seem to remember saying this months ago), any time you're
>>> deploying, you should be paying extra attention to the app anyways.
>>> That means testing with actual HTTP requests, not just seeing of the
>>> process is up. Just having a running process serving error pages is
>>> equally worthless as not running the server at all.
>>
>> Hm.. It's makes sense.
>> But much better if it will be writing in the documentation for the future.
>
> Here's what I've pushed out, let me know if everything makes
> sense. I think I'll release 0.991.0 in a little bit.
Ok! Thank you!
Good. After release I will update golden_brindle too :)
>
>> From f9baab18705218dae4c37c2c92d1ceea151bba8e Mon Sep 17 00:00:00 2001
> From: Eric Wong <normalperson@yhbt.net>
> Date: Thu, 10 Jun 2010 17:57:08 -0700
> Subject: [PATCH] docs: hopefully clarify preload_app=false behavior
>
> While we're at it, inform people of why they might use
> a symlink
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Unicorn future plans
@ 2010-06-14 22:10 3% ` Eric Wong
2010-06-15 7:09 3% ` John-Paul Bader
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2010-06-14 22:10 UTC (permalink / raw)
To: unicorn list
John-Paul Bader <hukl@berlin.ccc.de> wrote:
> Hey,
>
> your plans sound great except for ignoring FreeBSD ;)
>
> Seriously many many Servers, including many which I administrate run
> on FreeBSD and they do so very well. I'm not very familiar with the
> internals of the FreeBSD Kernel but I can assure you that it runs
> perfectly on 8 and 16 core machines and I would extremely happy if I
> could continue to use unicorn on them.
Hi John-Paul,
Reread my post carefully, not much is changing for 8-16 core users.
FreeBSD and SMP will continue to be supported. I wasn't referring
to SMP at all when I was talking about Linux-only bits.
> It should be easier to maintain a FreeBSD version as they (FreeBSD
> developers) tend to aim for more consistency across releases than
> Linux. You said once that you don't run any BSD machines but I'd be
> happy to offer you access to one of my BSD servers for testing.
Thanks for the offer. I'll keep it in mind if I need it again.
> Furthermore since Mac OS X and FreeBSD share many features like kqueue
> and many Ruby developers are running macs it would make even more
> sense to at least care about BSD even if its not you primary platform.
I know, I've fixed some things on OSX platforms via FreeBSD (OSX scares
me with its GUI-ness). Keep in mind kqueue (and epoll) are worthless
for Unicorn itself since Unicorn only handles one client at a time.
However, Rainbows! can already use kqueue/epoll with EventMachine and
Rev.
> I don't want to start a flame war or say that one OS is better than
> another. Linux is certainly great but so is FreeBSD. For the SMP part
> I recommend the following page on freebsd.org:
Again, I wasn't talking about SMP at all. SMP works fine on current
Unicorn and mid-sized servers. When going to more cores, SMP itself
is a bottleneck, not the kernel.
With Unicorn 2.x, I'm targeting NUMA hardware that isn't available in
commodity servers yet. Currently, NUMA makes _zero_ sense in web
application servers, but in case things change in a few years, we'll be
ready.
It may be a chicken-and-egg problem, too. Hardware manufacturers are
slow to commoditize NUMA because a good chunk of software can't even
utilize SMP effectively :)
> To conclude I can only say that I'm running unicorn on several FreeBSD
> hosts and so far I couldn't be happier with it. As stated before, I'd
> be more than happy to continue using that setup.
Again, don't worry :) You'll be able to continue using everything
as-is.
> Kindest regards and thanks for all you efforts,
No problem. Please don't top post in the future.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 3%]
* Re: Unicorn future plans
2010-06-14 22:10 3% ` Eric Wong
@ 2010-06-15 7:09 3% ` John-Paul Bader
0 siblings, 0 replies; 200+ results
From: John-Paul Bader @ 2010-06-15 7:09 UTC (permalink / raw)
To: unicorn list
On 15.06.2010, at 00:10, Eric Wong wrote:
> John-Paul Bader <hukl@berlin.ccc.de> wrote:
>> Hey,
>>
>> your plans sound great except for ignoring FreeBSD ;)
>>
>> Seriously many many Servers, including many which I administrate run
>> on FreeBSD and they do so very well. I'm not very familiar with the
>> internals of the FreeBSD Kernel but I can assure you that it runs
>> perfectly on 8 and 16 core machines and I would extremely happy if I
>> could continue to use unicorn on them.
>
> Hi John-Paul,
>
> Reread my post carefully, not much is changing for 8-16 core users.
> FreeBSD and SMP will continue to be supported. I wasn't referring
> to SMP at all when I was talking about Linux-only bits.
Hey again,
sorry if I misinterpreted your post. My mail could've been shorter by saying that there are FreeBSD supercomputer clusters and that if we get 1000 cores or more FreeBSD will be fast to adapt so don't let it slip out of focus ;)
My offer for a FreeBSD machine account stands and I'd be more than happy to report issues as they occur to help you out if BSD issues occur.
Thanks again,
John
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 3%]
* Re: Forking off the unicorn master process to create a background worker
@ 2010-06-15 17:55 2% ` Russell Branca
2010-06-15 22:14 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Russell Branca @ 2010-06-15 17:55 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn list
Hello Eric,
Sorry for the delayed response, with the combination of being sick and
heading out of town for a while, this project got put on the
backburner. I really appreciate your response and think its a clean
solution for what I'm trying to do. I've started back in getting the
job queue working this week, and will hopefully have a working
solution in the next day or two. A little more information about what
I'm doing, I'm trying to create a centralized resque job queue server
that each of the different applications can queue work into, so I'll
be using redis behind resque for storing jobs and what not, which
brings me an area I'm not sure of the best approach on. So when we hit
the job queue endpoint in the rack app, it spawns the new worker, and
then immediately returns the 200 ok started background job message,
which cuts off communication back to the job queue. My plan is to save
a status message of the result of the background task into redis, and
have resque check that to verify the task was successful. Is there a
better approach for returning the resulting status code with unicorn,
or is this a reasonable approach? Thanks again for your help.
-Russell
On Wed, May 26, 2010 at 2:05 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Russell Branca <chewbranca@gmail.com> wrote:
>> Hello,
>>
>> I'm trying to find an efficient way to create a new instance of a
>> rails application to perform some background tasks without having to
>> load up the entire rails stack every time, so I figured forking off
>> the master process would be a good way to go. Now I can easily just
>> increment the worker count and then send a web request in, but the new
>> worker would be part of the main worker pool, so in the time between
>> spawning a new worker and sending the request, another request could
>> have come in and snagged that worker. Is it possible to create a new
>> worker and not have it enter the main worker pool so I could access it
>> directly?
>
> Hi Russell,
>
> You could try having an endpoint in your webapp (with authentication, or
> have it reject env['REMOTE_ADDR'] != '127.0.0.1') that runs the
> background task for you. Since it's a background app, you should
> probably fork + Process.setsid + fork (or Process.daemon in 1.9), and
> return an HTTP response immediately so your app can serve other
> requests.
>
> The following example should be enough to get you started (totally
> untested)
>
> ------------ config.ru -------------
> require 'rack/lobster'
>
> map "/.seekrit_endpoint" do
> use Rack::ContentLength
> use Rack::ContentType, 'text/plain'
> run(lambda { |env|
> return [ 403, {}, [] ] if env['REMOTE_ADDR'] != '127.0.0.1'
> pid = fork
> if pid
> Process.waitpid(pid)
>
> # cheap way to avoid unintentional fd sharing with our children,
> # this causes the current Unicorn worker to exit after sending
> # the response:
> # Otherwise you'd have to be careful to disconnect+reconnect
> # databases/memcached/redis/whatever (in both the parent and
> # child) to avoid unintentional sharing that'll lead to headaches
> Process.kill(:QUIT, $$)
>
> [ 200, {}, [ "started background process\n" ] ]
> else
> # child, daemonize it so the unicorn master won't need to
> # reap it (that's the job of init)
> Process.setsid
> exit if fork
>
> begin
> # run your background code here instead of sleeping
> sleep 5
> env["rack.logger"].info "done sleeping"
> rescue => e
> env["rack.logger"].error(e.inspect)
> end
> # make sure we don't enter the normal response cycle back in the
> # worker...
> exit!(0)
> end
> })
> end
>
> map "/" do
> run Rack::Lobster.new
> end
>
>> I know this is not your typical use case for unicorn, and you're
>> probably thinking there is a lot better ways to do this, however, I
>> currently have a rails framework that powers a handful of standalone
>> applications on a server with limited resources, and I'm trying to
>> make a centralized queue that all the applications use, so the queue
>> needs to be able to spawn a new worker for each of the applications
>> efficiently, and incrementing/decrementing worker counts in unicorn is
>> the most efficient way I've found to spawn a new rails instance.
>
> Yeah, it's definitely an odd case and there are ways to shoot yourself
> in the foot with it (especially with unintentional fd sharing), but Ruby
> exposes all the Unix process management goodies better than most
> languages (probably better than anything else I've used).
>
>> Any help, suggestions or insight into this would be greatly appreciated.
>
> Let us know how it goes :)
>
> --
> Eric Wong
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Forking off the unicorn master process to create a background worker
2010-06-15 17:55 2% ` Russell Branca
@ 2010-06-15 22:14 0% ` Eric Wong
2010-06-15 22:51 0% ` Russell Branca
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2010-06-15 22:14 UTC (permalink / raw)
To: Russell Branca; +Cc: unicorn list
Russell Branca <chewbranca@gmail.com> wrote:
> Hello Eric,
>
> Sorry for the delayed response, with the combination of being sick and
> heading out of town for a while, this project got put on the
> backburner. I really appreciate your response and think its a clean
> solution for what I'm trying to do. I've started back in getting the
> job queue working this week, and will hopefully have a working
> solution in the next day or two. A little more information about what
> I'm doing, I'm trying to create a centralized resque job queue server
> that each of the different applications can queue work into, so I'll
> be using redis behind resque for storing jobs and what not, which
> brings me an area I'm not sure of the best approach on. So when we hit
> the job queue endpoint in the rack app, it spawns the new worker, and
> then immediately returns the 200 ok started background job message,
> which cuts off communication back to the job queue. My plan is to save
> a status message of the result of the background task into redis, and
> have resque check that to verify the task was successful. Is there a
> better approach for returning the resulting status code with unicorn,
> or is this a reasonable approach? Thanks again for your help.
Hi Russell, please don't top post, thanks.
If you already have a queue server (and presumably a standalone app
processing the queue), I would probably forgo the background Unicorn
worker entirely.
Based on my ancient (mid-2000s) knowledge of user-facing web
applications: the application should queue the job, return 200, and have
HTML meta refresh to constantly reload the page every few seconds.
Hitting the reload endpoint would check the database (Redis in this
case) for completion, and return a new HTML page to stop the meta
refresh loop.
This means you're no longer keeping a single Unicorn worker idle and
wasting it. Nowadays you could do it with long-polling on
Rainbows!/Thin/Zbatery, too, but long-polling is less reliable for
people switching between WiFi access points. The meta refresh method
can be a waste of power/bandwidth on the client side if the background
job takes a long time, though.
I'm familiar at all with Resque or Redis, but I suspect other folks
on this mailing list should be able to help you flesh out the details.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Forking off the unicorn master process to create a background worker
2010-06-15 22:14 0% ` Eric Wong
@ 2010-06-15 22:51 0% ` Russell Branca
2010-06-16 0:06 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Russell Branca @ 2010-06-15 22:51 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn list
On Tue, Jun 15, 2010 at 3:14 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Russell Branca <chewbranca@gmail.com> wrote:
>> Hello Eric,
>>
>> Sorry for the delayed response, with the combination of being sick and
>> heading out of town for a while, this project got put on the
>> backburner. I really appreciate your response and think its a clean
>> solution for what I'm trying to do. I've started back in getting the
>> job queue working this week, and will hopefully have a working
>> solution in the next day or two. A little more information about what
>> I'm doing, I'm trying to create a centralized resque job queue server
>> that each of the different applications can queue work into, so I'll
>> be using redis behind resque for storing jobs and what not, which
>> brings me an area I'm not sure of the best approach on. So when we hit
>> the job queue endpoint in the rack app, it spawns the new worker, and
>> then immediately returns the 200 ok started background job message,
>> which cuts off communication back to the job queue. My plan is to save
>> a status message of the result of the background task into redis, and
>> have resque check that to verify the task was successful. Is there a
>> better approach for returning the resulting status code with unicorn,
>> or is this a reasonable approach? Thanks again for your help.
>
> Hi Russell, please don't top post, thanks.
>
> If you already have a queue server (and presumably a standalone app
> processing the queue), I would probably forgo the background Unicorn
> worker entirely.
>
> Based on my ancient (mid-2000s) knowledge of user-facing web
> applications: the application should queue the job, return 200, and have
> HTML meta refresh to constantly reload the page every few seconds.
>
> Hitting the reload endpoint would check the database (Redis in this
> case) for completion, and return a new HTML page to stop the meta
> refresh loop.
>
> This means you're no longer keeping a single Unicorn worker idle and
> wasting it. Nowadays you could do it with long-polling on
> Rainbows!/Thin/Zbatery, too, but long-polling is less reliable for
> people switching between WiFi access points. The meta refresh method
> can be a waste of power/bandwidth on the client side if the background
> job takes a long time, though.
>
> I'm familiar at all with Resque or Redis, but I suspect other folks
> on this mailing list should be able to help you flesh out the details.
>
> --
> Eric Wong
>
Hi Eric,
I have a queue server, but I don't have a standalone app processing
the jobs, because I have a large number of stand alone applications on
a single server. Right now I've got 12 separate apps running, so if I
wanted to have a standalone app for each, that would be 12 additional
applications in memory for handling background jobs. The whole reason
I want to go with the unicorn worker approach for handling background
jobs, is so I can fork off the master process as needed, avoid the
spawning time for a normal rails instance, and only use workers as
needed. This way I can have just a few workers running at any given
time, rather than 1 worker for each app. The number of apps is only
going to increase, but I want to keep the worker pool a constant. I'll
probably just update status of completion with redis, these jobs won't
be run by users, this is all background stuff like sending
notifications, data analysis, feed parsing, etc etc, so I'm planning
on just having resque initiate a request directly, and then use
unicorn to process the task in the background.
I didn't exactly follow what you meant when you were talking about a
unicorn worker being idle, from the example config.ru you responded
with earlier on, it looks like I can just spawn a new worker that will
be outside of the normal worker pool to handle the job. I'm pretty
sure this will work, I was curious about the best approach for
returning completion status, but I think just having the worker record
its status and exit is better than having long polling connections
open between the job queue and the new unicorn worker.
-Russell
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Forking off the unicorn master process to create a background worker
2010-06-15 22:51 0% ` Russell Branca
@ 2010-06-16 0:06 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-06-16 0:06 UTC (permalink / raw)
To: Russell Branca; +Cc: unicorn list
Russell Branca <chewbranca@gmail.com> wrote:
> On Tue, Jun 15, 2010 at 3:14 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Russell Branca <chewbranca@gmail.com> wrote:
> >> Hello Eric,
> >>
> >> Sorry for the delayed response, with the combination of being sick and
> >> heading out of town for a while, this project got put on the
> >> backburner. I really appreciate your response and think its a clean
> >> solution for what I'm trying to do. I've started back in getting the
> >> job queue working this week, and will hopefully have a working
> >> solution in the next day or two. A little more information about what
> >> I'm doing, I'm trying to create a centralized resque job queue server
> >> that each of the different applications can queue work into, so I'll
> >> be using redis behind resque for storing jobs and what not, which
> >> brings me an area I'm not sure of the best approach on. So when we hit
> >> the job queue endpoint in the rack app, it spawns the new worker, and
> >> then immediately returns the 200 ok started background job message,
> >> which cuts off communication back to the job queue. My plan is to save
> >> a status message of the result of the background task into redis, and
> >> have resque check that to verify the task was successful. Is there a
> >> better approach for returning the resulting status code with unicorn,
> >> or is this a reasonable approach? Thanks again for your help.
> >
> > Hi Russell, please don't top post, thanks.
> >
> > If you already have a queue server (and presumably a standalone app
> > processing the queue), I would probably forgo the background Unicorn
> > worker entirely.
> >
> > Based on my ancient (mid-2000s) knowledge of user-facing web
> > applications: the application should queue the job, return 200, and have
> > HTML meta refresh to constantly reload the page every few seconds.
> >
> > Hitting the reload endpoint would check the database (Redis in this
> > case) for completion, and return a new HTML page to stop the meta
> > refresh loop.
> >
> > This means you're no longer keeping a single Unicorn worker idle and
> > wasting it. Nowadays you could do it with long-polling on
> > Rainbows!/Thin/Zbatery, too, but long-polling is less reliable for
> > people switching between WiFi access points. The meta refresh method
> > can be a waste of power/bandwidth on the client side if the background
> > job takes a long time, though.
> >
> > I'm familiar at all with Resque or Redis, but I suspect other folks
> > on this mailing list should be able to help you flesh out the details.
>
> Hi Eric,
>
> I have a queue server, but I don't have a standalone app processing
> the jobs, because I have a large number of stand alone applications on
> a single server. Right now I've got 12 separate apps running, so if I
> wanted to have a standalone app for each, that would be 12 additional
> applications in memory for handling background jobs. The whole reason
> I want to go with the unicorn worker approach for handling background
> jobs, is so I can fork off the master process as needed, avoid the
> spawning time for a normal rails instance, and only use workers as
> needed. This way I can have just a few workers running at any given
> time, rather than 1 worker for each app. The number of apps is only
> going to increase, but I want to keep the worker pool a constant. I'll
> probably just update status of completion with redis, these jobs won't
> be run by users, this is all background stuff like sending
> notifications, data analysis, feed parsing, etc etc, so I'm planning
> on just having resque initiate a request directly, and then use
> unicorn to process the task in the background.
Ah, so I guess it's a single queue server but multiple queues? I
guess thats where I got confused with your description.
> I didn't exactly follow what you meant when you were talking about a
> unicorn worker being idle, from the example config.ru you responded
> with earlier on, it looks like I can just spawn a new worker that will
> be outside of the normal worker pool to handle the job. I'm pretty
> sure this will work, I was curious about the best approach for
> returning completion status, but I think just having the worker record
> its status and exit is better than having long polling connections
> open between the job queue and the new unicorn worker.
Yes, having the fork as I made in the example should work. I haven't
tested it, of course :) My instincts tell me recording the status and
exiting ASAP is better because it uses less memory.
You should test and experiment with it either way. You know your apps,
requirements, and Redis/Resque far better than I do :) Consider
software an evolutionary process, so whatever the "best approach" may
be, another one can usurp it eventually or be completely wrong in a
slightly different setting :)
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Purpose of "Status" header in HTTP responses?
@ 2010-06-24 14:56 2% ` Craig Davey
0 siblings, 0 replies; 200+ results
From: Craig Davey @ 2010-06-24 14:56 UTC (permalink / raw)
To: mongrel-unicorn
Eric, thanks for the detailed reply and the proxy_hide_header tip.
We used proxy_hide_header to remove the Status header from our responses.
After the change our client continued to receive the same error so I’m
pretty sure that the Status header was never the problem in this case.
Sorry for raising this issue unnecessarily. Thanks again for your help.
Craig
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* SIGWINCH
@ 2010-07-13 4:25 2% Lawrence Pit
2010-07-13 4:34 0% ` SIGWINCH Eric Wong
0 siblings, 1 reply; 200+ results
From: Lawrence Pit @ 2010-07-13 4:25 UTC (permalink / raw)
To: unicorn list
Hi,
I followed the procedure to replace a running unicorn. Works fine,
except for one thing: after I decide to back out, ie I send a HUP to the
old master followed by a QUIT to the new master, then:
1. the new master does die, and the old master does work again
2. the "old" master keeps running with a process name of "master (old)"
3. the "old" master keeps running with a pid file named unicorn.pid.oldbin
Because the file is now named unicorn.pid.oldbin a later attempt to
deploy using scripts is made rather difficult as they assume a pid file
named unicorn.pid.
How could I get unicorn to rename the pid file back to unicorn.pid ? I
see some code in method +reap_all_workers+ that suggests it would rename
the pid file back to its original, but for some reason it doesn't for
me. Am I missing a step?
PS. Using unicorn 1.1.1.
Cheers,
Lawrence
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: SIGWINCH
2010-07-13 4:25 2% SIGWINCH Lawrence Pit
@ 2010-07-13 4:34 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-07-13 4:34 UTC (permalink / raw)
To: unicorn list
Lawrence Pit <lawrence.pit@gmail.com> wrote:
> Hi,
>
> I followed the procedure to replace a running unicorn. Works fine,
> except for one thing: after I decide to back out, ie I send a HUP to the
> old master followed by a QUIT to the new master, then:
>
> 1. the new master does die, and the old master does work again
>
> 2. the "old" master keeps running with a process name of "master (old)"
>
> 3. the "old" master keeps running with a pid file named unicorn.pid.oldbin
>
> Because the file is now named unicorn.pid.oldbin a later attempt to
> deploy using scripts is made rather difficult as they assume a pid file
> named unicorn.pid.
>
> How could I get unicorn to rename the pid file back to unicorn.pid ? I
> see some code in method +reap_all_workers+ that suggests it would rename
> the pid file back to its original, but for some reason it doesn't for
> me. Am I missing a step?
Hi Lawrence,
Are you using a config file to specify pid or --pid from the
command-line? The config file should be more reliable. I'll take a
deeper look at it in a bit.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* javascript disappears
@ 2010-08-17 23:38 3% Jimmy Soho
2010-08-18 1:14 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Jimmy Soho @ 2010-08-17 23:38 UTC (permalink / raw)
To: mongrel-unicorn
Hi All,
When I deploy new code I always go through the USR2 - QUIT process.
However, after a few deployments it seems our base javascript file is
missing. I've experienced this a few times now. We use rails 2.3.8,
and have something like this in our layouts file:
<%= javascript_include_tag 'webtoolkit', 'jquery', 'jquery.metadata',
'jquery.url_utils', 'jquery.form', 'jquery.hotkeys',
'jquery.livequery', 'application', :cache => 'base' %>
When it's missing it doesn't matter how often I kill with USR2 / QUIT,
it won't appear. If I stop the unicorn master process completely and
then start again, then it does create the base.js file again.
Anyone experienced this as well? Anyone know what could be the reason for this?
Jimmy
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 3%]
* Re: javascript disappears
2010-08-17 23:38 3% javascript disappears Jimmy Soho
@ 2010-08-18 1:14 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2010-08-18 1:14 UTC (permalink / raw)
To: unicorn list
Jimmy Soho <jimmy.soho@gmail.com> wrote:
> Hi All,
>
> When I deploy new code I always go through the USR2 - QUIT process.
> However, after a few deployments it seems our base javascript file is
> missing. I've experienced this a few times now. We use rails 2.3.8,
> and have something like this in our layouts file:
>
> <%= javascript_include_tag 'webtoolkit', 'jquery', 'jquery.metadata',
> 'jquery.url_utils', 'jquery.form', 'jquery.hotkeys',
> 'jquery.livequery', 'application', :cache => 'base' %>
>
> When it's missing it doesn't matter how often I kill with USR2 / QUIT,
> it won't appear. If I stop the unicorn master process completely and
> then start again, then it does create the base.js file again.
>
> Anyone experienced this as well? Anyone know what could be the reason
> for this?
Hi Jimmy, Are you deploying with Capistrano and you originally started
Unicorn in a working_directory that no longer exists on the filesystem?
Under Linux, you can check the working directory of any process
with `ls -l /proc/$PID/cwd`
That's the only thing I can think of at the moment.
In your other email:
> I get the feeling that memory from the previous master/workers somehow
> isn't released and is used by the new unicorn master and/or worker
> processes.
Shouldn't be possible unless your OS is broken. A more likely
explanation is that Unicorn process is pinned to the working
directory of an old deploy.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: javascript disappears
@ 2010-08-18 13:20 2% ` Jimmy Soho
2010-08-18 23:34 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Jimmy Soho @ 2010-08-18 13:20 UTC (permalink / raw)
To: unicorn list
> Aha! I believe there's an option for Bundler to use the /srv/app/shared
> directory with Capistrano. That's probably your best option.
I am using the shared bundler gems option, I use the example floating
around the internets:
run("cd #{release_path} && bundle install vendor/bundler_gems
--without development test --disable-shared-gems")
> You can also try force the environment variables with the
> before_exec hook in your config file:
>
> before_exec do |server|
> ENV['GEM_HOME'] = '/srv/app/current/vendor/bundler_gems'
> end
After a few hours hacking I've come to the conclusion that's the only
way it can work, for now.
Having delved into bundler I think Bundler.setup needs to change. I'll
file an issue for that with them.
PS. I: for those interested, this is now the unicorn-conf.rb file I'm
using, so I can now happily prune all old releases:
http://gist.github.com/534668
PS. II: While I trying out various things I found that if I modified
the before_exec hook, then deployed that code, send USR2, then it
executes the before_exec hook of the old master, not the new master.
So I then had to send a USR2 a second time to see the new code in
before_exec taking effect. Is that intentional behaviour?
PS. III: Haven't tested it yet, but in case a USR2 fails, and unicorn
reverts back to the old master, does the old master then have the new
environment variable values? Or does it run the before_exec hook again
for the old master?
Jimmy
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: javascript disappears
2010-08-18 13:20 2% ` Jimmy Soho
@ 2010-08-18 23:34 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-08-18 23:34 UTC (permalink / raw)
To: unicorn list
Jimmy Soho <jimmy.soho@gmail.com> wrote:
> PS. II: While I trying out various things I found that if I modified
> the before_exec hook, then deployed that code, send USR2, then it
> executes the before_exec hook of the old master, not the new master.
> So I then had to send a USR2 a second time to see the new code in
> before_exec taking effect. Is that intentional behaviour?
Yes. USR2 doesn't reload the config file before by default. Send a HUP
before USR2 if you want it to reload the config file. Just keep in mind
if you add any new dependencies on your new deploy and HUP, things could
break, too, especially if you're using "preload_app true".
> PS. III: Haven't tested it yet, but in case a USR2 fails, and unicorn
> reverts back to the old master, does the old master then have the new
> environment variable values? Or does it run the before_exec hook again
> for the old master?
The old master doesn't get affected by environment (or any other local)
changes in before_exec. before_exec only runs in the new master
process, so it can only affect the new master process (unless it sends
signals to the old master).
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Unicorn fails to restart gracefully on capistrano deploy
@ 2010-09-15 6:40 2% Eirik Dentz Sinclair
2010-09-15 21:52 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Eirik Dentz Sinclair @ 2010-09-15 6:40 UTC (permalink / raw)
To: mongrel-unicorn
Lawrence, Jamie
Thanks very much for your responses.
The before_exec block in Lawrence's gist was just what was needed. Not sure how many others are using Capistrano and bundler, but seems like that before_exec block would be a good addition to this page: http://unicorn.bogomips.org/Sandbox.html
Thanks again for the help.
-Eirik
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Unicorn fails to restart gracefully on capistrano deploy
2010-09-15 6:40 2% Unicorn fails to restart gracefully on capistrano deploy Eirik Dentz Sinclair
@ 2010-09-15 21:52 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-09-15 21:52 UTC (permalink / raw)
To: unicorn list
Eirik Dentz Sinclair <eirik@efficiency20.com> wrote:
> Lawrence, Jamie
>
> Thanks very much for your responses.
>
> The before_exec block in Lawrence's gist was just what was needed. Not sure how many others are using Capistrano and bundler, but seems like that before_exec block would be a good addition to this page: http://unicorn.bogomips.org/Sandbox.html
>
> Thanks again for the help.
Thanks for confirming the fix. I've updated the Sandbox document
with the following patch.
Patches/pull-requests to the mailing list for documentation are very
much appreciated, especially for Bundler and maybe other things I don't
use myself. Lets try to treat the in-source documentation much like a
wiki, except anybody who grabs the source also has the latest up-to-date
docs ready for offline reading (and readable without a web browser :)
>From 1a75966a5d1a1f6307ed3386e2f91a28bbb72ca0 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Wed, 15 Sep 2010 14:42:54 -0700
Subject: [PATCH] doc: update Sandbox document for Bundler
Thanks to Lawrence Pit, Jamie Wilkinson, and Eirik Dentz Sinclair.
ref: mid.gmane.org/4C8986DA.7090603@gmail.com
ref: mid.gmane.org/5F1A02DB-CBDA-4302-9E26-8050C2D72433@efficiency20.com
---
Sandbox | 9 +++++++++
1 files changed, 9 insertions(+), 0 deletions(-)
diff --git a/Sandbox b/Sandbox
index d2f7590..d101106 100644
--- a/Sandbox
+++ b/Sandbox
@@ -24,6 +24,9 @@ this:
Then use HUP to reload, and then continue with the USR2+QUIT upgrade
sequence.
+Environment variable pollution when exec-ing a new process (with USR2)
+is the primary issue with sandboxing tools such as Bundler and Isolate.
+
== Bundler
=== Running
@@ -42,6 +45,12 @@ This is no longer be an issue as of bundler 0.9.17
ref: http://mid.gmane.org/8FC34B23-5994-41CC-B5AF-7198EF06909E@tramchase.com
+=== Other ENV pollution issues
+
+You may need to set or reset BUNDLE_GEMFILE, GEM_HOME, GEM_PATH and PATH
+environment variables in the before_exec hook as illustrated by
+http://gist.github.com/534668
+
== Isolate
=== Running
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 0%]
* Unicorn doesn't reload the app after the HUP signal
@ 2010-10-02 0:57 2% Rubén Dávila
2010-10-02 8:13 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Rubén Dávila @ 2010-10-02 0:57 UTC (permalink / raw)
To: mongrel-unicorn
Hi folks, I've experimented a problem while deploying my app, I've
sent the HUP signal to the master process, I've checked everything is
ok: new master and workers are spawned and the old ones are
killed(I've checked the PIDs), but the new code deployed isn't
reflected in the living site, so I've to stop and start again unicorn
in order to see the new changes, here is my unicorn config:
rails_env = ENV['RAILS_ENV'] || 'production'
worker_processes (rails_env == 'production' ? 3 : 1)
preload_app true
timeout 30
listen File.expand_path(File.join(__FILE__,
"../../tmp/sockets/unicorn.sock")), :backlog => 2048
if GC.respond_to?(:copy_on_write_friendly=)
GC.copy_on_write_friendly = true
end
before_fork do |server, worker|
old_pid = RAILS_ROOT + '/tmp/pids/unicorn.pid.oldbin'
if File.exists?(old_pid) && server.pid != old_pid
begin
Process.kill("QUIT", File.read(old_pid).to_i)
rescue Errno::ENOENT, Errno::ESRCH
# someone else did our job for us
end
end
end
after_fork do |server, worker|
ActiveRecord::Base.establish_connection
begin
worker.user('peruautos', 'www-data') if Process.euid == 0
rescue => e
if RAILS_ENV == 'development'
STDERR.puts "couldn't change user, oh well"
else
raise e
end
end
end
I'm using unicorn 1.1.2 and rack 1.0.1
Regards.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Unicorn doesn't reload the app after the HUP signal
2010-10-02 0:57 2% Unicorn doesn't reload the app after the HUP signal Rubén Dávila
@ 2010-10-02 8:13 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-10-02 8:13 UTC (permalink / raw)
To: unicorn list
Rubén Dávila <ruben.grunge84@gmail.com> wrote:
> Hi folks, I've experimented a problem while deploying my app, I've
> sent the HUP signal to the master process, I've checked everything is
> ok: new master and workers are spawned and the old ones are
> killed(I've checked the PIDs), but the new code deployed isn't
> reflected in the living site, so I've to stop and start again unicorn
> in order to see the new changes, here is my unicorn config:
>
> rails_env = ENV['RAILS_ENV'] || 'production'
> worker_processes (rails_env == 'production' ? 3 : 1)
> preload_app true
> timeout 30
> listen File.expand_path(File.join(__FILE__,
> "../../tmp/sockets/unicorn.sock")), :backlog => 2048
Hi Rubén,
Are you using cap or something else that makes symlinks during
deployment? It could've been started in an old release directory while
it didn't have the "current" symlink pointing to it. You can work
around this by putting:
working_directory "/path/to/app/current"
in your config file.
<snip>
> I'm using unicorn 1.1.2 and rack 1.0.1
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* [PATCH] avoid unlinking actively listening sockets
@ 2010-10-04 4:22 4% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2010-10-04 4:22 UTC (permalink / raw)
To: unicorn list
While we've always unlinked dead sockets from nuked/leftover
processes, blindly unlinking them can cause unnecessary failures
when an active process is already listening on them. We now
make a simple connect(2) check to ensure the socket is not in
use before unlinking it.
Thanks to Jordan Ritter for the detailed bug report leading to
this fix.
ref: http://mid.gmane.org/8D95A44B-A098-43BE-B532-7D74BD957F31@darkridge.com
---
Eric Wong <normalperson@yhbt.net> wrote:
> A simpler check would be to use connect(2) (but not make any HTTP request)
> to see if the socket is alive. Patch coming.
s/simpler/better/
Also pushed out to "master". I guess a 1.1.4 release with this fix
only is on the way since there isn't much else to release, yet.
lib/unicorn/socket_helper.rb | 10 ++++-
t/t0011-active-unix-socket.sh | 79 +++++++++++++++++++++++++++++++++++++++
test/unit/test_socket_helper.rb | 9 ++++-
3 files changed, 95 insertions(+), 3 deletions(-)
create mode 100644 t/t0011-active-unix-socket.sh
diff --git a/lib/unicorn/socket_helper.rb b/lib/unicorn/socket_helper.rb
index 9a155e1..1d03eab 100644
--- a/lib/unicorn/socket_helper.rb
+++ b/lib/unicorn/socket_helper.rb
@@ -111,8 +111,14 @@ module Unicorn
sock = if address[0] == ?/
if File.exist?(address)
if File.socket?(address)
- logger.info "unlinking existing socket=#{address}"
- File.unlink(address)
+ begin
+ UNIXSocket.new(address).close
+ # fall through, try to bind(2) and fail with EADDRINUSE
+ # (or succeed from a small race condition we can't sanely avoid).
+ rescue Errno::ECONNREFUSED
+ logger.info "unlinking existing socket=#{address}"
+ File.unlink(address)
+ end
else
raise ArgumentError,
"socket=#{address} specified but it is not a socket!"
diff --git a/t/t0011-active-unix-socket.sh b/t/t0011-active-unix-socket.sh
new file mode 100644
index 0000000..6f9ac53
--- /dev/null
+++ b/t/t0011-active-unix-socket.sh
@@ -0,0 +1,79 @@
+#!/bin/sh
+. ./test-lib.sh
+t_plan 11 "existing UNIX domain socket check"
+
+read_pid_unix () {
+ x=$(printf 'GET / HTTP/1.0\r\n\r\n' | \
+ socat - UNIX:$unix_socket | \
+ tail -1)
+ test -n "$x"
+ y="$(expr "$x" : '\([0-9]\+\)')"
+ test x"$x" = x"$y"
+ test -n "$y"
+ echo "$y"
+}
+
+t_begin "setup and start" && {
+ rtmpfiles unix_socket unix_config
+ rm -f $unix_socket
+ unicorn_setup
+ grep -v ^listen < $unicorn_config > $unix_config
+ echo "listen '$unix_socket'" >> $unix_config
+ unicorn -D -c $unix_config pid.ru
+ unicorn_wait_start
+ orig_master_pid=$unicorn_pid
+}
+
+t_begin "get pid of worker" && {
+ worker_pid=$(read_pid_unix)
+ t_info "worker_pid=$worker_pid"
+}
+
+t_begin "fails to start with existing pid file" && {
+ rm -f $ok
+ unicorn -D -c $unix_config pid.ru || echo ok > $ok
+ test x"$(cat $ok)" = xok
+}
+
+t_begin "worker pid unchanged" && {
+ test x"$(read_pid_unix)" = x$worker_pid
+ > $r_err
+}
+
+t_begin "fails to start with listening UNIX domain socket bound" && {
+ rm $ok $pid
+ unicorn -D -c $unix_config pid.ru || echo ok > $ok
+ test x"$(cat $ok)" = xok
+ > $r_err
+}
+
+t_begin "worker pid unchanged (again)" && {
+ test x"$(read_pid_unix)" = x$worker_pid
+}
+
+t_begin "nuking the existing Unicorn succeeds" && {
+ kill -9 $unicorn_pid $worker_pid
+ while kill -0 $unicorn_pid
+ do
+ sleep 1
+ done
+ check_stderr
+}
+
+t_begin "succeeds in starting with leftover UNIX domain socket bound" && {
+ test -S $unix_socket
+ unicorn -D -c $unix_config pid.ru
+ unicorn_wait_start
+}
+
+t_begin "worker pid changed" && {
+ test x"$(read_pid_unix)" != x$worker_pid
+}
+
+t_begin "killing succeeds" && {
+ kill $unicorn_pid
+}
+
+t_begin "no errors" && check_stderr
+
+t_done
diff --git a/test/unit/test_socket_helper.rb b/test/unit/test_socket_helper.rb
index bbce359..c6d0d42 100644
--- a/test/unit/test_socket_helper.rb
+++ b/test/unit/test_socket_helper.rb
@@ -101,7 +101,14 @@ class TestSocketHelper < Test::Unit::TestCase
def test_bind_listen_unix_rebind
test_bind_listen_unix
- new_listener = bind_listen(@unix_listener_path)
+ new_listener = nil
+ assert_raises(Errno::EADDRINUSE) do
+ new_listener = bind_listen(@unix_listener_path)
+ end
+ assert_nothing_raised do
+ File.unlink(@unix_listener_path)
+ new_listener = bind_listen(@unix_listener_path)
+ end
assert UNIXServer === new_listener
assert new_listener.fileno != @unix_listener.fileno
assert_equal sock_name(new_listener), sock_name(@unix_listener)
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 4%]
* Re: [ANN] unicorn 3.0.0 - disable rewindable input!
@ 2010-11-22 1:21 2% ` Jeremy Evans
0 siblings, 0 replies; 200+ results
From: Jeremy Evans @ 2010-11-22 1:21 UTC (permalink / raw)
To: unicorn list
On Sun, Nov 21, 2010 at 3:49 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Jeremy Evans <jeremyevans0@gmail.com> wrote:
>> On Sat, Nov 20, 2010 at 8:05 PM, Eric Wong <normalperson@yhbt.net> wrote:
>> > Michael Guterl <mguterl@gmail.com> wrote:
>> >> We've been using Unicorn 1.x very successfully for some time with a
>> >> Rails 2.3 application. Would you recommend upgrading to Unicorn 3.x?
>> >
>> > Yes if you're running a recent Linux (>= 2.6.28), it might help somewhat
>> > with performance (but may not be noticeable with a particular app).
>> > I haven't done extensive benchmarking, but the reduction of syscalls
>> > should help a bit, and newer versions will exploit some of that more.
> <snip>
>> > If you're willing to help iron out portability bugs to other kernels,
>> > then please upgrade and report back :)
>>
>> Just an anecdotal data point here, using OpenBSD-current. Moving from
>> Unicorn 1.0 to 2.0 sped up the scaffolding_extensions integration test
>> suite by a factor of 3 (~10 seconds to ~3 seconds). I'm guessing this
>> is due mainly to the use of kgio. Hoping to upgrade to Unicorn 3.0
>> today.
>
> Wow! I didn't expect that kind of performance improvement![1] What
> is this test doing? I'm actually suspicious there could be a bug
> somewhere in Unicorn or kgio causing wrong results :)
>
> Let us know how 3.0 goes, thanks!
>
> [1] - dalli reported 10-20%, which is closer to what I expected
> (1-10%).
Looks like it must have been some other change that caused this speed
up, as after testing again with 1.x, I'm getting about the same in
terms of performance. Sorry for the misinformation.
Jeremy
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Unicorn and HAProxy, 500 Internal errors after checks
@ 2010-12-01 17:14 2% ` Pierre
0 siblings, 0 replies; 200+ results
From: Pierre @ 2010-12-01 17:14 UTC (permalink / raw)
To: unicorn list
Dear Eric,
On Wed, Dec 1, 2010 at 5:52 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Hi Pierre, HAProxy should be configured to send proper HTTP checks and
> not just TCP connection checks, the problem will go away then.
I understood this could be fixed this way and we will probably do that
soon. However, I think this is also the responsibility of Unicorn not
to reply anything when there's no request or at least log the error
somewhere :)
> Also, I
> can not recommend HAProxy unless you're certain all your clients are on
> a LAN and can be trusted to never trickle uploads nor reading large
> responses.
While I understand that uploads are very complicated to handle on the
stack (even nginx can be confused at upload sometimes), HAProxy proved
it was very good at managing tons of connections and high volume
traffic from the Internet. All the more so as it allows a very high
level of redundancy at a very small cost that cannot be achieved
simply otherwise. Do you have any pointers about your worrying
non-recommendation of HAProxy ?
As far as I'm concerned, I've used HAProxy in worldwide web context of
website serving up to 25 Millions pages with no problem, ever.
Thanks again for your support, :)
--
Pierre <oct@fotopedia.com>
Server Shepherd at http://www.fotopedia.com/
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* license is NOT changing
@ 2011-02-16 19:11 2% Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2011-02-16 19:11 UTC (permalink / raw)
To: mongrel-unicorn
Since Ruby 1.9.3dev is under the 2-clause BSD License (+ extra terms)
now, I've clarified Unicorn will be remaining under the same terms as
Ruby 1.8 and Mongrel 1.1.5[1].
I have zero interest (nor rights to relicense) the code in Unicorn to a
BSD license.
For the record, I'm personally a huge fan and proponent of the GPL,
but again I can't relicense existing code without the permission
of everybody that ever contributed to Mongrel/Unicorn, either.
While I am more than aware there are several reasons people aren't
comfortable with Unicorn, but the GPL should NOT be one of those
reasons.
Thanks for reading.
[1] avoiding confusion with Mongrel2 which is also under a BSD license
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: [RFC/PATCH] Bundler/Sandbox documentation updates
@ 2011-03-09 10:39 2% ` Lawrence Pit
2011-03-10 0:52 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Lawrence Pit @ 2011-03-09 10:39 UTC (permalink / raw)
To: unicorn list
Hi Eric,
I tested this out:
- in the before_exec and after_fork I log PATH GEM_HOME GEM_PATH
BUNDLE_GEMFILE for inspection after deployment
- deploy latest release, make sure to shutdown unicorn, then start cleanly
- prune all releases, except the last one
- deploy latest release again (which only sets the BUNDLE_GEMFILE env var)
I've tried this a few times, and everytime I get something like this:
https://gist.github.com/331b0decab62fd58483c
If I revert back to setting the GEM_HOME, GEM_PATH and PATH in the
before_exec it works fine.
PS. When deploying I use USR2 followed by QUIT
Cheers,
Lawrence
> I started playing around with Bundler 1.0.10 today and noticed it's
> quite different than previous versions (based on my limited experiences)
> and the out-of-the-box experience is pretty good regarding (lack of)
> ENV pollution.
>
> Can any more experienced Bundler (and possibly Capistrano) users
> comment on the below changes and see if they make sense?
>
> Thanks in advance!
>
> diff --git a/Sandbox b/Sandbox
> index d101106..46dfb91 100644
> --- a/Sandbox
> +++ b/Sandbox
> @@ -45,11 +45,20 @@ This is no longer be an issue as of bundler 0.9.17
>
> ref: http://mid.gmane.org/8FC34B23-5994-41CC-B5AF-7198EF06909E@tramchase.com
>
> +=== BUNDLE_GEMFILE for Capistrano users
> +
> +You may need to set or reset the BUNDLE_GEMFILE environment variable in
> +the before_exec hook:
> +
> + before_exec do |server|
> + ENV["BUNDLE_GEMFILE"] = "/path/to/app/current/Gemfile"
> + end
> +
> === Other ENV pollution issues
>
> -You may need to set or reset BUNDLE_GEMFILE, GEM_HOME, GEM_PATH and PATH
> -environment variables in the before_exec hook as illustrated by
> -http://gist.github.com/534668
> +If you're using an older Bundler version (0.9.x), you may need to set or
> +reset GEM_HOME, GEM_PATH and PATH environment variables in the
> +before_exec hook as illustrated by http://gist.github.com/534668
>
> == Isolate
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: [RFC/PATCH] Bundler/Sandbox documentation updates
2011-03-09 10:39 2% ` Lawrence Pit
@ 2011-03-10 0:52 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2011-03-10 0:52 UTC (permalink / raw)
To: unicorn list
Lawrence Pit <lawrence.pit@gmail.com> wrote:
> Hi Eric,
>
> I tested this out:
Thanks for taking your time to test.
> - in the before_exec and after_fork I log PATH GEM_HOME GEM_PATH
> BUNDLE_GEMFILE for inspection after deployment
>
> - deploy latest release, make sure to shutdown unicorn, then start cleanly
>
> - prune all releases, except the last one
>
> - deploy latest release again (which only sets the BUNDLE_GEMFILE env var)
>
> I've tried this a few times, and everytime I get something like this:
>
> https://gist.github.com/331b0decab62fd58483c
Yikes, it looks like the UNICORN_FD env is getting clobbered somehow.
Not sure what could be causing that.
> If I revert back to setting the GEM_HOME, GEM_PATH and PATH in the
> before_exec it works fine.
Which Bundler version is this with?
> PS. When deploying I use USR2 followed by QUIT
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Unicorn error
@ 2011-04-07 21:11 1% Ernesto Rocha
2011-04-07 22:04 0% ` Clifton King
0 siblings, 1 reply; 200+ results
From: Ernesto Rocha @ 2011-04-07 21:11 UTC (permalink / raw)
To: mongrel-unicorn
Hi guys,
These days I'm experimenting a strange behavior from Unicorn. Now the issue is,
when I try to access a page it's always times out, so, on browser I see:
Nginx - "bad gateway - timed out" page error
When i start unicorn with -d parameter I only see:
{:daemonize=>false,
:app=>
#<Proc:0x00007feee244d2d8@/usr/local/rvm/gems/ruby-1.8.7-
p302@project/gems/unicorn-3.5.0/bin/unicorn_rails:135>,
:unicorn_options=>
{:config_file=>"/home/user/project/current/config/unicorn.rb",
:listeners=>[]}}
Exception `Errno::EEXIST' at /usr/local/rvm/rubies/ruby-1.8.7-
p302/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/cache
Exception `Errno::EEXIST' at /usr/local/rvm/rubies/ruby-1.8.7-
p302/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/pids
Exception `Errno::EEXIST' at /usr/local/rvm/rubies/ruby-1.8.7-
p302/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/sessions
Exception `Errno::EEXIST' at /usr/local/rvm/rubies/ruby-1.8.7-
p302/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/sockets
And at unicorn.stderr.log which is about 2gb now with these lines repeated over
and over again:
worker=0 spawning...
worker=1 spawning...
worker=0 spawned pid=30557
/usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/util.rb:19:in `chown': Operation not permitted -
/home/user/project/shared/log/unicorn.stderr.log (Errno::EPERM)
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/util.rb:19:in `chown_logs'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/util.rb:18:in `each_object'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/util.rb:18:in `chown_logs'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/worker.rb:31:in `user'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/http_server.rb:561:in `init_worker_process'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/http_server.rb:578:in `worker_loop'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/http_server.rb:495:in `spawn_missing_workers'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/http_server.rb:492:in `fork'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/http_server.rb:492:in `spawn_missing_workers'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/http_server.rb:488:in `each'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/http_server.rb:488:in `spawn_missing_workers'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/http_server.rb:502:in `maintain_worker_count'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/http_server.rb:161:in `start'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn.rb:13:in `run'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/bin/unicorn_rails:208
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/bin/unicorn_rails:19:in
`load'
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/bin/unicorn_rails:19
master process ready
worker=1 spawned pid=30558
/usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/util.rb:19:in `chown': Operation not permitted -
/home/user/project/shared/log/unicorn.stderr.log (Errno::EPERM)
.... and on....
$ rails -v
Rails 3.0.6
Using rack (1.2.2)
$unicorn -v
unicorn v3.5.0
Ubuntu 10.10
$ nginx -v
nginx version: nginx/0.7.67
I don't know what more information you may need. But, that's it. I don't have
any idea to fix this. Any thought ?
Thansk,
Ernesto
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 1%]
* Re: Unicorn error
2011-04-07 21:11 1% Unicorn error Ernesto Rocha
@ 2011-04-07 22:04 0% ` Clifton King
0 siblings, 0 replies; 200+ results
From: Clifton King @ 2011-04-07 22:04 UTC (permalink / raw)
To: unicorn list
Are permissions set correctly in all of your application folders?
/usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
3.5.0/lib/unicorn/util.rb:19:in `chown': Operation not permitted -
/home/user/project/shared/log/unicorn.stderr.log (Errno::EPERM)
from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
Looks to me you may need to do a chown -r app_user /app_directory/
where app_user is the user unicorn is running on. Does unicorn work
with sudo?
On Thu, Apr 7, 2011 at 4:11 PM, Ernesto Rocha <ernestorrocha@gmail.com> wrote:
>
> Hi guys,
>
> These days I'm experimenting a strange behavior from Unicorn. Now the issue is,
> when I try to access a page it's always times out, so, on browser I see:
>
> Nginx - "bad gateway - timed out" page error
>
> When i start unicorn with -d parameter I only see:
>
>
> {:daemonize=>false,
> :app=>
> #<Proc:0x00007feee244d2d8@/usr/local/rvm/gems/ruby-1.8.7-
> p302@project/gems/unicorn-3.5.0/bin/unicorn_rails:135>,
> :unicorn_options=>
> {:config_file=>"/home/user/project/current/config/unicorn.rb",
> :listeners=>[]}}
> Exception `Errno::EEXIST' at /usr/local/rvm/rubies/ruby-1.8.7-
> p302/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/cache
> Exception `Errno::EEXIST' at /usr/local/rvm/rubies/ruby-1.8.7-
> p302/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/pids
> Exception `Errno::EEXIST' at /usr/local/rvm/rubies/ruby-1.8.7-
> p302/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/sessions
> Exception `Errno::EEXIST' at /usr/local/rvm/rubies/ruby-1.8.7-
> p302/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/sockets
>
>
> And at unicorn.stderr.log which is about 2gb now with these lines repeated over
> and over again:
>
> worker=0 spawning...
> worker=1 spawning...
> worker=0 spawned pid=30557
> /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/util.rb:19:in `chown': Operation not permitted -
> /home/user/project/shared/log/unicorn.stderr.log (Errno::EPERM)
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/util.rb:19:in `chown_logs'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/util.rb:18:in `each_object'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/util.rb:18:in `chown_logs'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/worker.rb:31:in `user'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/http_server.rb:561:in `init_worker_process'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/http_server.rb:578:in `worker_loop'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/http_server.rb:495:in `spawn_missing_workers'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/http_server.rb:492:in `fork'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/http_server.rb:492:in `spawn_missing_workers'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/http_server.rb:488:in `each'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/http_server.rb:488:in `spawn_missing_workers'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/http_server.rb:502:in `maintain_worker_count'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/http_server.rb:161:in `start'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn.rb:13:in `run'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/bin/unicorn_rails:208
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/bin/unicorn_rails:19:in
> `load'
> from /usr/local/rvm/gems/ruby-1.8.7-p302@project/bin/unicorn_rails:19
> master process ready
> worker=1 spawned pid=30558
> /usr/local/rvm/gems/ruby-1.8.7-p302@project/gems/unicorn-
> 3.5.0/lib/unicorn/util.rb:19:in `chown': Operation not permitted -
> /home/user/project/shared/log/unicorn.stderr.log (Errno::EPERM)
> .... and on....
>
> $ rails -v
> Rails 3.0.6
>
> Using rack (1.2.2)
>
> $unicorn -v
> unicorn v3.5.0
>
> Ubuntu 10.10
>
> $ nginx -v
> nginx version: nginx/0.7.67
>
> I don't know what more information you may need. But, that's it. I don't have
> any idea to fix this. Any thought ?
>
> Thansk,
> Ernesto
>
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: leading/trailing linear whitespace in headers
@ 2011-05-21 5:57 2% ` Eric Wong
2011-05-27 7:32 0% ` Lawrence Pit
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2011-05-21 5:57 UTC (permalink / raw)
To: unicorn list
Lawrence Pit <lawrence.pit@gmail.com> wrote:
> Hi Eric,
>
> Since I upgraded from unicorn 1 to 3, Unicorn will freeze for 60
> seconds at random moments, and then restart. Only on my dev machine.
> Not in 1 project, but in all my projects that use unicorn. I get no
> log output, it just freezes for 60 seconds. Strangely this happens
> significantly more often with using Chrome as a client than with
> Firefox.
Even through nginx? Which Ruby version is it?
The bug I'm referring to in this thread is only for /request/ headers,
and I don't think any common HTTP clients send trailing whitespace.
> So I hope this will fix that, though I have no idea if it's related
> in any way. I was thinking it may also have to do with unicorn not
> officially supporting Mac OS X.
I'm sure you're not the only OS X Unicorn user, perhaps somebody else
knows what's going on...
If it's kgio-related, but I think Jeremy fixed all the issues under
OpenBSD with kgio 2.4.0 and some of that probably helped other
*BSD-based systems. I've been meaning to setup a VM on FreeBSD
again but haven't had the time.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: leading/trailing linear whitespace in headers
2011-05-21 5:57 2% ` Eric Wong
@ 2011-05-27 7:32 0% ` Lawrence Pit
2011-05-30 7:02 2% ` Lawrence Pit
0 siblings, 1 reply; 200+ results
From: Lawrence Pit @ 2011-05-27 7:32 UTC (permalink / raw)
To: unicorn list
Hi Eric,
I thought I'll wait a few days before replying just to make sure:
believe it or not, since I upgraded 4 days ago to kgio 2.4.1 not once
did I run into this issue. While before that unicorn would timeout at
least once every hour, sometimes a dozen times in a row.
I don't run through nginx on my dev machine, just straight into
unicorn_rails using mostly Chrome 11 and FF 4. I use:
ruby 1.8.7 (2010-04-19 patchlevel 253) [i686-darwin10.4.0], MBARI
0x6770, Ruby Enterprise Edition 2010.02
Cheers,
Lawrence
> Hi Eric,
>> Since I upgraded from unicorn 1 to 3, Unicorn will freeze for 60
>> seconds at random moments, and then restart. Only on my dev machine.
>> Not in 1 project, but in all my projects that use unicorn. I get no
>> log output, it just freezes for 60 seconds. Strangely this happens
>> significantly more often with using Chrome as a client than with
>> Firefox.
> Even through nginx? Which Ruby version is it?
>
> The bug I'm referring to in this thread is only for /request/ headers,
> and I don't think any common HTTP clients send trailing whitespace.
>
>> So I hope this will fix that, though I have no idea if it's related
>> in any way. I was thinking it may also have to do with unicorn not
>> officially supporting Mac OS X.
> I'm sure you're not the only OS X Unicorn user, perhaps somebody else
> knows what's going on...
>
> If it's kgio-related, but I think Jeremy fixed all the issues under
> OpenBSD with kgio 2.4.0 and some of that probably helped other
> *BSD-based systems. I've been meaning to setup a VM on FreeBSD
> again but haven't had the time.
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: leading/trailing linear whitespace in headers
2011-05-27 7:32 0% ` Lawrence Pit
@ 2011-05-30 7:02 2% ` Lawrence Pit
2011-05-31 4:49 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Lawrence Pit @ 2011-05-30 7:02 UTC (permalink / raw)
To: unicorn list
Cheered too early after all, just got one again:
E, [2011-05-30T16:59:36.005000 #7351] ERROR -- : worker=0 PID:7352
timeout (60.004931s > 60s), killing
E, [2011-05-30T16:59:36.058276 #7351] ERROR -- : reaped
#<Process::Status: pid=7352,signaled(SIGKILL=9)> worker=0
I, [2011-05-30T16:59:36.059122 #7351] INFO -- : worker=0 spawning...
I, [2011-05-30T16:59:36.062990 #8505] INFO -- : worker=0 spawned pid=8505
I, [2011-05-30T16:59:36.063395 #8505] INFO -- : Refreshing Gem list
=> ruby 1.8.7 (i686-darwin10.4.0) [2010-04-19] Ruby Enterprise Edition
worker=0 ready
Cheers,
Lawrence
> Hi Eric,
>
> I thought I'll wait a few days before replying just to make sure:
> believe it or not, since I upgraded 4 days ago to kgio 2.4.1 not once
> did I run into this issue. While before that unicorn would timeout at
> least once every hour, sometimes a dozen times in a row.
>
> I don't run through nginx on my dev machine, just straight into
> unicorn_rails using mostly Chrome 11 and FF 4. I use:
>
> ruby 1.8.7 (2010-04-19 patchlevel 253) [i686-darwin10.4.0], MBARI
> 0x6770, Ruby Enterprise Edition 2010.02
>
>
>
> Cheers,
> Lawrence
>
>> Hi Eric,
>>> Since I upgraded from unicorn 1 to 3, Unicorn will freeze for 60
>>> seconds at random moments, and then restart. Only on my dev machine.
>>> Not in 1 project, but in all my projects that use unicorn. I get no
>>> log output, it just freezes for 60 seconds. Strangely this happens
>>> significantly more often with using Chrome as a client than with
>>> Firefox.
>> Even through nginx? Which Ruby version is it?
>>
>> The bug I'm referring to in this thread is only for /request/ headers,
>> and I don't think any common HTTP clients send trailing whitespace.
>>
>>> So I hope this will fix that, though I have no idea if it's related
>>> in any way. I was thinking it may also have to do with unicorn not
>>> officially supporting Mac OS X.
>> I'm sure you're not the only OS X Unicorn user, perhaps somebody else
>> knows what's going on...
>>
>> If it's kgio-related, but I think Jeremy fixed all the issues under
>> OpenBSD with kgio 2.4.0 and some of that probably helped other
>> *BSD-based systems. I've been meaning to setup a VM on FreeBSD
>> again but haven't had the time.
>>
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: leading/trailing linear whitespace in headers
2011-05-30 7:02 2% ` Lawrence Pit
@ 2011-05-31 4:49 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2011-05-31 4:49 UTC (permalink / raw)
To: Lawrence Pit; +Cc: unicorn list
Lawrence Pit <lawrence.pit@gmail.com> wrote:
> Cheered too early after all, just got one again:
Can you ever reproduce this with nginx in front? Does anybody else
on any *BSD platform get this?
Btw, please don't waste my bandwidth top posting, I'd rather people
not quote at all than top post, thanks!
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: workers not utilizing multiple CPUs
[not found] ` <BANLkTikC5D+0pUDRDgvQbzu=dpwmdKNY=A@mail.gmail.com>
@ 2011-06-01 6:51 2% ` Nate Clark
0 siblings, 0 replies; 200+ results
From: Nate Clark @ 2011-06-01 6:51 UTC (permalink / raw)
To: unicorn list
Thanks for the responses, all.
Eric, you were right, our load was not enough. We had just started on
load testing our app, and I think we started with too many app servers
and not enough load. Once we cranked up the load and used fewer
instances, we're now definitely seeing all CPU cores being utilized. I
was not aware that the kernel would optimize like you described.
Once we did start seeing heavier load, our collectd data and htop were
reporting usage on the virtual cores correctly.
Thanks again, very happy with the results so far,
Nate
On Wed, Jun 1, 2011 at 2:35 PM, Nate Clark <nate@pivotallabs.com> wrote:
>
> Thanks for the responses, all.
> Eric, you were right, our load was not enough. We had just started on load testing our app, and I think we started with too many app servers and not enough load. Once we cranked up the load and used fewer instances, we're now definitely seeing all CPU cores being utilized. I was not aware that the kernel would optimize like you described.
> Once we did start seeing heavier load, our collectd data and htop were reporting usage on the virtual cores correctly.
> Thanks again, very happy with the results so far,
> Nate
> On Tue, May 31, 2011 at 11:55 PM, Clifton King <cliftonk@gmail.com> wrote:
>>
>> Thanks Eric, I had expected that to be the case (we are under light
>> load as of now).
>>
>> On Tue, May 31, 2011 at 10:48 AM, Eric Wong <normalperson@yhbt.net> wrote:
>> > Clifton King <cliftonk@gmail.com> wrote:
>> >> We experience the same problem. I believe the problem has more to do
>> >> with the kernel CPU scheduler than anything else. If you figure put a
>> >> reliable way to spread the load, I'd like to hear it.
>> >
>> > Load not being spread is /not/ a problem unless there are requests that
>> > get stuck in the listen queue.
>> >
>> > If no requests are actually stuck in the queue (light load), the kernel
>> > is right to put requests into the most recently used worker since it can
>> > get better CPU cache behavior this way.
>> >
>> >
>> > == The real problem
>> >
>> > Under high loads (many cores, fast responses), Unicorn currently uses
>> > more resources because of non-blocking accept() + select(). This isn't
>> > a noticeable problem for most machines (1-16 cores).
>> >
>> > Future versions of Unicorn may take advantage of /blocking/ accept()
>> > optimizations under Linux. Rainbows! already lets you take advantage
>> > of this behavior if you meet the following requirements:
>> >
>> > * Ruby 1.9.x under Linux
>> > * only one listen socket (if worker_connections == 1 under Rainbows!)
>> > * use ThreadPool|XEpollThreadPool|XEpollThreadSpawn|XEpoll
>> >
>> > I haven't had a chance to benchmark any of this on very big machines so
>> > I have no idea how well it actually works compared to Unicorn, only how
>> > well it works in theory :)
>> >
>> >
>> > Blocking accept() under Ruby 1.9.x + Linux should distribute load evenly
>> > across workers in all situations, even in the non-busy cases where load
>> > distribution doesn't matter (your case :).
>> >
>> > [1] - http://rainbows.rubyforge.org/Rainbows/XEpollThreadPool.html
>> >
>> > --
>> > Eric Wong
>> > _______________________________________________
>> > Unicorn mailing list - mongrel-unicorn@rubyforge.org
>> > http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>> > Do not quote signatures (like this one) or top post when replying
>> >
>> _______________________________________________
>> Unicorn mailing list - mongrel-unicorn@rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: problem setting multiple cookies
@ 2011-06-21 0:25 2% ` Jason Su
0 siblings, 1 reply; 200+ results
From: Jason Su @ 2011-06-21 0:25 UTC (permalink / raw)
To: unicorn list
Hey Eric,
I'm using Rails 2.3.8, and Rack 1.1.2
I'm setting cookies like this:
cookies[:user_id] = { :domain => ".#{domain}", :value => "#{user.id}",
:expires => 10.years.from_now }
How can I get the response to show up correctly, with individual
Set-Cookie lines for each?
Thanks again,
Jason
On Mon, Jun 20, 2011 at 5:06 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Jason Su <jason@lookbook.nu> wrote:
>> Hey guys,
>>
>> When I try to set multiple cookies in one request, the 'Set-Cookies'
>> header in the http response doesn't look right -- all the cookies are
>> set without any line breaks.
>>
>> Looks like this:
>>
>> Set-Cookie:first=; domain=.domain.com; path=/; expires=Thu,
>> 01-Jan-1970 00:00:00 GMTsecond=1; domain=.domain.com; path=/;
>> expires=Sun, 20-Jun-2021 23:34:30 GMT
>
> Which versions of Rails and Rack are you using? How is your app
> setting cookies?
>
> Older versions of Rack may have specified a different delimiter (or an
> Array). Current versions only use "\n" inside a string, so it should be
> something like:
>
> headers["Set-Cookie"] = "first=33; path=/\n" \
> "second=; path=/\n" \
> "third=; path=/"
>
> Using Rack::Utils.set_cookie_header! (or a Rails-level wrapper) is
> recommended, though.
>
>> Should look more like this, which is what I got from a fresh Rails project test:
>>
>> Set-Cookie:first=33; path=/
>> second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
>> third=69; path=/
>
> The response should have 3 individual Set-Cookie lines:
>
> Set-Cookie: first=33; path=/
> Set-Cookie: second=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
> Set-Cookie: third=69; path=/
>
>> In the first example, neither cookies (first/second) are set. The
>> second example, all 3 cookies are set.
>>
>> I think it's happening somewhere here, but I can't figure out what's
>> going on... http://bogomips.org/unicorn.git/tree/lib/unicorn/cgi_wrapper.rb
>
> The code that does this transformation of the Rack header to the
> client socket is here:
>
> http://bogomips.org/unicorn.git/tree/lib/unicorn/http_response.rb
>
> cgi_wrapper.rb is only for Rails <= 2.2, where it is called before
> http_response.rb
>
> --
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: problem setting multiple cookies
@ 2011-06-21 1:35 2% ` Jason Su
1 sibling, 0 replies; 200+ results
From: Jason Su @ 2011-06-21 1:35 UTC (permalink / raw)
To: unicorn list
Sorry to post again, but the response headers were captured after I
cleared my cookies, so it's not consistent with the first output.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: problem setting multiple cookies
@ 2011-06-21 2:48 2% ` Jason Su
2011-06-21 3:15 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Jason Su @ 2011-06-21 2:48 UTC (permalink / raw)
To: unicorn list
Hey Eric,
I called p headers in JoinCookie before manipulating the headers at
all, so what you saw was probably already set correctly by rails. I'm
not setting the _test_session cookie myself.
Here's what I get in the response, with the _test_session cookie line:
Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
21-Jun-2021 02:40:27 GMT
_test_session=40658e37c9e8b39c5f8389e09682c47b; domain=.test.com;
path=/; HttpOnly
It doesn't add a second Set-Cookie: line, but the line break seems to work...
2.3.11 is giving me a ton of trouble.. I've tried it before.. and I
just tried again with little luck... Is there anything that I can just
grab from the commits to make it work on 2.3.8?
Thanks,
Jason
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: problem setting multiple cookies
2011-06-21 2:48 2% ` Jason Su
@ 2011-06-21 3:15 0% ` Eric Wong
2011-06-21 3:38 2% ` Jason Su
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2011-06-21 3:15 UTC (permalink / raw)
To: unicorn list
Jason Su <jason@lookbook.nu> wrote:
> Here's what I get in the response, with the _test_session cookie line:
>
> Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
> 00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
> 21-Jun-2021 02:40:27 GMT
> _test_session=40658e37c9e8b39c5f8389e09682c47b; domain=.test.com;
> path=/; HttpOnly
>
> It doesn't add a second Set-Cookie: line, but the line break seems to work...
Huh? Which version of unicorn are you using? The /\n/ splitting in
headers has been in the earliest releases of unicorn...
> 2.3.11 is giving me a ton of trouble.. I've tried it before.. and I
> just tried again with little luck... Is there anything that I can just
> grab from the commits to make it work on 2.3.8?
Look at the individual commits in rails.git. I just used
git log -p v2.3.8..v2.3.11 -- actionpack
in rails.git to find those for you. I just looked for things that
would manipulate cookies/headers.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: problem setting multiple cookies
2011-06-21 3:15 0% ` Eric Wong
@ 2011-06-21 3:38 2% ` Jason Su
2011-06-21 18:29 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Jason Su @ 2011-06-21 3:38 UTC (permalink / raw)
To: unicorn list
I'm using unicorn 3.6.2
What I was trying to say is that the JoinCookie middleware didn't do
anything. headers["Set-Cookie"] is a string, and the _test_session
cookie was always being set correctly. The problem happens only when I
try to set multiple cookies myself.
Here's the output of p headers in JoinCookie again
{"X-Runtime"=>"92", "Content-Type"=>"text/html",
"Content-Length"=>"1", "Set-Cookie"=>"first=; domain=.test.com;
path=/; expires=Thu, 01-Jan-1970 00:00:00 GMTsecond=1;
domain=.test.com; path=/; expires=Mon, 21-Jun-2021 03:35:30
GMT\n_test_session=1804126e6bad8cf4b25602b74218f003; domain=.test.com;
path=/; HttpOnly", "Cache-Control"=>"no-cache"}
Where is JoinCookie is getting it's headers from?
On Mon, Jun 20, 2011 at 8:15 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Jason Su <jason@lookbook.nu> wrote:
>> Here's what I get in the response, with the _test_session cookie line:
>>
>> Set-Cookie:first=; domain=.test.com; path=/; expires=Thu, 01-Jan-1970
>> 00:00:00 GMTsecond=1; domain=.test.com; path=/; expires=Mon,
>> 21-Jun-2021 02:40:27 GMT
>> _test_session=40658e37c9e8b39c5f8389e09682c47b; domain=.test.com;
>> path=/; HttpOnly
>>
>> It doesn't add a second Set-Cookie: line, but the line break seems to work...
>
> Huh? Which version of unicorn are you using? The /\n/ splitting in
> headers has been in the earliest releases of unicorn...
>
>> 2.3.11 is giving me a ton of trouble.. I've tried it before.. and I
>> just tried again with little luck... Is there anything that I can just
>> grab from the commits to make it work on 2.3.8?
>
> Look at the individual commits in rails.git. I just used
>
> git log -p v2.3.8..v2.3.11 -- actionpack
>
> in rails.git to find those for you. I just looked for things that
> would manipulate cookies/headers.
>
> --
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: problem setting multiple cookies
2011-06-21 3:38 2% ` Jason Su
@ 2011-06-21 18:29 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2011-06-21 18:29 UTC (permalink / raw)
To: unicorn list
Jason Su <jason@lookbook.nu> wrote:
> I'm using unicorn 3.6.2
>
> What I was trying to say is that the JoinCookie middleware didn't do
> anything. headers["Set-Cookie"] is a string, and the _test_session
> cookie was always being set correctly. The problem happens only when I
> try to set multiple cookies myself.
Yeah, you're not getting an Array (at least not at that stage) as I
semi-hoped so JoinCookie wouldn't solve it.
> Here's the output of p headers in JoinCookie again
>
> {"X-Runtime"=>"92", "Content-Type"=>"text/html",
> "Content-Length"=>"1", "Set-Cookie"=>"first=; domain=.test.com;
> path=/; expires=Thu, 01-Jan-1970 00:00:00 GMTsecond=1;
> domain=.test.com; path=/; expires=Mon, 21-Jun-2021 03:35:30
> GMT\n_test_session=1804126e6bad8cf4b25602b74218f003; domain=.test.com;
> path=/; HttpOnly", "Cache-Control"=>"no-cache"}
>
> Where is JoinCookie is getting it's headers from?
app.call(env) is what dispatches your Rails app, so the headers are what
Rails (and middlewares further down the stack) sets.
Any luck with looking at Rails changes that might impact this?
At this point it's clear it's a Rails/Rack issue (and for an old
version, at that) and not something that can be done with Unicorn.
Maybe you can fix whatever the Rails v2.3.8..v2.3.11 upgrade broke
in your app, but whatever's broken is broken at the application layer
and not in Unicorn.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: problem setting multiple cookies
@ 2011-06-22 4:18 1% ` Jason Su
2011-06-22 6:02 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Jason Su @ 2011-06-22 4:18 UTC (permalink / raw)
To: unicorn list
Hey Eric,
Actually I think I am using OldRails -- don't know what it is or why
I'm doing it, but this is in my config.ru:
map '/' do
if RAILS_ENV != 'production'
begin
require 'rack/bug'
use Rack::Bug, :password => nil
rescue LoadError
end
use Rails::Rack::Static
end
run Unicorn::App::OldRails.new
end
Should I not be using OldRails?
I'm also trying to upgrade to Rails 2.3.11 again. Here's the error
that's stopping me:
You have a nil object when you didn't expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.split You have a nil object
when you didn't expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.split
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/actionpack-2.3.11/lib/action_controller/cgi_process.rb:52:in
`dispatch_cgi'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/actionpack-2.3.11/lib/action_controller/dispatcher.rb:101:in
`dispatch_cgi'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/actionpack-2.3.11/lib/action_controller/dispatcher.rb:27:in
`dispatch'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/app/old_rails.rb:22:in
`call'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/app/old_rails/static.rb:56:in
`call'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/rack-1.1.2/lib/rack/urlmap.rb:47:in
`call'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/rack-1.1.2/lib/rack/urlmap.rb:41:in
`each'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/rack-1.1.2/lib/rack/urlmap.rb:41:in
`call'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:545:in
`process_client'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/oob_gc.rb:60:in
`process_client'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:623:in
`worker_loop'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:621:in
`each'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:621:in
`worker_loop'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:509:in
`spawn_missing_workers'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:507:in
`fork'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:507:in
`spawn_missing_workers'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:503:in
`each'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:503:in
`spawn_missing_workers'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:516:in
`maintain_worker_count'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn/http_server.rb:166:in
`start'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/lib/unicorn.rb:30:in
`run'
/Users/kittyflat/.rvm/rubies/ruby-1.8.7-p334/lib/ruby/gems/1.8/gems/unicorn-3.6.2/bin/unicorn_rails:208
/Users/kittyflat/.rvm/gems/ruby-1.8.7-p334/bin/unicorn_rails:19:in `load'
/Users/kittyflat/.rvm/gems/ruby-1.8.7-p334/bin/unicorn_rails:19
I found this, which I think is somehow related to the fix:
https://gist.github.com/826692
Do you see anything that looks familiar?
Thanks,
jason
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 1%]
* Re: problem setting multiple cookies
2011-06-22 4:18 1% ` Jason Su
@ 2011-06-22 6:02 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2011-06-22 6:02 UTC (permalink / raw)
To: unicorn list
Jason Su <jason@lookbook.nu> wrote:
> Hey Eric,
>
> Actually I think I am using OldRails -- don't know what it is or why
> I'm doing it, but this is in my config.ru:
It might be because you were on Rails 2.2 or earlier and upgraded?
> map '/' do
> if RAILS_ENV != 'production'
> begin
> require 'rack/bug'
> use Rack::Bug, :password => nil
> rescue LoadError
> end
>
> use Rails::Rack::Static
> end
>
> run Unicorn::App::OldRails.new
> end
>
> Should I not be using OldRails?
OldRails was for Rails <= 2.2, so you should probably try this instead:
run ActionController::Dispatcher.new
This should use the pure Rack backend that avoids the old CGI
and CGIWrapper code which Rails <=2.2 relied on.
> I'm also trying to upgrade to Rails 2.3.11 again. Here's the error
> that's stopping me:
>
> You have a nil object when you didn't expect it!
> You might have expected an instance of Array.
> The error occurred while evaluating nil.split You have a nil object
> when you didn't expect it!
<snip>
> I found this, which I think is somehow related to the fix:
> https://gist.github.com/826692
>
> Do you see anything that looks familiar?
I haven't seen it before, but it looks to be on the right track.
OldRails uses the CGIWrapper code which is based on the original
code in Mongrel, too.
If you avoid CGIWrapper, you should be able to avoid the
workarounds/issues with it like the above gist.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Unicorn vs Apache
@ 2011-07-11 16:07 2% Matt Smith
2011-07-11 18:45 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Matt Smith @ 2011-07-11 16:07 UTC (permalink / raw)
To: mongrel-unicorn
Hello all,
I have always deploys rails apps with unicorn and nginx as a reverse
proxy in the past. However, I am working with a new firm and they would
like to use Apache with mod_pagespeed in front of unicorn.
We will be deploying a rails 3.1 app with streaming. To me,
mod_pagespeed does not seem like a magic bullet, as it appears to be a
collection of best practices which have to be rewritten as a page is
being served. Why not code with the best practices in mind as to not
have to scan and yet again reprocess the content?
Searching on the web, I find few entries about mod_page speed and rails,
and relatively few entries about apache with unicorn.
To me it seems that using best practices, with appropriate use of
caching, and nginx would be a better solution than apache with
mod_pagespeed.
Questions:
1) Does nginx provide any necessary services to unicorn, that apache
cannot provide?
2) If apache can provide all necessary services, does it do them as well
as nginx.
3) We would like the fastest user experience possible. Does apache with
mod_pagespeed hold any weight here over nginx?
4) Part of a fast user experience is how fast you get the page, correct?
Seems like nginx has the upper hand here, due to nginx's nio vs apache's
threading.
5) Would it make since to put nginx in front of apache w/mod_pagespeed
(or visa versa), to use the applicable mod_pagespeed filters? Or is this
a bad idea to begin with?
6) Is there anything else I am missing? Are there any specific resources
I need to look at?
7) Should we go with nginx or apache? (Opinions ok.)
Much thanks,
Matt Smith
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Unicorn vs Apache
2011-07-11 16:07 2% Unicorn vs Apache Matt Smith
@ 2011-07-11 18:45 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2011-07-11 18:45 UTC (permalink / raw)
To: unicorn list
Matt Smith <matthewcalebsmith@gmail.com> wrote:
> Hello all,
>
> I have always deploys rails apps with unicorn and nginx as a reverse
> proxy in the past. However, I am working with a new firm and they would
> like to use Apache with mod_pagespeed in front of unicorn.
>
> We will be deploying a rails 3.1 app with streaming. To me,
> mod_pagespeed does not seem like a magic bullet, as it appears to be a
> collection of best practices which have to be rewritten as a page is
> being served.
note: mod_pagespeed is totally independent if Rails 3.1 streaming.
Keep in mind the most important feature of nginx Unicorn relies on (full
request/response buffering) also defeats Rails 3.1 streaming because of
how it's currently implemented.
> Why not code with the best practices in mind as to not
> have to scan and yet again reprocess the content?
My thoughts exactly :) But implementing those best practices
isn't cheap, either. If you have to run mod_pagespeed, the safest
configuration would be:
nginx <-> mod_pagespeed <-> unicorn
Of course that would introduce latency. Maybe there's Rack middleware
that works like mod_pagespeed...
> Searching on the web, I find few entries about mod_page speed and rails,
> and relatively few entries about apache with unicorn.
Apache + Unicorn is still unsupported since (as far as anybody knows),
it doesn't fully buffer responses and requests to completely isolate
Unicorn from the harmful effects of slow clients.
> To me it seems that using best practices, with appropriate use of
> caching, and nginx would be a better solution than apache with
> mod_pagespeed.
I agree.
> Questions:
>
> 1) Does nginx provide any necessary services to unicorn, that apache
> cannot provide?
nginx provides full request + response buffering (all uploads, and large
response bodies that can't fit into socket buffers). nginx also
provides inexpensive handling of idle keepalive connections; so you
can have a lot of idle connections for little memory usage.
Apache 2.x mpm_event /should/ be as good *if* (and only if) it an
provide the full request/response buffering; but I don't have any
experience with it. mpm_worker can be fairly efficient with Linux+NPTL
if you can use smaller pthreads stack sizes (and are using 64-bit).
> 2) If apache can provide all necessary services, does it do them as well
> as nginx.
Buffering + keepalive maintenance is the most important thing I care
about, and as far as I know, Apache doesn't do that as well as nginx.
> 3) We would like the fastest user experience possible. Does apache with
> mod_pagespeed hold any weight here over nginx?
The full request/response buffering of nginx also hurts latency a
little, so the user experience is slightly worse under low load.
However, nginx does the best job of keeping things going under high
load.
> 4) Part of a fast user experience is how fast you get the page, correct?
> Seems like nginx has the upper hand here, due to nginx's nio vs apache's
> threading.
Not sure what "nio" means... non-blocking I/O? Apache can use mpm_event
which is like the non-blocking + event loop nginx uses, but AFAIK the
buffering of requests responses is not like what nginx implements.
> 5) Would it make since to put nginx in front of apache w/mod_pagespeed
> (or visa versa), to use the applicable mod_pagespeed filters? Or is this
> a bad idea to begin with?
nginx <-> mod_pagespeed <-> unicorn should be alright, but of course not
ideal because of the extra copying + latency involved.
For non-Ruby/Rack apps, I've always recommended nginx sit in front of
any Apache 1.3 or 2.x+mpm_prefork apps that are exposed to slow clients,
too.
> 6) Is there anything else I am missing? Are there any specific resources
> I need to look at?
If you're willing to take a risk with Rainbows!, I posted some notes
about making it work with Rails 3.1 streaming here:
http://thread.gmane.org/gmane.comp.lang.ruby.rainbows.general/229
Keep in mind nobody I know of uses Rainbows! in production; but
it /should/ be able to work without nginx (or any proxy).
> 7) Should we go with nginx or apache? (Opinions ok.)
nginx remains the only supported reverse proxy for Unicorn at this
moment.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Strange quit behavior
@ 2011-08-02 20:09 2% James Cox
2011-08-02 20:34 0% ` Alex Sharp
2011-08-02 21:53 0% ` Eric Wong
0 siblings, 2 replies; 200+ results
From: James Cox @ 2011-08-02 20:09 UTC (permalink / raw)
To: unicorn list
Hey,
So here are some tasks for managing unicorn:
https://gist.github.com/1121076
I've found that it's very unreliable for quitting / terminating
unicorn and restarting with new code. When doing rapid deployments
particularly, i've found that i have to go in and kill -9 the master
process, and start again.
any thoughts on why it seems ineffective?
Thanks.
PS: here's the unicorn config:
https://gist.github.com/0dd07c5ad00c56d161c7
James
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Strange quit behavior
2011-08-02 20:09 2% Strange quit behavior James Cox
@ 2011-08-02 20:34 0% ` Alex Sharp
2011-08-02 21:54 0% ` Eric Wong
2011-08-02 21:53 0% ` Eric Wong
1 sibling, 1 reply; 200+ results
From: Alex Sharp @ 2011-08-02 20:34 UTC (permalink / raw)
To: unicorn list
> I've found that it's very unreliable for quitting / terminating
> unicorn and restarting with new code. When doing rapid deployments
> particularly, i've found that i have to go in and kill -9 the master
> process, and start again.
We've noticed some of this behavior as well. We've seen the new master
start spinning and consume 100% cpu (according to top). The old master
and workers are still running and working, but the new master just
hangs, and we have to kill -9. Our solution was to add the following
to our unicorn config, which *seems* to have solved the problem:
Unicorn::HttpServer::START_CTX[0] =
"#{path_to_app}/shared/bundle/ruby/1.9.1/bin/unicorn_rails"
This is outlined a bit here: http://unicorn.bogomips.org/Sandbox.html.
I'm not really sure why this seems to have fixed the problem -- all I
can tell you is that we haven't seen it since adding this line to the
config.
- Alex
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Strange quit behavior
2011-08-02 20:09 2% Strange quit behavior James Cox
2011-08-02 20:34 0% ` Alex Sharp
@ 2011-08-02 21:53 0% ` Eric Wong
2011-08-02 22:46 0% ` James Cox
1 sibling, 1 reply; 200+ results
From: Eric Wong @ 2011-08-02 21:53 UTC (permalink / raw)
To: unicorn list; +Cc: James Cox
James Cox <james@imaj.es> wrote:
> Hey,
>
> So here are some tasks for managing unicorn:
>
> https://gist.github.com/1121076
Can we ignore the :restart task? It's a bit fragile since it doesn't
wait for the old process to die (SIGTERM means
kill-as-quickly-as-possible, but given a loaded system it can still take
some time).
> I've found that it's very unreliable for quitting / terminating
> unicorn and restarting with new code. When doing rapid deployments
> particularly, i've found that i have to go in and kill -9 the master
> process, and start again.
If you SIGQUIT/SIGTERM before the app is loaded, the signal could
be ignored. This behavior should probably change...
> any thoughts on why it seems ineffective?
>
> Thanks.
Which version of Unicorn are you using? I recall some minor tweaks
to avoid/minimize race conditions over the years so maybe some
are fixed.
> PS: here's the unicorn config:
>
> https://gist.github.com/0dd07c5ad00c56d161c7
Avoid the top piece of the before_fork hook to TTOU workers, it's
needlessly complex for most deployments unless you're really
memory-constrained.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Strange quit behavior
2011-08-02 20:34 0% ` Alex Sharp
@ 2011-08-02 21:54 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2011-08-02 21:54 UTC (permalink / raw)
To: unicorn list
Alex Sharp <ajsharp@gmail.com> wrote:
> > I've found that it's very unreliable for quitting / terminating
> > unicorn and restarting with new code. When doing rapid deployments
> > particularly, i've found that i have to go in and kill -9 the master
> > process, and start again.
>
> We've noticed some of this behavior as well. We've seen the new master
> start spinning and consume 100% cpu (according to top). The old master
> and workers are still running and working, but the new master just
> hangs, and we have to kill -9.
Can you try to strace (or equivalent) the old master to see what's using
100% CPU?
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Strange quit behavior
2011-08-02 21:53 0% ` Eric Wong
@ 2011-08-02 22:46 0% ` James Cox
0 siblings, 0 replies; 200+ results
From: James Cox @ 2011-08-02 22:46 UTC (permalink / raw)
To: Eric Wong, unicorn list
On Tue, Aug 2, 2011 at 17:53, Eric Wong <normalperson@yhbt.net> wrote:
> James Cox <james@imaj.es> wrote:
>> Hey,
>>
>> So here are some tasks for managing unicorn:
>>
>> https://gist.github.com/1121076
>
> Can we ignore the :restart task? It's a bit fragile since it doesn't
> wait for the old process to die (SIGTERM means
> kill-as-quickly-as-possible, but given a loaded system it can still take
> some time).
We mostly restart (what surprise). What pattern works best here for
that? (speed of deploy is important, but definitely assume a long boot
time)
>
>> I've found that it's very unreliable for quitting / terminating
>> unicorn and restarting with new code. When doing rapid deployments
>> particularly, i've found that i have to go in and kill -9 the master
>> process, and start again.
>
> If you SIGQUIT/SIGTERM before the app is loaded, the signal could
> be ignored. This behavior should probably change...
>
>> any thoughts on why it seems ineffective?
>>
>> Thanks.
>
> Which version of Unicorn are you using? I recall some minor tweaks
> to avoid/minimize race conditions over the years so maybe some
> are fixed.
>
gem 'unicorn' - so whatever seems up to date. My lock says 4.0.1
>> PS: here's the unicorn config:
>>
>> https://gist.github.com/0dd07c5ad00c56d161c7
>
> Avoid the top piece of the before_fork hook to TTOU workers, it's
> needlessly complex for most deployments unless you're really
> memory-constrained.
>
So what should that look like? all but that nr-workers stuff? can i
just remove the before fork? and what would you say is a super good
unicorn config to start from?
thanks!
james
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Unicorn in the jail
@ 2011-08-03 16:41 2% Tatsuya Ono
0 siblings, 0 replies; 200+ results
From: Tatsuya Ono @ 2011-08-03 16:41 UTC (permalink / raw)
To: mongrel-unicorn
I ran into a problem when sending USR2 signal to master process which
is listening a TCP port. It shows the following error in log.
err: adding listener failed addr=0.0.0.0:3010 (in use)
I found similar error reported on this mailing list and yes, the host
I got the error was in the FreeBSD jail.
http://rubyforge.org/pipermail/mongrel-unicorn/2009-December/000205.html
http://rubyforge.org/pipermail/mongrel-unicorn/2009-December/000212.html
I dug into the problem and found some interesting thing. I want to
share it and the solution here.
* Why does it happen in the Jail?
When you bind 0.0.0.0 address to the socket on a Jail host, Jail
doesn't bind 0.0.0.0. Instead of it, it binds to the IP address of the
host such as 10.100.1.50, 192.168.1.50, etc.
Then the problem happens in Unicorn::HttpServer#inherit_listeners!
when it gets signal like USR2.
https://github.com/defunkt/unicorn/blob/406b8b0e2ed6e5be34d8ec3cd4b16048233c2856/lib/unicorn/http_server.rb#L703-731
Here you get 10.100.1.50 (for example) as inherited address from
ENV['UNICORN_FD'] but 0.0.0.0. So Unicorn tries to bind to 0.0.0.0
(again) at line 728. Then it fails because it is actually in use.
* How to solve the issue?
Thankfully unicorn.conf is ruby script so that I could write a logic
for Jail hosts without specifying IP address of those hosts (which we
want to avoid since we deploy it multiple servers). I share the
configuration file.
https://gist.github.com/1122965
Then Unicorn flies in the jail happily.
Tatsuya Ono
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Strange quit behavior
@ 2011-08-17 4:26 1% ` Alex Sharp
0 siblings, 1 reply; 200+ results
From: Alex Sharp @ 2011-08-17 4:26 UTC (permalink / raw)
To: unicorn list
On Fri, Aug 5, 2011 at 1:07 AM, Eric Wong <normalperson@yhbt.net> wrote:
> Is this Unicorn 3.x? Which 3.x version exactly? Maybe give 4.0.1
> or the 4.0.2 git prerelease a try, too.
We're using version 3.6.2 with ruby 1.9.2-p180 on ubuntu 11.x. I know
the kernel on this version of ubuntu has a know signal trapping bug,
but I don't think that's what happening here. The processes respond
after I send the restart signal to them again (USR2 + QUIT).
> Can I get more? (until the next select() call, at least). I'd like to
> see the timeout argument that gets passed to the next select().
Here's some more output. This is from the old master though, not a new
master that is pegging the CPU. In this instance it's almost like the
old master just ignores the signal.
select(4, [3], NULL, NULL, {8, 129984}) = 0 (Timeout)
wait4(-1, 0x7fff16b82e4c, WNOHANG, NULL) = 0
clock_gettime(CLOCK_REALTIME, {1313554942, 259408364}) = 0
fstat(12, {st_dev=makedev(202, 1), st_ino=20373, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/12-23:14:20, st_mtime=2011/08/12-23:14:20,
st_ctime=2011/08/17-04:22:21}) = 0
clock_gettime(CLOCK_REALTIME, {1313554942, 259775504}) = 0
fstat(13, {st_dev=makedev(202, 1), st_ino=20381, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/12-23:14:20, st_mtime=2011/08/12-23:14:20,
st_ctime=2011/08/17-04:22:17}) = 0
clock_gettime(CLOCK_REALTIME, {1313554942, 259936816}) = 0
fstat(14, {st_dev=makedev(202, 1), st_ino=20382, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/12-23:14:21, st_mtime=2011/08/12-23:14:21,
st_ctime=2011/08/17-04:22:19}) = 0
clock_gettime(CLOCK_REALTIME, {1313554942, 260086380}) = 0
fstat(15, {st_dev=makedev(202, 1), st_ino=20185, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/12-23:14:21, st_mtime=2011/08/12-23:14:21,
st_ctime=2011/08/17-04:22:17}) = 0
clock_gettime(CLOCK_REALTIME, {1313554942, 260235797}) = 0
fstat(16, {st_dev=makedev(202, 1), st_ino=20255, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/12-23:14:21, st_mtime=2011/08/12-23:14:21,
st_ctime=2011/08/17-04:22:19}) = 0
clock_gettime(CLOCK_REALTIME, {1313554942, 260384849}) = 0
fstat(17, {st_dev=makedev(202, 1), st_ino=20383, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/12-23:14:21, st_mtime=2011/08/12-23:14:21,
st_ctime=2011/08/17-04:22:19}) = 0
clock_gettime(CLOCK_REALTIME, {1313554942, 260534792}) = 0
fstat(18, {st_dev=makedev(202, 1), st_ino=20384, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/12-23:14:21, st_mtime=2011/08/12-23:14:21,
st_ctime=2011/08/17-04:22:19}) = 0
clock_gettime(CLOCK_REALTIME, {1313554942, 260684278}) = 0
fstat(10, {st_dev=makedev(202, 1), st_ino=20196, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/16-16:33:46, st_mtime=2011/08/16-16:33:46,
st_ctime=2011/08/17-04:22:19}) = 0
clock_gettime(CLOCK_REALTIME, {1313554942, 260833725}) = 0
select(4, [3], NULL, NULL, {8, 976047}
> If you see select() with very small timeout, use "strace -v" to get the
> st_ctime from fstat()s...
>
> I could have a math bug in murder_lazy_workers (please read/review the
> logic in that method, I haven't noticed issues myself).
>
> I made some tweaks to the master sleep strategy within the past year to
> reduce wakeups during idle hours. This is intended to go with Ruby
> 1.9.3 which will have /much/ better thread wakeup strategy that reduces
> power consumption during idle times.
>
>> The first line was when the master was idle, and then I threw a few
>> requests at it.
>
> Are all workers responding as expected and not dying?
The old workers appear to be serving requests.
- Alex
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 1%]
* Re: Strange quit behavior
@ 2011-08-18 23:13 2% ` Alex Sharp
0 siblings, 1 reply; 200+ results
From: Alex Sharp @ 2011-08-18 23:13 UTC (permalink / raw)
To: unicorn list
Ok, thanks a lot for the for the patch attempt and subsequent in-depth
explanation. I know this thread is long-lived and a bit confusing,
mostly because I've seen two different sets of scenarios that yield
the same result, which is that unicorn does not restart.
The first scenario (and the one that started this thread) was that
unicorn actually receives the signal, spawns a new master, and that
process pegs the cpu and spins into eternity. The second scenario is
that the old master seems to completely ignore the USR2 signal the
first time, and when sent again, properly restarts. In both scenarios
the old master and workers continue to serve the old code.
I thought that setting the Unicorn::HttpServer::START_CTX[0] in my
unicorn config had solved the first scenario (pegged cpu), but I found
a new occurrence of it today, and I now have some new strace output
for this scenario (strace -v):
select(4, [3], NULL, NULL, {60, 727229}) = 0 (Timeout)
wait4(-1, 0x7ffffd70f72c, WNOHANG, NULL) = 0
clock_gettime(CLOCK_REALTIME, {1313708081, 720757662}) = 0
fstat(8, {st_dev=makedev(202, 1), st_ino=39806, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/18-14:32:10, st_mtime=2011/08/18-14:32:10,
st_ctime=2011/08/18-22:54:06}) = 0
clock_gettime(CLOCK_REALTIME, {1313708081, 721131305}) = 0
fstat(10, {st_dev=makedev(202, 1), st_ino=45370, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/18-14:32:10, st_mtime=2011/08/18-14:32:10,
st_ctime=2011/08/18-22:54:05}) = 0
clock_gettime(CLOCK_REALTIME, {1313708081, 721290800}) = 0
select(4, [3], NULL, NULL, {565, 34005}) = 0 (Timeout)
wait4(-1, 0x7ffffd70f72c, WNOHANG, NULL) = 0
clock_gettime(CLOCK_REALTIME, {1313708646, 853870540}) = 0
fstat(8, {st_dev=makedev(202, 1), st_ino=39806, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/18-14:32:10, st_mtime=2011/08/18-14:32:10,
st_ctime=2011/08/18-22:59:06}) = 0
clock_gettime(CLOCK_REALTIME, {1313708646, 854102750}) = 0
fstat(10, {st_dev=makedev(202, 1), st_ino=45370, st_mode=S_IFREG,
st_nlink=0, st_uid=1001, st_gid=1001, st_blksize=4096, st_blocks=0,
st_size=0, st_atime=2011/08/18-14:32:10, st_mtime=2011/08/18-14:32:10,
st_ctime=2011/08/18-23:04:05}) = 0
clock_gettime(CLOCK_REALTIME, {1313708646, 854260655}) = 0
select(4, [3], NULL, NULL, {598, 630876}
With respect to the second scenario (ignoring signals), I'm going to
recommend we table that issue for now, as we're currently running on a
version of ubuntu (11.10) which has a known signal trapping bug with
ruby 1.9.2-p180, and downgrading to 10.04 may solve that problem.
Granted, when I've observed this in the past with other libraries, the
process becomes completely unresponsive, whereas the behavior in
unicorn is more intermittent. Either way, we are in the process of
downgrading our servers to ubuntu 10.04, so let's not waste anymore
time trying to debug something that may well be occurring due to a
kernel bug. If we continue to see this after the downgrade to 10.04,
I'll report back and we can keep digging.
Again, my apologies for the confusion between the two scenarios, and
thanks for all your help.
- Alex
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Strange quit behavior
@ 2011-08-23 16:49 2% ` Alex Sharp
0 siblings, 0 replies; 200+ results
From: Alex Sharp @ 2011-08-23 16:49 UTC (permalink / raw)
To: unicorn list
On Tuesday, August 23, 2011 at 12:12 AM, Eric Wong wrote:
> Did you send any signals to 18862 while you were making this strace?
Not while, no. I had already sent a USR2 signal to this process (the old master), and then I attached strace to it.
I'll try sending another USR2 signal next time while strace is attached.
> > I went ahead and ran strace with the same flags on the *new* master,
> > and saw a bunch of output that looked bundler-related:
> > https://gist.github.com/138344b5b19ec6ba1a4c
> >
> > Even more bizarre, eventually the process started successfully :-/ Is
> > it possible this had something to do with strace de-taching?
>
> That looks like pretty normal "require" behavior. strace would slow
> down requires, a lot.
>
> So this was with preload_app=true? While you're debugging problems,
> I suggest keeping preload_app=false and worker_problems=1 to minimize
> the variables.
Ok, I'll change those and report back. I'm guessing you meant worker_processes (not problems)?
>
> > You can see this in the unicorn.stderr.log file I included in the
> > gist. Check out these two lines in particular, which occur 25 minutes
> > apart:
> >
> > I, [2011-08-23T02:15:08.396868 #22169] INFO -- : Refreshing Gem list
> > I, [2011-08-23T02:40:16.621210 #22925] INFO -- : worker=1 spawned pid=22925
>
> Wow, it takes 25 minutes to load your application? strace makes the
> application /much/ slower, so I can actually believe it takes that long.
No, my mistake. Loading the application only takes about 10 seconds, and I only had strace attached to this process for a few seconds (less than 10). My point here was to show that the new master just spun for a good 25 minutes (presumably trying to load files over and over again), and then, seemingly out of nowhere, the new master came up and spawned the new workers.
Next time I'll try to get attached with strace earlier and record more output.
>
> Loading large applications is very slow under Ruby 1.9.2, there's some
> pathological load performance issues fixed in 1.9.3.
>
Yep, I've read about those, and I've seen Xavier's patch, but I don't think that's the issue here (though, it appears that's why the files attempting to be loaded in the strace output do not exist). Under normal circumstances, loading the app takes about 10 seconds and doesn't peg the cpu while doing it.
>
> So you're saying /without/ strace, CPU usage _stays_ at 100% and _never_
> goes down?
Correct.
>
>
> Generally, can you reproduce this behavior on a plain (empty) Rails
> application with no extra gems?
>
Good idea, I'll try that next.
- Alex
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Unicorn for Rails development mode?
@ 2011-08-28 23:14 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2011-08-28 23:14 UTC (permalink / raw)
To: unicorn list
Jay Levitt <jlevitt@tiptap.com> wrote:
> I'm tired of the bugginess of Webrick, so I want to upgrade to
> something modern for developing Rails apps on my Mac, and I'm a bit
> out of the loop. Is Unicorn a drop-in replacement these days in
> development mode? Does it have advantages or disadvantages compared to
> Thin? I don't even know what capabilities I should care about, but
> some obvious ones are:
unicorn should be a drop-in replacement, you shouldn't need a
unicorn-specific config file for development.
The "unicorn_rails"[1] command was designed Rails 1.x/2.x users, while
"unicorn" is a better fit for Rails 3 (and all Rack frameworks),
but either works for Rails 3
Thin is a great server and I've had no issues with Webrick for
development, either.
You don't have to worry about slow clients hitting Unicorn in
development, so you can forgo nginx for development.
> - Doesn't stop logging after a while (hi, Webrick)
This is not a problem I've heard of with either server. It /may/ be an
issue with the Rails buffered logger implementation, though. The Ruby
standard library Logger used by Unicorn does no buffering.
> - Logs to STDOUT or STDERR
This is the default, and Unicorn explicitly disables output buffering
on both of these.
> - Reloads everything that ought to be reloaded on each request
Set RAILS_ENV=development in your environment.
> - Works well with 1.9.2
Yes, I regularly test against 1.8.7, 1.9.2, 1.9.3dev and trunk.
> Any advice? I'd think this would be a FAQ, but I haven't found any
> discussions on the topic.
I don't do much Rails development, but everything /should/ work fine. I
will never officially support non-Free systems, but from what I've heard
it works fine on the ones names after fruits.
Others on this list can hopefully chime in, too.
[1] - There's some confusion that was the result of "unicorn_rails"
being an automatic compatibility layer for old Rails. If I
could to do it all over again, I'd leave "unicorn_rails" out
and force folks to setup the compatibility layer themselves
to learn how Rack works.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Sending ABRT to timeout-errant process before KILL
@ 2011-09-08 9:04 1% J. Austin Hughey
0 siblings, 0 replies; 200+ results
From: J. Austin Hughey @ 2011-09-08 9:04 UTC (permalink / raw)
To: mongrel-unicorn
Hello there -
I've recently been working with a customer in my capacity as a support engineer at Engine Yard who's having some trouble with Unicorn. Basically, they're finding their application being force-killed once it reaches the default timeout. Rather than simply increasing the timeout, we're trying to find out _why_ their application is being killed. Unfortunately we can't quite do that because the application is being force-killed without warning, making it so that the customer's logging can't actually be written to the disk. (This is an intermittent issue as opposed to something that happens 100% of the time.)
In discussing the matter internally and with our customer, I came up with a simple monkey patch to Unicorn that _sort of_ works, but I'm having some trouble with it once the number of Unicorn workers goes beyond one. I originally limited it to just one worker because I wanted to limit the possibility that multiple workers could cause problems in figuring things out -- turns out I was right.
I'm going to show the patch in two ways: 1) inline, at the bottom of this post, and 2) by link to GitHub:
https://github.com/jaustinhughey/unicorn/blob/abort_prior_to_kill_workers/lib/unicorn/http_server.rb#L438
The general idea is that I'd like to have some way to "warn" the application when it's about to be killed. I've patched murder_lazy_workers to send an abort signal via kill_worker, sleep for 5 seconds, then look and see if the process still exists by using Process.getpgid. If so, it sends the original kill command, and if not, a rescue block kicks in to prevent the raised error from Process.getpgid from making things explode.
I've created a simulation app, built on Rails 3.0, that uses a generic "posts" controller to simulate a long-running request. Instead of just throwing a straight-up sleep 65 in there, I have it running through sleeping 1 second on a decrementing counter, and doing that 65 times. The reason is because, assuming I've read the code correctly, even with my "skip sleeping workers" commented line below, it'll skip over the process, thus rendering my simulation of a long-running process invalid. However, clarification on this point is certainly welcome. You can see the app here: https://github.com/jaustinhughey/unicorn_test/blob/master/app/controllers/posts_controller.rb
The problem I'm running into, and where I could use some help, is when I increase the number of Unicorn workers from one to two. When running only one Unicorn worker, I can access my application's posts_controller's index action, which has the intentionally long-running code. At that time I tail -f unicorn.log and production.log. Those two logs look like this with one Unicorn worker:
WITH ONE UNICORN WORKER:
========================
production.log:
---------------
Sleeping 1 second (65 to go)…
... continued ...
Sleeping 1 second (7 to go)...
Sleeping 1 second (6 to go)...
Sleeping 1 second (5 to go)...
Caught Unicorn kill exception!
Sleeping 1 second (4 to go)...
Sleeping 1 second (3 to go)...
Sleeping 1 second (2 to go)...
Sleeping 1 second (1 to go)...
Completed 500 Internal Server Error in 65131ms
NoMethodError (undefined method `query_options' for nil:NilClass):
app/controllers/posts_controller.rb:32:in `index'
(I think the NoMethodError issue above is due to me calling a disconnect on ActiveRecord in the Signal.trap block, so I think that can be safely ignored.)
As you can see, the Signal.trap block inside the aforementioned posts_controller is working in this case. Corresponding log entries in unicorn.log concur:
unicorn.log:
------------
worker=0 ready
master process ready
[2011-09-08 00:31:01 PDT] worker=0 PID: 28921 timeout hit, sending ABRT to process 28921 then sleeping 5 seconds...
[2011-09-08 00:31:06 PDT] worker=0 PID:28921 timeout (61s > 60s), killing
reaped #<Process::Status: pid 28921 SIGKILL (signal 9)> worker=0
worker=0 ready
So with one worker, everything seems cool. But with two workers?
WITH TWO UNICORN WORKERS:
=========================
production.log:
---------------
Sleeping 1 second (8 to go)...
Sleeping 1 second (7 to go)...
Sleeping 1 second (6 to go)...
Sleeping 1 second (5 to go)...
Sleeping 1 second (4 to go)...
Sleeping 1 second (3 to go)...
Sleeping 1 second (2 to go)...
Sleeping 1 second (1 to go)...
Rendered posts/index.html.erb within layouts/application (13.2ms)
Completed 200 OK in 65311ms (Views: 16.9ms | ActiveRecord: 0.5ms)
Note that there is no notice that the ABRT signal was trapped, nor is there a NoMethodError (likely caused by disconnecting from the database) as above. Odd.
unicorn.log:
------------
Nothing. No new data whatsoever.
The only potential clue I can see at this point would be a start-up message in unicorn.log. After increasing the number of Unicorn workers to two, I examined unicorn.log again and found this:
master complete
I, [2011-09-08T00:34:40.499437 #29572] INFO -- : unlinking existing socket=/var/run/engineyard/unicorn_ut.sock
I, [2011-09-08T00:34:40.499888 #29572] INFO -- : listening on addr=/var/run/engineyard/unicorn_ut.sock fd=5
I, [2011-09-08T00:34:40.504542 #29572] INFO -- : Refreshing Gem list
worker=0 ready
master process ready
[2011-09-08 00:34:49 PDT] worker=1 PID: 29582 timeout hit, sending ABRT to process 29582 then sleeping 5 seconds...
[2011-09-08 00:34:50 PDT] worker=1 PID:29582 timeout (1315467289s > 60s), killing
reaped #<Process::Status: pid 29582 SIGIOT (signal 6)> worker=1
worker=1 ready
So it looks like Worker 1 is hitting a strange/false timeout of 1315467289 seconds, which isn't really possible as it wasn't even running 1315467289 seconds prior to that (which equates to roughly 41 years ago if my math is right).
---
Needless to say, I'm a bit stumped at this point, and would sincerely appreciate another point of view on this. Am I going about this all wrong? Is there a better approach I should consider? And if I'm on the right track, how can I get this to work regardless of how many Unicorn workers are running?
Thank you very much for any assistance you can provide!
-- INLINE VERSION OF PATCH --
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 78d80b4..8a2323f 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -429,6 +429,11 @@ class Unicorn::HttpServer
proc_name 'master (old)'
end
+ # A custom formatted timestamp for debugging
+ def custom_timestamp
+ return Time.now.strftime("[%Y-%m-%d %T %Z]")
+ end
+
# forcibly terminate all workers that haven't checked in in timeout seconds. The timeout is implemented using an unlinked File
def murder_lazy_workers
t = @timeout
@@ -436,16 +441,40 @@ class Unicorn::HttpServer
now = Time.now.to_i
WORKERS.dup.each_pair do |wpid, worker|
tick = worker.tick
- 0 == tick and next # skip workers that are sleeping
+
+ # REMOVE THE FOLLOWING COMMENT WHEN TESTING PRODUCTION
+# 0 == tick and next # skip workers that are sleeping
+ # ^ needs to be active, commented here for simulation purposes
+
diff = now - tick
tmp = t - diff
if tmp >= 0
next_sleep < tmp and next_sleep = tmp
next
end
- logger.error "worker=#{worker.nr} PID:#{wpid} timeout " \
- "(#{diff}s > #{t}s), killing"
- kill_worker(:KILL, wpid) # take no prisoners for timeout violations
+
+
+ # Send an ABRT signal to Unicorn and wait 5 seconds before attempting an
+ # actual kill, if and only if the process is still running.
+
+ begin
+ # Send the ABRT signal.
+ logger.debug "#{custom_timestamp} worker=#{worker.nr} PID: #{wpid} timeout hit, sending ABRT to process #{wpid} then sleeping 5 seconds..."
+ kill_worker(:ABRT, wpid)
+
+ sleep 5
+
+ # Now see if the process still exists after being given five
+ # seconds to terminate on its own, and if so, do a hard kill.
+ if Process.getpgid(wpid)
+ logger.error "#{custom_timestamp} worker=#{worker.nr} PID:#{wpid} timeout " \
+ "(#{diff}s > #{@timeout}s), killing"
+ kill_worker(:KILL, wpid) # take no prisoners for timeout violations
+ end
+ rescue Errno::ESRCH => e
+ # No process identified - maybe it exited on its own?
+ logger.debug "#{custom_timestamp} worker=#{worker.nr} PID: #{wpid} responded to ABRT on its own and no longer exists. (Received message: #{e})"
+ end
end
next_sleep
end
-- END INLINE VERSION OF PATCH --
--
J. Austin Hughey
Application Support Engineer - Engine Yard
www.engineyard.com | jhughey@engineyard.com
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 1%]
* Re: Peformance up - using OobGC & GC.disable
@ 2011-10-12 16:00 2% ` Tatsuya Ono
0 siblings, 0 replies; 200+ results
From: Tatsuya Ono @ 2011-10-12 16:00 UTC (permalink / raw)
To: unicorn list
Yes, you are right, Eric. The usage of memory stops increasing at a
certain point. Besides I do not see any significant page I/O with it.
I give the patch go for our live service without the UnicornKiller. I
will report if we experience any issues occurred in the wild.
Thanks Yuichi again for submitting the patch and sharing your knowledge.
By the way, I tested this with Rails 2.3/Ruby 1.8.7/FreeBSD 8.2
Tatsuya
On 11 October 2011 00:03, Tatsuya Ono <ononoma@gmail.com> wrote:
> Thanks Eric for the feedback.
>
> I actually had read that email and I think I understand it. But what I
> am experiencing seems a different story. Our rails app uses around
> 250MB memory usually. After applying this patch and calling
> GC.disabled on after_fork, the usage of memory increases on every
> request and goes up to 1GB easily.
>
> However, yes, I must say that I need to test more carefully. Let me
> come back later. I am going to have some stress test and monitor if
> Unicorn introduces swapping on VM with this solution. Hopefully I can
> do it tomorrow or later this week.
>
>
> Tatsuya
>
> On 10 October 2011 22:53, Eric Wong <normalperson@yhbt.net> wrote:
>> Tatsuya Ono <ononoma@gmail.com> wrote:
>>> I don't actually understand is why GC.disable solution could introduce
>>> more memory leak. If I simplify the problem, the code is something
>>> like bellow:
>>>
>>> ---------------
>>> GC.disable
>>> (do something)
>>> GC.enable
>>> GC.start
>>> ---------------
>>>
>>> When the code block finishes, I expect that memory size should be
>>> (almost) equal with the case GC is enabled at begging. But it doesn't
>>> seems so from our experience.
>>>
>>> Do anyone know why there could be significant difference on memory
>>> usage because of timing of GC? It might be a question on Ruby rather
>>> than Unicorn, though, I thought even just sharing my experience could
>>> be worth to someone here.
>>
>> Basically, the free(3) function in the C standard library does not
>> guarantee memory is released back to the kernel (speed vs memory usage
>> tradeoff).
>>
>> There was discussion of this on the usp.ruby mailing list starting at
>> Message-ID: 20110914234917.GA2480@dcvr.yhbt.net
>>
>> usp.ruby archives are at http://bogomips.org/usp.ruby/archives/2011.mbox.gz
>> _______________________________________________
>> Unicorn mailing list - mongrel-unicorn@rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>> Do not quote signatures (like this one) or top post when replying
>>
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Should USR2 always work?
@ 2011-11-23 5:31 3% ` Lawrence Pit
0 siblings, 0 replies; 200+ results
From: Lawrence Pit @ 2011-11-23 5:31 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn list
> I brought up a "bring-your-own-executable" deployment a few months ago
> but haven't heard any feedback. Here it is for Bundler:
>
> http://mid.gmane.org/20110819022316.GA2951@dcvr.yhbt.net
I tried this out by going back to a previous version of bundler, then deployed, unfortunately doesn't seem to work for me:
executing ["/srv/ec/current/script/unicorn_byoe", "-D", "-E", "staging", "-c", "/srv/ec/current/config/unicorn/staging.rb"] (in /srv/ec/releases/20111123052840)
/opt/ree/bin/ruby: no such file to load -- bundler/setup (LoadError)
Sending QUIT and then start anew works fine. Upgraded back to latest, tried again, and again no luck. QUIT + start is the only thing that works then.
Tried a slightly modified version of unicorn_byoe, I'm using rails:
#!/opt/ree/bin/ruby
require File.expand_path("../../config/boot.rb", __FILE__)
# Gem.bin_path(gem_name, executable_name, *version_requirements)
load Gem.bin_path("unicorn", "unicorn_rails")
Cheers,
Lawrence
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 3%]
* COW friendly GC and PostgreSQL
@ 2011-11-30 15:24 2% Laas Toom
2011-11-30 20:54 0% ` Jeremy Evans
0 siblings, 1 reply; 200+ results
From: Laas Toom @ 2011-11-30 15:24 UTC (permalink / raw)
To: mongrel-unicorn
Hello,
I stumbled upon a weird issue when upgrading a Rails app and trying to run in on Unicorn!:
http://stackoverflow.com/questions/8262803/unicorn-and-postgresql/
The problem manifested itself via a few different error messages as if garbage/nil was returned from database instead of actual data and it only happened with more than one concurrent request (i.e under load test).
This turned out to be an issue with COW-friendly GC that I had enabled (taken from the example unicorn.conf). Removing this from unicorn.conf made the app run smooth again.
Is this something known or did I do something wrong?
Best regards,
Laas
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: COW friendly GC and PostgreSQL
2011-11-30 15:24 2% COW friendly GC and PostgreSQL Laas Toom
@ 2011-11-30 20:54 0% ` Jeremy Evans
0 siblings, 0 replies; 200+ results
From: Jeremy Evans @ 2011-11-30 20:54 UTC (permalink / raw)
To: unicorn list
On Wed, Nov 30, 2011 at 7:24 AM, Laas Toom <laas.toom@eenet.ee> wrote:
> Hello,
>
> I stumbled upon a weird issue when upgrading a Rails app and trying to run in on Unicorn!:
> http://stackoverflow.com/questions/8262803/unicorn-and-postgresql/
>
> The problem manifested itself via a few different error messages as if garbage/nil was returned from database instead of actual data and it only happened with more than one concurrent request (i.e under load test).
>
> This turned out to be an issue with COW-friendly GC that I had enabled (taken from the example unicorn.conf). Removing this from unicorn.conf made the app run smooth again.
>
> Is this something known or did I do something wrong?
If you didn't disconnect the database connection before forking, that
could be the cause of the problem, though you should have issues
regardless of whether the GC is CoW or not. Did you just turn off
CoW, or did you turn off preload_app as well?
Looking at your config file, try adding a before_fork callback that
disconnects the database connection and things may work fine.
Jeremy
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Master repeatedly killing workers due to timeouts
@ 2011-12-08 22:24 2% ` Jonathan del Strother
0 siblings, 0 replies; 200+ results
From: Jonathan del Strother @ 2011-12-08 22:24 UTC (permalink / raw)
To: unicorn list
On 8 Dec 2011, at 20:23, Eric Wong <normalperson@yhbt.net> wrote:
> Jonathan del Strother <maillist@steelskies.com> wrote:
>> Hi,
>> We're using unicorn as a Rails server on Solaris, and it's been
>> running great for several months. We've recently been having a few
>> problems and I'm at a loss what might cause it. A number of times in
>> the past few days, our unicorn slaves keep timing out & the master
>> keeps restarting them. unicorn.log looks something like :
>
> Which versions of Ruby and Unicorn are you running?
> <snip>
Hi Eric, thanks for the comprehensive reply. As it happens, shortly after posting this, the problem started again...in the past it's been fairly sporadic, whereas this time it was extremely persistent so we were forced into tracking it down then & there. I ended up replacing unicorn with a single webrick instance, which would serve a few requests then hang...after a long debugging session we finally managed to track down the problem to a hung sphinx instance, which we use for our search engine. Ruby would try to talk to that and just sit there without ever timing out. I'm going to go through the sphinx code tomorrow and see if I can figure out why that might occur.
Thanks for the help,
Jonathan
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: FreeBSD jail and unicorn
@ 2012-01-31 18:50 2% ` Charles Hornberger
0 siblings, 0 replies; 200+ results
From: Charles Hornberger @ 2012-01-31 18:50 UTC (permalink / raw)
To: Eric Wong, unicorn list; +Cc: Philipp Bruell, "wrote:"@dcvr.yhbt.net
On 1/31/12 7:39 PM, normalperson@yhbt.net wrote:
>Philipp Bruell <Philipp.Bruell@skrill.com> wrote:
>>
>> Does someone ran into the same problem? Does someone has an idea?
>
>Tatsuya Ono documented a workaround for jails here (see gist):
>http://mid.gmane.org/CAHBuKRj09FdxAgzsefJWotexw-7JYZGJMtgUp_dhjPz9VbKD6Q@m
>ail.gmail.com
>
>(http://unicorn.bogomips.org/KNOWN_ISSUES.html refers to this link, too)
Philipp's gone afk for the evening, so I'll take the liberty of replying
with what I know ...
We tried the fix mentioned above, and it didn't work. We also tried
switching to unix sockets; no joy. (Actually it worked once, then refused
to work again.)
>If that didn't work, maybe checking stderr.log will tell you something
>more.
Nothing shows up in stderr.log It's as if the master doesn't even get the
-USR2 signal. Or as if whatever it's sending to stderr is not actually
getting to the filesystem...
In any case, we don't see anything.
Any further ideas for how to debug would be much appreciated, and my
apologies in advance for mixing up any details; Philipp was doing the work
on this, not me.
-c
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: FreeBSD jail and unicorn
[not found] ` <CB4EBD2A.7DF%philipp.bruell@skrill.com>
@ 2012-02-01 18:14 0% ` Eric Wong
[not found] ` <CB5014D7.892%philipp.bruell@skrill.com>
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2012-02-01 18:14 UTC (permalink / raw)
To: unicorn list; +Cc: Philipp Bruell, Charles Hornberger
Philipp Bruell <Philipp.Bruell@skrill.com> wrote:
> First of all, thank you for your fast reply.
No problem, depends on the time of day of course :>
> The behaviour details Charles described are correct and we are using ruby
> version 1.9.3.
>
> It's good that you've asked for the other signals. I've checked them in
> particular and it seems that it is a common signal handling problem. The
> process freaks out on each of them :-(
>
> I've attached the output of truss -f for a QUIT signal. That signal took a
> quite long time to get processed (and took all CPU cycles), but finally
> worked.
I only saw the output from the master process there, nothing from the
worker. It seems like the master is OK, but trying to kill the worker
is not. I wonder if it's related to
https://bugs.ruby-lang.org/issues/5240
With the following script, can you try sending SIGQUIT to the parent
and see what happens?
------------------------------- 8< -----------------------------
pid = fork do
r, w = IO.pipe
trap(:QUIT) do
puts "SIGQUIT received in child, exiting"
w.close
end
r.read
end
trap(:QUIT) do
puts "SIGQUIT received in parent, killing child"
Process.kill(:QUIT, pid)
p Process.waitpid2(pid)
exit
end
sleep 1 # wait for child to setup sig handler
puts "Child ready on #{pid}, parent on #$$"
sleep
----------------------------------------------------------------
If the above fails, try with different variables:
* without a jail on the same FreeBSD version/release/patchlevel
* Ruby 1.9.2 (which has a different signal handling implementation)
* different FreeBSD version
* different architecture[1]
Mixing either signal handling or fork()-ing in the presence of threads
is tricky. Ruby 1.9 uses a dedicated thread internally for signal
handling, I wouldn't be surprised if there's a bug lingering somewhere
in FreeBSD or Ruby...
Have you checked the FreeBSD mailing lists/bug trackers? I don't recall
seeing anything other than the aforementioned bug in ruby-core...
[1] - I expect there's ASM involved in signal/threading implementation
details, so there's a chance it's x86_64-specific...
> The output of USR2 signal is too long to attach it to a mail, but at a
> first sight, it repeats the following calls over and over again.
Don't send monster attachments, host it somewhere else so mail servers
won't reject it for wasting bandwidth. The mailman limit on rubyforge
is apparently 256K (already huge IMHO). Also, don't top post
> 24864:
> thr_kill(0x18c32,0x1a,0x800a8edc0,0x18a86,0x7fffffbeaf80,0x80480c000) = 0
> (0x0)
> 24864: select(4,{3},0x0,0x0,{0.100000 }) = 1 (0x1)
> 24864: read(3,"!",1024) = 1 (0x1)
OK, so the signal is received correctly by the Ruby VM in the master.
I just don't see anything in the worker, but the master does attempt
to forward SIGQUIT to the worker.
> It also seems to me, that observing the processes with truss changes the
> behaviour a lot. During the observed USR2 the master process spawns a lot
> (about 30) of <defunct> processes. I never had this before.
Some processes react strangely to being traced. Maybe there's something
better than truss nowadays?
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: FreeBSD jail and unicorn
[not found] ` <CB5014D7.892%philipp.bruell@skrill.com>
@ 2012-02-02 19:31 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2012-02-02 19:31 UTC (permalink / raw)
To: unicorn list; +Cc: Philipp Bruell, Charles Hornberger
Philipp Bruell <Philipp.Bruell@skrill.com> wrote:
> On 01/02/2012 19:14, "Eric Wong" <normalperson@yhbt.net> wrote:
> >Philipp Bruell <Philipp.Bruell@skrill.com> wrote:
> The scripts behaves exactly like unicorn. The master received the QUIT and
> passes it to the child. The child also receives it, but don't exit. While
> the master is waiting for the child to exit, it consumes all the cpu
> cycles.
Interesting, I suspect it's some bad interaction with fork() causing
signal handlers/pthreads to go bad. I expect the following simple
script to work flawlessly since it doesn't fork:
----------------------------------------
trap(:QUIT) { exit(0) }
puts "Ready for SIGQUIT on #$$"
sleep
----------------------------------------
> I don't have the option, to test without jail, on a different FreeBSD
> version nor a different architecture (and FreeBSD - on Mac OS X everything
> works perfect). But I tried ruby version 1.9.2 and that works! So I guess
> it's a bug with 1.9.3 on FreeBSD.
Can you report this as a bug to the Ruby core folks on
https://bugs.ruby-lang.org/ and also to whereever the FreeBSD hackers
take bug reports? Somebody from one of those camps should be able
to resolve the issue.
The good thing is my small sample script is enough to reproduce the
issue, so it should be easy for an experienced FreeBSD hacker to
track down.
> I've attached the truss -f output of the child process of the test script.
> But the observation with truss made the problem disappear again :-(
It could be a timing or race condition issue. I've had strace on linux
find/hide bugs because it slowed the program down enough.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Background jobs with #fork
@ 2012-04-12 17:36 2% paddor
2012-04-12 20:39 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: paddor @ 2012-04-12 17:36 UTC (permalink / raw)
To: mongrel-unicorn
Hi
I've migrated from Passenger to Unicorn about a week ago. It's great.
Great transparency and management, thanks for this great software!
A few of my Rails applications start background jobs using
Kernel#fork. Of course, the ActiveRecord connections are closed and
reopened again in the parent and child processes. The child process
also does its job.
Unfortunately, it seems that the parent (a Unicorn worker) waits for
the child (background job) to finish before serving any new requests.
Process.detach is done in the child. Process.setsid is not done. The
child's STDOUT, STDERR and the Rails logger are redirected to their
own files right after forking.
Software used:
ruby 1.9.1p376
Rubygems 1.8.17
Linux 2.6.16.60-0.21-smp (SUSE 10.2)
unicorn 4.2.1
nginx 0.8.53
The problem persists even when multiple workers are started. And the
problem was not present in the old setup with Passenger.
My question: Does Unicorn somehow check/wait for child processes
forked by the worker processes?
Thanks in advance for your help.
--
paddor
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Background jobs with #fork
2012-04-12 17:36 2% Background jobs with #fork paddor
@ 2012-04-12 20:39 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2012-04-12 20:39 UTC (permalink / raw)
To: unicorn list; +Cc: paddor
paddor <paddor@gmail.com> wrote:
> Hi
>
> I've migrated from Passenger to Unicorn about a week ago. It's great.
> Great transparency and management, thanks for this great software!
:>
> A few of my Rails applications start background jobs using
> Kernel#fork. Of course, the ActiveRecord connections are closed and
> reopened again in the parent and child processes. The child process
> also does its job.
OK, that's good.
> Unfortunately, it seems that the parent (a Unicorn worker) waits for
> the child (background job) to finish before serving any new requests.
> Process.detach is done in the child. Process.setsid is not done. The
> child's STDOUT, STDERR and the Rails logger are redirected to their
> own files right after forking.
So you're only calling fork and not exec (or system/popen, right?) It
may be the case that the client socket is kept alive in the background
process.
The client socket has the close-on-exec flag (FD_CLOEXEC) set, but
there's no close-on-fork flag, so you might have to find + close it
yourself. Here's is a nasty workaround for the child process:
ObjectSpace.each_object(Kgio::Socket) do |io|
io.close unless io.closed?
end
> The problem persists even when multiple workers are started. And the
> problem was not present in the old setup with Passenger.
>
> My question: Does Unicorn somehow check/wait for child processes
> forked by the worker processes?
Unicorn workers do not explicitly wait on child processes themselves,
unicorn workers set: trap(:CHLD,"DEFAULT") after forking, even (the
unicorn master must handle SIGCHLD, of course)
The difference between nginx+unicorn and Passenger is probabably: nginx
relies on unicorn generating an EOF to signal the end-of-response (nginx
<-> unicorn uses HTTP/1.0), this I'm sure about. I think Passenger uses a
protocol which can signal the end-of-request inline without relying on
an EOF on the socket (Hongli can correct me on this if I'm wrong).
However, nginx can still forward subsequent requests to the same unicorn
(even the same unicorn worker), because as far as the unicorn worker is
concerned (but not the OS), it's done with the original request. It's
just the original request (perhaps the original client) is stuck
waiting for the background process to finish.
I can probably writeup a better explanation (perhaps on the usp.ruby
(Unix Systems Programming for Ruby mailing list) if this doesn't make
sense.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Background jobs with #fork
@ 2012-04-12 23:04 3% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2012-04-12 23:04 UTC (permalink / raw)
To: unicorn list; +Cc: Patrik Wenger
Patrik Wenger <paddor@gmail.com> wrote:
> Thanks for the answers.
No problem.
> Isn't there another way to retrieve the right socket?
Actually, I think my proposed patch (in reply to Hongli) at
http://mid.gmane.org/20120412221022.GA20640@dcvr.yhbt.net
should fix your issue.
> > I can probably writeup a better explanation (perhaps on the usp.ruby
> > (Unix Systems Programming for Ruby mailing list) if this doesn't make
> > sense.
> Yeah I don't really understand this part. The "hanging" Unicorn worker
> can read another request because the client socket wasn't closed
> because it's still open in the child process? I would appreciate a
> better explanation, thank you.
Basically, fork() has a similar effect as dup() in that it creates
multiple references to the same kernel object (the client socket).
close() basically lowers the refcount of a kernel object, when the
refcount is zero, resources inside the kernel are freed. When
the refcount of a kernel object reaches zero, a shutdown(SHUT_RDWR)
is implied.
This works for 99% of Rack apps since they don't fork() nor call dup()
on the client socket, so refcount==1 when unicorn calls close(), leading
to unicorn setting refcount=0 upon close() => everything is freed.
However, since the client socket increments refcount via fork(),
close() in the parent (unicorn worker) no longer implies
shutdown(SHUT_RDWR).
parent timeline | child timeline
------------------------------------------------------------------
|
accept() -> sockfd created | (child doesn't exist, yet)
sockfd.refcount == 1 |
|
fork() | child exists, now
|
sockfd is shared by both processes now: sockfd.refcount == 2
if either the child or parent forks again: sockfd.recount += 1
|
close() => sockfd.recount -= 1 | child continues running
since sockfd.refcount == 1 at this point, the socket is still
considerd "alive" by the kernel. If the child calls close()
(or exits), sockfd.refcount is decremented again (and now
reaches zero).
Now, to write this as a commit message :>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 3%]
* Re: app error: Socket is not connected (Errno::ENOTCONN)
@ 2012-04-27 19:47 2% ` Matt Smith
0 siblings, 0 replies; 200+ results
From: Matt Smith @ 2012-04-27 19:47 UTC (permalink / raw)
To: Eric Wong; +Cc: mongrel-unicorn, George, Joel Nimety
> In production, I use a UNIX listener. (I am seeing no errors in production.)
(Again unicorn is not serving static files in production.)
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* unicorn + sleep = long wake up loading time
@ 2012-04-28 2:46 2% adam k
2012-04-28 4:40 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: adam k @ 2012-04-28 2:46 UTC (permalink / raw)
To: mongrel-unicorn
Hey,
I'm running unicorn behind nginx. If the site isn't accessed for a
long period of time, the next access of the site is loads very slowly,
sometimes even a 404 error shows. Subsequent requests have a normal
loading time until the site isn't accessed for a long period of time
again. How do I stop this behavior?
I looked at the code for unicorn and it seems to happen when unicorn
is looking to kill "lazy" workers and then if it chooses not to, it
creates a sleep period based on a timeout + 1. I also read the
documentation and searched the mailing list starting from 2011.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: unicorn + sleep = long wake up loading time
2012-04-28 2:46 2% unicorn + sleep = long wake up loading time adam k
@ 2012-04-28 4:40 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2012-04-28 4:40 UTC (permalink / raw)
To: unicorn list
adam k <adam.ffa@gmail.com> wrote:
> I'm running unicorn behind nginx. If the site isn't accessed for a
> long period of time, the next access of the site is loads very slowly,
> sometimes even a 404 error shows. Subsequent requests have a normal
> loading time until the site isn't accessed for a long period of time
> again. How do I stop this behavior?
unicorn is likely getting swapped out or a backend connection
(e.g. database connection) is getting timed out due to
inactivity.
* How much memory (RSS) is each worker using when the site is active?
* Is RSS noticeably lower when the site's been idle for a long time?
* How many unicorn worker processes?
(the above 3 questions also apply to nginx, but nginx usually uses
much less memory than unicorn)
* How much physical RAM do you have?
* What is the timeout value of unicorn? (default is 60s)
* What else is running on the machine?
* Anything mentioned in the unicorn stderr logs, or your application logs?
The updatedb cronjob is one major culprit of causing apps to swap out,
but it could be any number of things (backup jobs, rootkit scanners,
prelink, ...).
Try setting up a cronjob run curl to hit an endpoint of your app
every few minutes. Make sure that endpoint exercises whatever
non-local dependencies (hits the DB, etc...) your app has.
> I looked at the code for unicorn and it seems to happen when unicorn
> is looking to kill "lazy" workers and then if it chooses not to, it
> creates a sleep period based on a timeout + 1. I also read the
> documentation and searched the mailing list starting from 2011.
I think you're looking at the master process, which shouldn't
affect performance of the workers. The workers wake up whenever
there's socket activity, but it can take longer if they're swapped
out.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Address already in use
@ 2012-06-25 21:03 1% ` Manuel Palenciano Guerrero
0 siblings, 0 replies; 200+ results
From: Manuel Palenciano Guerrero @ 2012-06-25 21:03 UTC (permalink / raw)
To: unicorn list
Hi,
First, thanks Eric, Jérémy and Aaron for replying. I really appreciate it.
Yes Eric, I can see the line... "inherited addr=/tmp/unicorn.app.sock fd=..."
here is the full log
-------------------------------------------------
I, [2012-06-21T11:40:44.282224 #29212] INFO -- : inherited addr=/tmp/unicorn.sublimma_staging.sock fd=3
I, [2012-06-21T11:40:44.282480 #29212] INFO -- : Refreshing Gem list
master process ready
worker=0 ready
worker=1 ready
reaped #<Process::Status: pid=28683,exited(0)> worker=0
reaped #<Process::Status: pid=28684,exited(0)> worker=1
master complete
E, [2012-06-21T11:40:46.386486 #29401] ERROR -- : adding listener failed addr=/tmp/unicorn.sublimma_staging.sock (in use)
E, [2012-06-21T11:40:46.386669 #29401] ERROR -- : retrying in 0.5 seconds (4 tries left)
E, [2012-06-21T11:40:46.887724 #29401] ERROR -- : adding listener failed addr=/tmp/unicorn.sublimma_staging.sock (in use)
E, [2012-06-21T11:40:46.887832 #29401] ERROR -- : retrying in 0.5 seconds (3 tries left)
E, [2012-06-21T11:40:47.388813 #29401] ERROR -- : adding listener failed addr=/tmp/unicorn.sublimma_staging.sock (in use)
E, [2012-06-21T11:40:47.388894 #29401] ERROR -- : retrying in 0.5 seconds (2 tries left)
E, [2012-06-21T11:40:47.889878 #29401] ERROR -- : adding listener failed addr=/tmp/unicorn.sublimma_staging.sock (in use)
E, [2012-06-21T11:40:47.889957 #29401] ERROR -- : retrying in 0.5 seconds (1 tries left)
E, [2012-06-21T11:40:48.390939 #29401] ERROR -- : adding listener failed addr=/tmp/unicorn.sublimma_staging.sock (in use)
E, [2012-06-21T11:40:48.391020 #29401] ERROR -- : retrying in 0.5 seconds (0 tries left)
E, [2012-06-21T11:40:48.892002 #29401] ERROR -- : adding listener failed addr=/tmp/unicorn.sublimma_staging.sock (in use)
/var/www/sublimma/staging/shared/bundle/ruby/1.8/gems/unicorn-4.3.0/lib/unicorn/socket_helper.rb:140:in `initialize': Address already in use - /tmp/unicorn.sublimma_staging.sock (Errno::EADDRINUSE)
-------------------------------------------------
my unicorn.rb: https://gist.github.com/2991110
and my production_init.sh: http://unicorn.bogomips.org/examples/init.sh
I was planning on trying the following...
adding the killing of the old_pid to the before_fork(), as in...
old_pid = RAILS_ROOT + '/tmp/pids/unicorn.pid.oldbin'
if File.exists?(old_pid) && server.pid != old_pid
begin
Process.kill("QUIT", File.read(old_pid).to_i)
rescue Errno::ENOENT, Errno::ESRCH
# someone else did our job for us
end
end
and making the UPGRADE case in the "init.sh" to just...
--------------------------------------------------
upgrade)
sig USR2 && echo "Upgraded" && exit 0
echo >&2 "Couldn't upgrade, starting '$CMD' instead"
$CMD
;;
--------------------------------------------------
Regards and thanks again !
Manuel P.
On Jun 25, 2012, at 10:28 PM, Eric Wong wrote:
> Manuel Palenciano Guerrero <mpalenciano@gmail.com> wrote:
>> Hello there,
>>
>> I seem to have a problem with unix-sockets, and cannot see many people with the same situation when googling.
>>
>> The problem is when upgrading (USR2 + QUIT) our applications. I get the following error very frequently but not always.
>>
>> E, [2012-06-21T11:40:46.386486 #29401] ERROR -- : adding listener failed addr=/tmp/unicorn.app.sock (in use)
>
> You should've seen an INFO message saying something like:
>
> inherited addr=/tmp/unicorn.app.sock fd=...
>
> in your logs.
>
> Can you share your unicorn config? Are you using a before_exec hook
> at all and could that hook be clobbering ENV["UNICORN_FD"]?
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 1%]
* Re: Any signal other than -9 causes full CPU utilization by master unicorn process on FreeBSD
@ 2012-07-17 21:23 2% ` Mark Mccraw
2012-07-17 22:17 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Mark Mccraw @ 2012-07-17 21:23 UTC (permalink / raw)
To: unicorn list
On Jul 17, 2012, at 7:56 AM, Mark McCraw wrote:
>
> On Jul 16, 2012, at 10:05 PM, Eric Wong wrote:
>
>> Mark Mccraw <Mark.Mccraw@sas.com> wrote:
>>> Hi There!
>>>
>>> I'm having a devil of a time figuring out a weird issue I'm running
>>> into. I have unicorn configured to start 4 worker processes, and that
>>> works great. However, when it's time to cycle the app, everything
>>> goes haywire. By trial and error, I have narrowed it down to this:
>>> sending any signal to the master process other than SIGKILL fails
>>> miserably. No new master process is created, as described in the
>>> documentation, nothing happens to the existing workers, nothing gets
>>> written to any log, and if I run top -u, I can see that very quickly
>>> the master ramps up to 100% CPU utilization. This happens if I run
>>> 'kill -HUP <master pid>', 'kill -USR2 <master pid>', even 'kill -QUIT
>>> <master pid>'.
>>
>> This sounds like a Ruby/FreeBSD bug we've seen before. My script
>> in http://mid.gmane.org/20120201181445.GA31624@dcvr.yhbt.net should
>> reproduce the issue w/o unicorn.
>
> You are absolutely correct! Your script replicates the problem perfectly.
>
>>> ruby 1.9.3p0 (2011-10-30 revision 33570) [amd64-freebsd9]
>>
>> I think this is a Ruby bug that was fixed in 1.9.3-p30 according to
>> naruse:
>> http://mid.gmane.org/CAK6HhsppWVPijWLyZMwcKueYDT5sZroGv6ADXkgreht3aLfR9A@mail.gmail.com
>>
>> Since 1.9.3 p194 is the latest, can you try that out and confirm the
>> fix? I don't remember the other bug reported confirmed this issue was
>> fixed by upgrading Ruby.
>
> We're upgrading now to see what happens. I'm so glad you knew about this.
> There's no telling how long it would have taken me to question the ruby interpreter implementation, and
> since it's FreeBSD, I never would have found it by googling.
> Thanks for hours (days?) of my life back.
>
> Mark
>
Just to follow up and close out the thread - Eric's recollection was spot on.
We upgraded ruby on our FreeBSD server to the latest thing, and the problem completely disappeared. Thanks again!
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Any signal other than -9 causes full CPU utilization by master unicorn process on FreeBSD
2012-07-17 21:23 2% ` Mark Mccraw
@ 2012-07-17 22:17 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2012-07-17 22:17 UTC (permalink / raw)
To: unicorn list
Mark Mccraw <Mark.Mccraw@sas.com> wrote:
> On Jul 17, 2012, at 7:56 AM, Mark McCraw wrote:
> > We're upgrading now to see what happens. I'm so glad you knew about
> > this. There's no telling how long it would have taken me to
> > question the ruby interpreter implementation, and since it's
> > FreeBSD, I never would have found it by googling. Thanks for hours
> > (days?) of my life back.
>
> Just to follow up and close out the thread - Eric's recollection was
> spot on. We upgraded ruby on our FreeBSD server to the latest thing,
> and the problem completely disappeared. Thanks again!
Thanks for confirming this fix!
Fwiw, the Ruby core team probably uses/tests on GNU/Linux more than any
other platform. Bugs on less common development platforms (especially
w.r.t tricky thread/fork/signal handling issues) may go unnoticed
elsewhere. If you're focused on using Ruby + *BSD in a production
system, I suggest testing/fixing/reporting issues against the Ruby
development branches as much as possible before they hit production :)
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* SIGUSR2 from symlinked working directory
@ 2012-08-07 7:26 1% Danial Pearce
0 siblings, 0 replies; 200+ results
From: Danial Pearce @ 2012-08-07 7:26 UTC (permalink / raw)
To: mongrel-unicorn
Hi all,
I am battling with my unicorn deploy scripts at the moment. First it
was the "sleep" thing after sending a SIGUSR2 which I read about on
the mailing list just now so it seems I'm not the only one there. The
default of "2" was rarely working for me, upping that to 5 helps out
but still not ideal as it still fails occasionally, e.g. you might
sleep for the perfect amount of time to the point where the
unicorn.pid file disappears for a very short period of time.
Anyway, my main issue is that I am keeping 5 deploys, capistrano style
where it symlinks the current deploy as "current". I have made sure my
unicorn init scripts and configs all only use this "current" path,
nothing refers directly to the timestamped path invented by the deploy
system (chef in my case).
Yet, I am see that when I send SIGUSR2 to the unicorn master, the
following ends up in the log:
I, [2012-08-07T06:54:34.716138 #26569] INFO -- : executing
["/home/my_user/my_app/releases/20120807030020/gems/ruby/1.9.1/bin/unicorn_rails",
"-D", "-E", "production", "-c",
"/home/my_user/my_app/current/config/unicorn.rb",
{7=>#<Kgio::TCPServer:fd 7>}] (in
/home/my_user/my_app/releases/20120807065124)
I presume here the first path is the current running unicorn and the
second path is the supposed working directory. Neither of these are
correct, since it has gone and followed the "current" symlink when
executing that unicorn the first time, as well as when resolving the
working directory that second time. (Since you will notice they are 2
different releases). The first incorrect path is from when I first did
an initial start of the unicorn_rails command.
I say "incorrect path", those paths do in fact exist, for now. The
issue is when that first release that I used to "start" the unicorn
process becomes older than the 5th release and is deleted from disk.
Then the SIGUSR2 will fail saying it can't find that path to restart
the app. I then need to jump in and send a QUIT manually and start it
again.
I'm sure heaps of people must be doing this, so clearly I'm doing
something wrong, I just can't figure out what it is. As mentioned,
I've triple checked and I am not referencing the "20120807030020" path
anywhere, according to grep anyway. My config/init script most
definitely just reference /home/my_user/my_app/current.
Furthmore, this one just weirds me out, the "gems" path is also a
symlink, to /home/my_user/my_app/shared/gems, so why would unicorn be
following the symlink on /home/my_user/my_app/current but not on
/home/my_user/my_app/current/gems?
I'm not subscribed to this mailing list, so if you could CC me in
response that would be super. Although I will be checking the archives
anyway to make sure I don't miss a possible solution.
regards,
Danial
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 1%]
* Re: Unused Unicorn processes
@ 2012-08-22 18:16 2% ` Konstantin Gredeskoul
0 siblings, 0 replies; 200+ results
From: Konstantin Gredeskoul @ 2012-08-22 18:16 UTC (permalink / raw)
To: unicorn list
Thanks Eric! Appreciate well thought out answers.
We are actually using Rainbows also, in an project where long
server-side HTTP calls are part of the application design and are
necessary, and it's working out really well.
Our main web app mostly spends it's time in CPU cycles serving web
pages (about 80% of the request time), and 20% in database operations.
Thanks again
K
On Tue, Aug 21, 2012 at 2:11 AM, Eric Wong <normalperson@yhbt.net> wrote:
> Konstantin Gredeskoul <kig@wanelo.com> wrote:
>> Greetings!
>>
>> I have a question on optimal # of unicorn worker processes.
>>
>> We are running Unicorn 4.3.1 + Rails 3.2.6 (without threading), on
>> ruby 1.9.3-p194, hosted on SmartOS/Joyent.
>>
>> At the moment, unicorns are configured to start 30 worker processes. I
>> know this is a lot, and I am going to reduce this number. But in
>> trying to figure out what is a more appropriate number of workers to
>> run, I noticed something interesting that I couldn't explain.
>>
>> If I look at the process table on each machine (see top output below),
>> I notice that some unicorn processes are heavily used (and have
>> accumulated longer CPU times, as well as have grown their RAM usage
>> since boot time), but other processes (at the bottom of the top
>> output) appear to potentially not having been used at all. There are
>> several processes with RSS size of 143Mb, which I believe is unicorn
>> size before it processes any requests.
>>
>> What I am gathering from this, is that only 16 unicorn processes are
>> actually processing requests, while the rest are just sitting there
>> idle.
>>
>> Is this expected behavior?
>
> It's not _unexpected_ if your load is low. The OS scheduler does the
> load balancing itself as unicorn uses non-blocking accept(). The
> scheduler may choose busier processes because they're likely to be
> hotter in cache/memory.
>
> So you're probably getting better performance than if you had perfectly
> balanced traffic.
>
> Another possible explanation is something is killing some workers
> (timeout, or crash buts, check stderr logs to confirm). But even in a
> situation where workers never die, it's perfectly normal to see workers
> that never/rarely serve traffic.
>
>> This Joyent SmartMachine can burst up to 8 cores. Given that our web
>> requests spend only 80% of their time in ruby, I figured we could run
>> 10 unicorn processes for maximum efficiency. However seeing that 16
>> are actually used I am curious whether 16 is actually a better number.
>
> It depends what your app is waiting on.
>
> The only rule is your machine should not be swap thrashing under
> peak load.
>
> Extra workers won't be detrimental as long as you:
> 1) have enough memory
> 2) have enough backend resources (e.g DB connections)
>
> I know of deployments that run 30 processes/core because the app spends
> much time waiting on slow HTTP/DB requests[1]
>
> But if your app is completely bound by resources local to a machine (no
> external DB/HTTP/memcached/redis/etc requests), then a 1:1 worker:core
> (or even 1:1 worker:disk) relationship will work, too.
>
>
>
> [1] Arguably a multi-threaded or non-blocking server such as Rainbows!
> would be most efficient of machine resources if you're waiting on HTTP
> calls, but the developers decided human time was more important and
> did not want to worry about thread-safety.
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
--
Konstantin Gredeskoul
CTO :: Wanelo Inc
cell: (415) 265 1054
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Is a client uploading a file a slow client from unicorn's point of view?
@ 2012-10-09 23:54 2% ` Eric Wong
2012-10-10 6:59 2% ` Laas Toom
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2012-10-09 23:54 UTC (permalink / raw)
To: unicorn list
Laas Toom <laas@toom.ee> wrote:
> On 09.10.2012, at 23:03, Eric Wong <normalperson@yhbt.net> wrote:
> > Laas Toom <laas@toom.ee> wrote:
> >> Afterwards it will only handle out the file location and Rails can
> >> complete it's work a lot faster, freeing up workers.
> >>
> >> Unicorn won't even see the file and Rails has the responsibility to
> >> delete the file if it's invalid.
> >
> > I think the only problem with this approach is it won't work well on
> > setups where nginx is on separate machines from unicorn. Shared
> > storage would be required, but that ends up adding to network I/O,
> > too...
>
> But won't (almost) the same network I/O be evident anyway, because of
> nginx transferring the data to Unicorn over network (as they are on
> different machines)?
It depends on your shared storage implementation.
It'll likely be a win if the shared storage is on the same server as
nginx (but that might mean you can only have one nginx server). But I
think it'll be a loss if there needs to be multiple nginx servers (and
multiple unicorn servers)...
* With nginx_upload_module + shared storage:
nginx server ------ shared storage -------- unicorn server
------------------------------------------------------------------
1. sequential write to shared storage
2. file could remain cached do processing on file parts
on nginx server, even if remotely, network latency
we'll never need to read from reads (and possible cache
it again coherency checks on rereads)
3. unlink on error unlink/rename/copy on success
* Without nginx_upload_module:
nginx server -------------------------- unicorn server
------------------------------------------------------------------
1. sequential write of tempfile
2. sequential read of tempfile ----------> sequential write by Rack
3. unlink (able to free up cache) do processing on file locally
(no remote cache coherency
checks)
The benefit of this approach is there's only 2 components interacting at
any one time, and the network costs are paid in full up front
Basically, it's the message passing concurrency model vs shared
memory+locking. There's no clear winner, it just depends on the
situation. 99% of the time I get away with keeping everything on
one machine :)
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Is a client uploading a file a slow client from unicorn's point of view?
2012-10-09 23:54 2% ` Eric Wong
@ 2012-10-10 6:59 2% ` Laas Toom
0 siblings, 0 replies; 200+ results
From: Laas Toom @ 2012-10-10 6:59 UTC (permalink / raw)
To: unicorn list
On 10.10.2012, at 2:54, Eric Wong <normalperson@yhbt.net> wrote:
> Basically, it's the message passing concurrency model vs shared
> memory+locking. There's no clear winner, it just depends on the
> situation. 99% of the time I get away with keeping everything on
> one machine :)
Exactly.
For now we have nginx and unicorn on the same machine, both interacting w/ the storage over network, so there is no difference if the initial write is done by Rails or Nginx.
Later, Resque will pick up the background processing task, which again could be on totally separate machine(s).
And in the most complex scenario - there could be multiple Nginx, multiple Unicorn and multiple Resque nodes, each doing it's part. In such a situation, the Unicorn node can skip the network I/O, if it can base it's upload validation solely on filename and attrs, disregarding the file data itself.
Best,
Laas
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Combating nginx 499 HTTP responses during flash traffic scenario
@ 2012-10-30 21:37 2% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2012-10-30 21:37 UTC (permalink / raw)
To: unicorn list
Tom Burns <tom.burns@jadedpixel.com> wrote:
> On Mon, Oct 29, 2012 at 5:53 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Maybe this gross hack can work for you guys. It writes the first
> > chunk of the HTTP response header out immediately after reading
> > the request headers, and sends the rest once it gets the status...
>
> I tested the patch today and it does what we want, dropping
> connections before passing them to the rails app when the client has
> already disconnected.
>
> I also benchmarked the patch to see if it had a negligible performance
> hit and it did not. The cost was absorbed by the variation in speed
> of the other components in the stack (nginx & rails).
Good to know. Thanks for reporting back.
> I noticed on my computer applying the patch breaks
> test_rack_lint_big_put in the unicorn test suite. This may be just my
> issue as the test suite does not run cleanly anyways if I checkout
> origin/master.
The test suite in master should be passing cleanly, at least on a
GNU/Linux machine...
Yes, this hacky patch breaks some tests/internals and screws up
exception error/reporting badly.
> We'd prefer to not have to fork unicorn for this change. How do you
> feel about merging this or a derivative thereof? I can develop this
> further if you can send me what you'd want.
Sure thing!
I strongly prefer this to be optional behavior and off-by-default.
Also, I'm nearly certain two write()s is all that's needed and the
each_char is unnecessary syscall/packet overhead.
This will trigger bugs in badly-written HTTP clients/parsers (probably
some test suites :x) which assume a header comes in a single read().
For TCP users, I believe this requires both tcp_nodelay:false and
tcp_nopush:false to be completely reliable, so we need to enforce that
those if this option is in effect. The current each_char usage is
probably masking the tcp_nodelay:false requirement.
Thanks again.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* select(): Interrupted system call from curb when stopping unicorn
@ 2012-11-07 16:42 2% Graham Bleach
2012-11-07 21:11 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Graham Bleach @ 2012-11-07 16:42 UTC (permalink / raw)
To: mongrel-unicorn
Hi,
We've just migrated one of our rails applications from nginx/passenger
running on REE 1.8.7 Ubuntu 8.04 to unicorn running on MRI 1.9.3 on
Ubuntu 10.04. The app makes a number of calls to internal services
using curb.
Our deployment script stops unicorn by sending SIGQUIT to the unicorn
master, sleeps for a few seconds to ensure that HAProxy has taken the
node out of service and then starts unicorn again.
However, since the migration, we've started seeing a few errors on
each deploy, when the workers receive the QUIT from the master:
ERROR ( XXX) {"exception":{"class":"RuntimeError","message":"select():
Interrupted system
call","backtrace":["/path/bundle/ruby/1.9.1/gems/curb-0.7.18/lib/curl/easy.rb:39:in
`perform'","/path/bundle/ruby/1.9.1/gems/curb-0.7.18/lib/curl/easy.rb:39:in
`perform'","/path/bundle/ruby/1.9.1/gems/songkick-transport-0.1.4/lib/songkick/transport/curb.rb:50:in
...
If we're reading this correctly, curb is inside a select() call, gets
interrupted by the QUIT signal and doesn't retry the select() and the
process terminates, causing an error for the unfortunate end user.
Our options for stopping our users seeing error pages seem to be:
1) Patch curb to retry the select() if errno = EINTR as per
https://github.com/taf2/curb/issues/117
We're not confident enough in our ability to do this properly, but
it's probably the most correct way to solve this.
2) Do something to make sure that requests have all completed before
sending the QUIT.
For example, we could change our deployment to make the health check
url that HAProxy polls start returning a non-200 return code, wait
until all connections have completed and then send the QUIT.
The main issue with this is that short of polling the HAProxy status
page, we're not entirely sure how to verify that all connections have
completed.
Does anyone have any suggested solutions or comments?
Regards,
Graham
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: select(): Interrupted system call from curb when stopping unicorn
2012-11-07 16:42 2% select(): Interrupted system call from curb when stopping unicorn Graham Bleach
@ 2012-11-07 21:11 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2012-11-07 21:11 UTC (permalink / raw)
To: unicorn list
Graham Bleach <graham@darkskills.org.uk> wrote:
> Hi,
>
> We've just migrated one of our rails applications from nginx/passenger
> running on REE 1.8.7 Ubuntu 8.04 to unicorn running on MRI 1.9.3 on
> Ubuntu 10.04. The app makes a number of calls to internal services
> using curb.
>
> Our deployment script stops unicorn by sending SIGQUIT to the unicorn
> master, sleeps for a few seconds to ensure that HAProxy has taken the
> node out of service and then starts unicorn again.
>
> However, since the migration, we've started seeing a few errors on
> each deploy, when the workers receive the QUIT from the master:
>
> ERROR ( XXX) {"exception":{"class":"RuntimeError","message":"select():
> Interrupted system
> call","backtrace":["/path/bundle/ruby/1.9.1/gems/curb-0.7.18/lib/curl/easy.rb:39:in
> `perform'","/path/bundle/ruby/1.9.1/gems/curb-0.7.18/lib/curl/easy.rb:39:in
> `perform'","/path/bundle/ruby/1.9.1/gems/songkick-transport-0.1.4/lib/songkick/transport/curb.rb:50:in
> ...
>
> If we're reading this correctly, curb is inside a select() call, gets
> interrupted by the QUIT signal and doesn't retry the select() and the
> process terminates, causing an error for the unfortunate end user.
>
> Our options for stopping our users seeing error pages seem to be:
>
> 1) Patch curb to retry the select() if errno = EINTR as per
> https://github.com/taf2/curb/issues/117
Taking a quick look at the curb source, I see no reason it implements
curb_select() itself instead of just using rb_thread_select() (or
rb_thread_fd_select(), the latter is recommended for new Rubies as
it handles high-numbered FDs better).
Any rb_thread*select() function implemented by Ruby is already aware of
multithreading and (for 1.9) calls rb_thread_blocking_region()
internally.
For now, you can probably just: #undef HAVE_RB_THREAD_BLOCKING_REGION
(Feel free to forward my comments to the curb folks, I don't like
logging into websites)
> We're not confident enough in our ability to do this properly, but
> it's probably the most correct way to solve this.
Practice :)
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Testing Unicorn on Rubinius
@ 2012-11-13 0:34 2% mike
0 siblings, 0 replies; 200+ results
From: mike @ 2012-11-13 0:34 UTC (permalink / raw)
To: mongrel-unicorn
Hey Eric,
I do have the isolate gem installed. In a number of gemsets as it
were. Here is a gem list:
mike@sm-mike-thinkpad:~/workspace/rubinius_testing/unicorn-4.4.0$ gem list
*** LOCAL GEMS ***
bundler (1.2.1)
isolate (3.2.2)
rack (1.4.1)
rake (0.9.2.2)
rubygems-bundler (1.1.0)
rvm (1.11.3.5)
Here is a "find" for isolate running against my .rvm directory:
mike@sm-mike-thinkpad:~/workspace/rubinius_testing/unicorn-4.4.0$ find /home/mike/.rvm/ -name isolate-3.2.2
/home/mike/.rvm/gems/rbx-2.0.testing@global/doc/isolate-3.2.2
/home/mike/.rvm/gems/rbx-2.0.testing@global/gems/isolate-3.2.2
/home/mike/.rvm/gems/rbx-2.0.testing/doc/isolate-3.2.2
/home/mike/.rvm/gems/rbx-2.0.testing/gems/isolate-3.2.2
/home/mike/.rvm/gems/rbx-2.0.testing@rubinius_testing/doc/isolate-3.2.2
/home/mike/.rvm/gems/rbx-2.0.testing@rubinius_testing/gems/isolate-3.2.2
and lastly, here is the setting I have for the "RUBYLIB" variable in my local.mk:
RUBYLIB=/home/mike/.rvm/gems/rbx-2.0.testing/
(FYI: this is injected on line 8 of your sample local.mk that ships with the project)
If I run irb, I cannot require 'isolate' without requiring 'rubygems' even with it in all these places. I assume there is some kind of path / inclusion issue I am facing here and I don't quite know how to go about getting around it. Perhaps I have RUBYLIB set wrong, or I need to try to approach this completely differently and not from rvm at all.
Thank you so much for all your efforts in trying to help me get going on this, they are very appreciated.
Also please let me know if there is a log output, setting/configuration file or something else that having access to/knowledge of would help.
Thanks again,
-Mike Thompson
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Combating nginx 499 HTTP responses during flash traffic scenario
@ 2012-11-29 15:52 1% ` Tom Burns
2012-11-29 20:41 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Tom Burns @ 2012-11-29 15:52 UTC (permalink / raw)
To: unicorn list
On Tue, Nov 6, 2012 at 4:23 PM, Eric Wong <normalperson@yhbt.net> wrote:
>
> Tom Burns <tom.burns@jadedpixel.com> wrote:
> > On Mon, Nov 5, 2012 at 6:48 AM, Eric Wong <normalperson@yhbt.net> wrote:
> > > We can wait until you can report on how this change improves your
> > > situation before fleshing this out. Real-world results are always
> > > good to have :>
> >
> > Agreed.
> >
> > We're going to be testing this this week so I should have some results
> > to share by the weekend. We only experience the problem during major
> > sales so it may or may not manifest without our traffic generation
> > tool.
>
> Holidays are coming up, should be lots of traffic :)
So we just finished the US Black Friday / Cyber Monday weekend running
unicorn forked with the last version of the patch I had sent you. It
worked splendidly and helped us handle huge flash sales without
increased response time over the weekend.
Whereas in previous flash traffic scenarios we would see the number of
HTTP 499 responses grow past the number of real HTTP 200 responses,
over the weekend we saw no growth in 499s during flash sales.
Unexpectedly the patch also helped us ward off a DoS attack where the
attackers were disconnecting immediately after making a request.
I've attached the patch again, with your last comments addressed. Let
me know if there's anything else.
Thanks again for your help in this matter.
Cheers,
Tom
>From 89c8c51355c75b0196469812580443fba390632e Mon Sep 17 00:00:00 2001
From: Tom Burns <tom.burns@jadedpixel.com>
Date: Tue, 30 Oct 2012 16:22:21 -0400
Subject: [PATCH] Begin writing HTTP request headers early to detect
disconnected clients
This patch checks incoming connections and avoids calling the application
if the connection has been closed.
It works by sending the beginning of the HTTP response before calling
the application to see if the socket can successfully be written to.
By enabling this feature users can avoid wasting application rendering
time only to find the connection is closed when attempting to write, and
throwing out the result.
When a client disconnects while being queued or processed, Nginx will log
HTTP response 499 but the application will log a 200.
Enabling this feature will minimize the time window during which the problem
can arise.
The feature is disabled by default and can be enabled by adding
'check_client_connection true' to the unicorn config.
---
examples/unicorn.conf.rb | 6 ++++++
ext/unicorn_http/unicorn_http.rl | 4 +++-
lib/unicorn/configurator.rb | 25 +++++++++++++++++++++++++
lib/unicorn/const.rb | 12 ++++++++----
lib/unicorn/http_request.rb | 19 +++++++++++++++++++
lib/unicorn/http_response.rb | 5 +++--
lib/unicorn/http_server.rb | 22 ++++++++++++++++++++--
test/exec/test_exec.rb | 7 ++++++-
test/unit/test_configurator.rb | 24 ++++++++++++++++++++++++
9 files changed, 114 insertions(+), 10 deletions(-)
diff --git a/examples/unicorn.conf.rb b/examples/unicorn.conf.rb
index 0238043..1f4c9c0 100644
--- a/examples/unicorn.conf.rb
+++ b/examples/unicorn.conf.rb
@@ -46,6 +46,12 @@ preload_app true
GC.respond_to?(:copy_on_write_friendly=) and
GC.copy_on_write_friendly = true
+# Enable this flag to have unicorn test client connections by writing the
+# beginning of the HTTP headers before calling the application. This
+# prevents calling the application for connections that have disconnected
+# while queued.
+check_client_connection false
+
before_fork do |server, worker|
# the following is highly recomended for Rails + "preload_app true"
# as there's no need for the master process to hold a connection
diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
index 96dcf83..1a8003f 100644
--- a/ext/unicorn_http/unicorn_http.rl
+++ b/ext/unicorn_http/unicorn_http.rl
@@ -115,7 +115,7 @@ struct http_parser {
} len;
};
-static ID id_clear, id_set_backtrace;
+static ID id_clear, id_set_backtrace, id_response_start_sent;
static void finalize_header(struct http_parser *hp);
@@ -626,6 +626,7 @@ static VALUE HttpParser_clear(VALUE self)
http_parser_init(hp);
rb_funcall(hp->env, id_clear, 0);
+ rb_ivar_set(self, id_response_start_sent, Qfalse);
return self;
}
@@ -1031,6 +1032,7 @@ void Init_unicorn_http(void)
SET_GLOBAL(g_http_connection, "CONNECTION");
id_clear = rb_intern("clear");
id_set_backtrace = rb_intern("set_backtrace");
+ id_response_start_sent = rb_intern("@response_start_sent");
init_unicorn_httpdate();
}
#undef SET_GLOBAL
diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb
index 89cbf5c..9752cdd 100644
--- a/lib/unicorn/configurator.rb
+++ b/lib/unicorn/configurator.rb
@@ -45,6 +45,7 @@ class Unicorn::Configurator
},
:pid => nil,
:preload_app => false,
+ :check_client_connection => false,
:rewindable_input => true, # for Rack 2.x: (Rack::VERSION[0] <= 1),
:client_body_buffer_size => Unicorn::Const::MAX_BODY,
:trust_x_forwarded => true,
@@ -96,6 +97,18 @@ class Unicorn::Configurator
if ready_pipe = RACKUP.delete(:ready_pipe)
server.ready_pipe = ready_pipe
end
+ if set[:check_client_connection]
+ set[:listeners].each do |address|
+ if set[:listener_opts][address][:tcp_nopush] == true
+ raise ArgumentError,
+ "check_client_connection is incompatible with tcp_nopush:true"
+ end
+ if set[:listener_opts][address][:tcp_nodelay] == true
+ raise ArgumentError,
+ "check_client_connection is incompatible with tcp_nodelay:true"
+ end
+ end
+ end
set.each do |key, value|
value == :unset and next
skip.include?(key) and next
@@ -454,6 +467,18 @@ class Unicorn::Configurator
set_int(:client_body_buffer_size, bytes, 0)
end
+ # When enabled, unicorn will check the client connection by writing
+ # the beginning of the HTTP headers before calling the application.
+ #
+ # This will prevent calling the application for clients who have
+ # disconnected while their connection was queued.
+ #
+ # This option cannot be used in conjunction with tcp_nodelay or
+ # tcp_nopush.
+ def check_client_connection(bool)
+ set_bool(:check_client_connection, bool)
+ end
+
# Allow redirecting $stderr to a given path. Unlike doing this from
# the shell, this allows the unicorn process to know the path its
# writing to and rotate the file if it is used for logging. The
diff --git a/lib/unicorn/const.rb b/lib/unicorn/const.rb
index b3d8d71..f0c4c12 100644
--- a/lib/unicorn/const.rb
+++ b/lib/unicorn/const.rb
@@ -29,12 +29,16 @@ module Unicorn::Const
# :stopdoc:
# common errors we'll send back
- ERROR_400_RESPONSE = "HTTP/1.1 400 Bad Request\r\n\r\n"
- ERROR_414_RESPONSE = "HTTP/1.1 414 Request-URI Too Long\r\n\r\n"
- ERROR_413_RESPONSE = "HTTP/1.1 413 Request Entity Too Large\r\n\r\n"
- ERROR_500_RESPONSE = "HTTP/1.1 500 Internal Server Error\r\n\r\n"
+ ERROR_400_RESPONSE = "400 Bad Request\r\n\r\n"
+ ERROR_414_RESPONSE = "414 Request-URI Too Long\r\n\r\n"
+ ERROR_413_RESPONSE = "413 Request Entity Too Large\r\n\r\n"
+ ERROR_500_RESPONSE = "500 Internal Server Error\r\n\r\n"
+
EXPECT_100_RESPONSE = "HTTP/1.1 100 Continue\r\n\r\n"
+ EXPECT_100_RESPONSE_SUFFIXED = "100 Continue\r\n\r\nHTTP/1.1 "
+ HTTP_RESPONSE_START = ['HTTP', '/1.1 ']
HTTP_EXPECT = "HTTP_EXPECT"
+
# :startdoc:
end
diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index a0435d6..79ead2e 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -22,11 +22,14 @@ class Unicorn::HttpParser
NULL_IO = StringIO.new("")
+ attr_accessor :response_start_sent
+
# :stopdoc:
# A frozen format for this is about 15% faster
REMOTE_ADDR = 'REMOTE_ADDR'.freeze
RACK_INPUT = 'rack.input'.freeze
@@input_class = Unicorn::TeeInput
+ @@check_client_connection = false
def self.input_class
@@input_class
@@ -35,6 +38,15 @@ class Unicorn::HttpParser
def self.input_class=(klass)
@@input_class = klass
end
+
+ def self.check_client_connection
+ @@check_client_connection
+ end
+
+ def self.check_client_connection=(bool)
+ @@check_client_connection = bool
+ end
+
# :startdoc:
# Does the majority of the IO processing. It has been written in
@@ -70,6 +82,13 @@ class Unicorn::HttpParser
# an Exception thrown from the parser will throw us out of the loop
false until add_parse(socket.kgio_read!(16384))
end
+
+ # detect if the socket is valid by writing a partial response:
+ if @@check_client_connection && headers?
+ @response_start_sent = true
+ Unicorn::Const::HTTP_RESPONSE_START.each { |c| socket.write(c) }
+ end
+
e[RACK_INPUT] = 0 == content_length ?
NULL_IO : @@input_class.new(socket, self)
e.merge!(DEFAULTS)
diff --git a/lib/unicorn/http_response.rb b/lib/unicorn/http_response.rb
index b781e20..9c2bacc 100644
--- a/lib/unicorn/http_response.rb
+++ b/lib/unicorn/http_response.rb
@@ -18,11 +18,12 @@ module Unicorn::HttpResponse
CRLF = "\r\n"
# writes the rack_response to socket as an HTTP response
- def http_response_write(socket, status, headers, body)
+ def http_response_write(socket, status, headers, body,
response_start_sent=false)
status = CODES[status.to_i] || status
+ http_response_start = response_start_sent ? '' : 'HTTP/1.1 '
if headers
- buf = "HTTP/1.1 #{status}\r\n" \
+ buf = "#{http_response_start}#{status}\r\n" \
"Date: #{httpdate}\r\n" \
"Status: #{status}\r\n" \
"Connection: close\r\n"
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 13df55a..8729830 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -17,6 +17,7 @@ class Unicorn::HttpServer
:listener_opts, :preload_app,
:reexec_pid, :orig_app, :init_listeners,
:master_pid, :config, :ready_pipe, :user
+
attr_reader :pid, :logger
include Unicorn::SocketHelper
include Unicorn::HttpResponse
@@ -355,6 +356,14 @@ class Unicorn::HttpServer
Unicorn::HttpParser.trust_x_forwarded = bool
end
+ def check_client_connection
+ Unicorn::HttpRequest.check_client_connection
+ end
+
+ def check_client_connection=(bool)
+ Unicorn::HttpRequest.check_client_connection = bool
+ end
+
private
# wait for a signal hander to wake us up and then consume the pipe
@@ -524,23 +533,32 @@ class Unicorn::HttpServer
Unicorn.log_error(@logger, "app error", e)
Unicorn::Const::ERROR_500_RESPONSE
end
+ msg = "HTTP/1.1 #{msg}" unless @request.response_start_sent
client.kgio_trywrite(msg)
client.close
rescue
end
+ def expect_100_response
+ if @request.response_start_sent
+ Unicorn::Const::EXPECT_100_RESPONSE_SUFFIXED
+ else
+ Unicorn::Const::EXPECT_100_RESPONSE
+ end
+ end
+
# once a client is accepted, it is processed in its entirety here
# in 3 easy steps: read request, call app, write app response
def process_client(client)
status, headers, body = @app.call(env = @request.read(client))
if 100 == status.to_i
- client.write(Unicorn::Const::EXPECT_100_RESPONSE)
+ client.write(expect_100_response)
env.delete(Unicorn::Const::HTTP_EXPECT)
status, headers, body = @app.call(env)
end
@request.headers? or headers = nil
- http_response_write(client, status, headers, body)
+ http_response_write(client, status, headers, body,
@request.response_start_sent)
client.shutdown # in case of fork() in Rack app
client.close # flush and uncork socket immediately, no keepalive
rescue => e
diff --git a/test/exec/test_exec.rb b/test/exec/test_exec.rb
index b30a3d6..1cee2b7 100644
--- a/test/exec/test_exec.rb
+++ b/test/exec/test_exec.rb
@@ -871,13 +871,14 @@ EOF
wait_for_death(pid)
end
- def hup_test_common(preload)
+ def hup_test_common(preload, check_client=false)
File.open("config.ru", "wb") { |fp| fp.syswrite(HI.gsub("HI", '#$$')) }
pid_file = Tempfile.new('pid')
ucfg = Tempfile.new('unicorn_test_config')
ucfg.syswrite("listen '#@addr:#@port'\n")
ucfg.syswrite("pid '#{pid_file.path}'\n")
ucfg.syswrite("preload_app true\n") if preload
+ ucfg.syswrite("check_client_connection true\n") if check_client
ucfg.syswrite("stderr_path 'test_stderr.#$$.log'\n")
ucfg.syswrite("stdout_path 'test_stdout.#$$.log'\n")
pid = xfork {
@@ -942,6 +943,10 @@ EOF
hup_test_common(false)
end
+ def test_check_client_hup
+ hup_test_common(false, true)
+ end
+
def test_default_listen_hup_holds_listener
default_listen_lock do
res, pid_path = default_listen_setup
diff --git a/test/unit/test_configurator.rb b/test/unit/test_configurator.rb
index c19c427..b9c22d7 100644
--- a/test/unit/test_configurator.rb
+++ b/test/unit/test_configurator.rb
@@ -139,6 +139,30 @@ class TestConfigurator < Test::Unit::TestCase
end
end
+ def test_check_client_connection
+ tmp = Tempfile.new('unicorn_config')
+ test_struct = TestStruct.new
+ tmp.syswrite("check_client_connection true\n")
+
+ assert_nothing_raised do
+ Unicorn::Configurator.new(:config_file => tmp.path).commit!(test_struct)
+ end
+
+ assert test_struct.check_client_connection
+ end
+
+ def test_check_client_connection_with_tcp_bad
+ tmp = Tempfile.new('unicorn_config')
+ test_struct = TestStruct.new
+ listener = "127.0.0.1:12345"
+ tmp.syswrite("check_client_connection true\n")
+ tmp.syswrite("listen '#{listener}', :tcp_nopush => true\n")
+
+ assert_raises(ArgumentError) do
+ Unicorn::Configurator.new(:config_file => tmp.path).commit!(test_struct)
+ end
+ end
+
def test_after_fork_proc
test_struct = TestStruct.new
[ proc { |a,b| }, Proc.new { |a,b| }, lambda { |a,b| } ].each do |my_proc|
--
1.7.10.2 (Apple Git-33)
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 1%]
* Re: Combating nginx 499 HTTP responses during flash traffic scenario
2012-11-29 15:52 1% ` Tom Burns
@ 2012-11-29 20:41 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2012-11-29 20:41 UTC (permalink / raw)
To: unicorn list
Tom Burns <tom.burns@jadedpixel.com> wrote:
> So we just finished the US Black Friday / Cyber Monday weekend running
> unicorn forked with the last version of the patch I had sent you. It
> worked splendidly and helped us handle huge flash sales without
> increased response time over the weekend.
>
> Whereas in previous flash traffic scenarios we would see the number of
> HTTP 499 responses grow past the number of real HTTP 200 responses,
> over the weekend we saw no growth in 499s during flash sales.
>
> Unexpectedly the patch also helped us ward off a DoS attack where the
> attackers were disconnecting immediately after making a request.
That is absolutely wonderful to hear. I've amended your commit message
with the above quoted portion.
> I've attached the patch again, with your last comments addressed. Let
> me know if there's anything else.
>
> Thanks again for your help in this matter.
Thank _you_ for the patch, documentation and most importantly:
testing with real traffic.
I fixed up some minor line-wrapping, signed-off, and added your
quote above to the commit message. Pushed as
commit 5c700fc2cf398848ddcf71a2aa3f0f2a6563e87b
to git://bogomips.org/unicorn.git
I'll tag and push a 4.5.0.preview1 gem soon
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: When a client terminates a connection
@ 2012-11-30 9:15 2% ` Andrew Stewart
2012-11-30 19:26 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Andrew Stewart @ 2012-11-30 9:15 UTC (permalink / raw)
To: unicorn list
On 30 Nov 2012, at 00:04, Eric Wong wrote:
> Iñaki Baz Castillo <ibc@aliax.net> wrote:
>> I don't understand why the application on top of the HTTP/TCP layer
>> (so the Rails app) should be interrupted because the TCP connection is
>> closed prematurely. The HTTP layer (which involves a single TCP
>> connection for each HTTP request/response pair) is a layer below the
>> application layer, should not disturb the Rails app IMHO.
>>
>> Is this a design issue of Rails?
>
> I suspect the the only possibility is when Rails is reading the
> multipart response in a largish upload (any upload request which
> can't fit in socket buffers).
On the three occasions I've had this problem it's been a tiny request.
The user clicks an ordinary Rails delete link. This POSTs to a URL with two parameters: an authenticity token and the _method.
The corresponding destroy action looks like:
def destroy
client = Client.find params[:id] # 1 (ignores soft-deleted records)
client.soft_delete # 2 (sets a deleted_at timestamp)
client.brands.each(&:destroy) # 3 (a client has many brands)
redirect_to clients_path
end
Nginx logged:
x.xxx.xx.xx - - [27/Nov/2012:14:40:28 +0000] "POST /clients/2248 HTTP/1.1" 499 0 "https://example.com/clients/2248/edit" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
x.xxx.xx.xx - - [27/Nov/2012:14:40:29 +0000] "POST /clients/2248 HTTP/1.1" 404 592 "https://example.com/companies/2248/edit" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
You can see the first request was terminated by the user. Hot on its heels the user tried again and got a 404.
The 404 was triggered by step 1 above: the first request had successfully soft-deleted the client. However the client's brands were still in the database. The first request had executed steps 1 and 2 but not step 3.
Unicorn didn't log anything. Rails didn't log the first request but did log the second.
I understand the steps to mitigate the problem...but I remain confused about why the Rails app was interrupted because the TCP connection was closed prematurely.
Yours,
Andrew Stewart
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* using unicorn with logging-rails gem
@ 2012-11-30 17:20 2% Yoav Aner
2012-11-30 19:15 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Yoav Aner @ 2012-11-30 17:20 UTC (permalink / raw)
To: mongrel-unicorn
Hope this is the right place to ask questions about unicorn. I'm not
on the mailing list though, so please CC my email address.
Since moving from phusion passenger to Unicorn, it seems like one
functionality of the logging-rails gem
(https://github.com/TwP/logging-rails) is (half) broken.
Logging-rails supports an email appender, which aggregates log
messages (typically ERROR and above) and then emails them. However,
since moving to Unicorn we stopped getting those emails.
The interesting thing however, is that when we USR2 signal our unicorn
and it restarts itself, then those emails somehow get flushed out and
get sent all at once... I tried sending USR1 signal to see if it has
something to do with log flushing, but this does not seem to make any
difference.
Any ideas how to investigate this / make it work again??
Cheers
Yoav
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: using unicorn with logging-rails gem
2012-11-30 17:20 2% using unicorn with logging-rails gem Yoav Aner
@ 2012-11-30 19:15 0% ` Eric Wong
2012-11-30 19:59 0% ` Yoav Aner
[not found] ` <CA+mA6PgTGcyLy2Dua9a6gDWhRwsQLtHEyk17fhBE8DW1TkO5EA@mail.gmail.com>
0 siblings, 2 replies; 200+ results
From: Eric Wong @ 2012-11-30 19:15 UTC (permalink / raw)
To: unicorn list; +Cc: Yoav Aner
Yoav Aner <yoav@kenhub.com> wrote:
> Hope this is the right place to ask questions about unicorn. I'm not
> on the mailing list though, so please CC my email address.
No problem!
> Since moving from phusion passenger to Unicorn, it seems like one
> functionality of the logging-rails gem
> (https://github.com/TwP/logging-rails) is (half) broken.
>
> Logging-rails supports an email appender, which aggregates log
> messages (typically ERROR and above) and then emails them. However,
> since moving to Unicorn we stopped getting those emails.
>
> The interesting thing however, is that when we USR2 signal our unicorn
> and it restarts itself, then those emails somehow get flushed out and
> get sent all at once... I tried sending USR1 signal to see if it has
> something to do with log flushing, but this does not seem to make any
> difference.
USR1 only affects logging to regular files, so is unlikely to help
in your case.
Are you using preload_app true or false? If true, perhaps some shared
resource (socket or temporary file) isn't being handled correctly?
> Any ideas how to investigate this / make it work again??
I'm not familiar with the "logging" or "logging-rails" gems.
I took a brief look at them but couldn't figure anything out,
can you ask the author of those gems?
If you do figure something out, please let the list know so it can
possibly help others in the future.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: When a client terminates a connection
2012-11-30 9:15 2% ` Andrew Stewart
@ 2012-11-30 19:26 0% ` Eric Wong
2012-12-03 14:42 0% ` Andrew Stewart
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2012-11-30 19:26 UTC (permalink / raw)
To: unicorn list
Andrew Stewart <boss@airbladesoftware.com> wrote:
> On 30 Nov 2012, at 00:04, Eric Wong wrote:
> > Iñaki Baz Castillo <ibc@aliax.net> wrote:
> >> I don't understand why the application on top of the HTTP/TCP layer
> >> (so the Rails app) should be interrupted because the TCP connection is
> >> closed prematurely. The HTTP layer (which involves a single TCP
> >> connection for each HTTP request/response pair) is a layer below the
> >> application layer, should not disturb the Rails app IMHO.
> >>
> >> Is this a design issue of Rails?
> >
> > I suspect the the only possibility is when Rails is reading the
> > multipart response in a largish upload (any upload request which
> > can't fit in socket buffers).
>
> On the three occasions I've had this problem it's been a tiny request.
>
> The user clicks an ordinary Rails delete link. This POSTs to a URL
> with two parameters: an authenticity token and the _method.
>
> The corresponding destroy action looks like:
>
> def destroy
> client = Client.find params[:id] # 1 (ignores soft-deleted records)
> client.soft_delete # 2 (sets a deleted_at timestamp)
> client.brands.each(&:destroy) # 3 (a client has many brands)
> redirect_to clients_path
> end
>
> Nginx logged:
>
> x.xxx.xx.xx - - [27/Nov/2012:14:40:28 +0000] "POST /clients/2248 HTTP/1.1" 499 0 "https://example.com/clients/2248/edit" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
> x.xxx.xx.xx - - [27/Nov/2012:14:40:29 +0000] "POST /clients/2248 HTTP/1.1" 404 592 "https://example.com/companies/2248/edit" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
>
> You can see the first request was terminated by the user. Hot on its
> heels the user tried again and got a 404.
Weird, and this is the only nginx instance that saw this request?
Also, according to your logs above, the two requests above came from
different Referer pages, so perhaps there were more requests involved
that hit a different nginx?
> The 404 was triggered by step 1 above: the first request had
> successfully soft-deleted the client. However the client's brands
> were still in the database. The first request had executed steps 1
> and 2 but not step 3.
>
> Unicorn didn't log anything. Rails didn't log the first request but
> did log the second.
Odd. It's been a long time since I looked at Rails; but doesn't Rails
log when a request _starts_? Also, doesn't Rails log all DB queries?
Or, by any chance, do you have query logging enabled in your DB to track
this down?
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: using unicorn with logging-rails gem
2012-11-30 19:15 0% ` Eric Wong
@ 2012-11-30 19:59 0% ` Yoav Aner
[not found] ` <CA+mA6PgTGcyLy2Dua9a6gDWhRwsQLtHEyk17fhBE8DW1TkO5EA@mail.gmail.com>
1 sibling, 0 replies; 200+ results
From: Yoav Aner @ 2012-11-30 19:59 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn list
Hi Eric,
Thanks a bunch for getting back so quickly on this.
I followed your suggestion and tried with `preload_app = false` and
looks like this seems to fix this problem! Any idea what can go wrong
when it's set to true or how I can try go about fixing this??
I haven't yet contacted the logging / logging-rails project. Perhaps
that's a good idea. Considering the gem did/does work fine on my dev
environment and with phusion passenger (and now it seems also with
Unicorn, albeit with preload_app = false), I wasn't sure whether the
problem is with this gem or elsewhere.
Any tips on how to investigate this further or resolve this, or what
information I can give the gem maintainer(s) would be much
appreciated.
Cheers
Yoav
On 30 November 2012 20:15, Eric Wong <normalperson@yhbt.net> wrote:
>
> Yoav Aner <yoav@kenhub.com> wrote:
> > Hope this is the right place to ask questions about unicorn. I'm not
> > on the mailing list though, so please CC my email address.
>
> No problem!
>
> > Since moving from phusion passenger to Unicorn, it seems like one
> > functionality of the logging-rails gem
> > (https://github.com/TwP/logging-rails) is (half) broken.
> >
> > Logging-rails supports an email appender, which aggregates log
> > messages (typically ERROR and above) and then emails them. However,
> > since moving to Unicorn we stopped getting those emails.
> >
> > The interesting thing however, is that when we USR2 signal our unicorn
> > and it restarts itself, then those emails somehow get flushed out and
> > get sent all at once... I tried sending USR1 signal to see if it has
> > something to do with log flushing, but this does not seem to make any
> > difference.
>
> USR1 only affects logging to regular files, so is unlikely to help
> in your case.
>
> Are you using preload_app true or false? If true, perhaps some shared
> resource (socket or temporary file) isn't being handled correctly?
>
> > Any ideas how to investigate this / make it work again??
>
> I'm not familiar with the "logging" or "logging-rails" gems.
> I took a brief look at them but couldn't figure anything out,
> can you ask the author of those gems?
>
> If you do figure something out, please let the list know so it can
> possibly help others in the future.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: using unicorn with logging-rails gem
@ 2012-11-30 20:17 3% ` Yoav Aner
2012-11-30 20:55 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Yoav Aner @ 2012-11-30 20:17 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn list
Thanks Eric,
I think I might have found a solution :)
There was a hint in the config file I had for the logging-rails gem:
if defined?(PhusionPassenger)
PhusionPassenger.on_event(:starting_worker_process) do |forked|
Logging.reopen if forked
end
end
I simply added `Logging.reopen` to the after_fork block and now it
looks like it's working again!
Not sure I still fully (or even partially) understand what's going on
there, but at least it's working.
Thanks again for your help.
Cheers
Yoav
On 30 November 2012 21:05, Eric Wong <normalperson@yhbt.net> wrote:
> Yoav Aner <yoav@kenhub.com> wrote:
>> Hi Eric,
>>
>> Thanks a bunch for getting back so quickly on this.
>>
>> I followed your suggestion and tried with `preload_app = false` and looks
>> like this seems to fix this problem! Any idea what can go wrong when it's
>> set to true or how I can try go about fixing this??
>
> preload_app is false by default because it's the most likely to work
> for all gems.
>
> I suspect there's a shared resource (file/socket) or some cached value
> that's initialized before fork and loses state after forking.
>
> It's a fairly common issue with database libraries
>
>> I haven't yet contacted the logging / logging-rails project. Perhaps that's
>> a good idea. Considering the gem did/does work fine on my dev environment
>> and with phusion passenger (and now it seems also with Unicorn, albeit with
>> preload_app = false), I wasn't sure whether the problem is with this gem or
>> elsewhere.
>>
>> Any tips on how to investigate this further or resolve this, or what
>> information I can give the gem maintainer(s) would be much appreciated.
>
> I would definitely contact the maintainer of logging/logging-rails
> on how to reinitialize any state after forking.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 3%]
* Re: using unicorn with logging-rails gem
2012-11-30 20:17 3% ` Yoav Aner
@ 2012-11-30 20:55 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2012-11-30 20:55 UTC (permalink / raw)
To: Yoav Aner; +Cc: unicorn list
Yoav Aner <yoav@kenhub.com> wrote:
> I simply added `Logging.reopen` to the after_fork block and now it
> looks like it's working again!
Awesome to know.
> Not sure I still fully (or even partially) understand what's going on
> there, but at least it's working.
Reading a manpage for the fork() syscall should help you understand
what's going on.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: When a client terminates a connection
2012-11-30 19:26 0% ` Eric Wong
@ 2012-12-03 14:42 0% ` Andrew Stewart
0 siblings, 0 replies; 200+ results
From: Andrew Stewart @ 2012-12-03 14:42 UTC (permalink / raw)
To: unicorn list
On 30 Nov 2012, at 20:26, Eric Wong wrote:
>> Nginx logged:
>>
>> x.xxx.xx.xx - - [27/Nov/2012:14:40:28 +0000] "POST /clients/2248 HTTP/1.1" 499 0 "https://example.com/clients/2248/edit" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
>> x.xxx.xx.xx - - [27/Nov/2012:14:40:29 +0000] "POST /clients/2248 HTTP/1.1" 404 592 "https://example.com/companies/2248/edit" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
>>
>> You can see the first request was terminated by the user. Hot on its
>> heels the user tried again and got a 404.
>
> Weird, and this is the only nginx instance that saw this request?
That's right, I only run a single nginx worker process.
> Also, according to your logs above, the two requests above came from
> different Referer pages, so perhaps there were more requests involved
> that hit a different nginx?
Bizarre as it sounds, that was a copy-paste mistake on my part. The second referrer should read: "https://example.com/clients/2248/edit".
> Odd. It's been a long time since I looked at Rails; but doesn't Rails
> log when a request _starts_? Also, doesn't Rails log all DB queries?
I think Rails logs when a request starts, then logs some more when the request ends – but I'm not certain and I couldn't find the answer when I looked in the Rails source. It must be there somewhere.
Rails doesn't log the DB queries in production, at least not by default.
> Or, by any chance, do you have query logging enabled in your DB to track
> this down?
Regrettably not.
It's a weird problem which I'd like to dismiss, but it's happened on two occasions now. I'm running an old version of Rails (3.0.12) which may or may not be relevant. Hmm.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Issue starting unicorn with non-ActiveRecord Rails app
@ 2012-12-05 9:56 2% ` Peter Hall
0 siblings, 0 replies; 200+ results
From: Peter Hall @ 2012-12-05 9:56 UTC (permalink / raw)
To: unicorn list
> What's L68 of your config/unicorn.rb file look like? Check to see if you have copied/pasted a line that references ActiveRecord in the before_fork or after_fork blocks.
>
> A lot of unicorn examples have an ActiveRecord::Base.connection.disconnect! (with a defined?(ActiveRecord::Base) but that might not be present in your config) statement to ensure that AR connections aren't shared between the master and worker processes.
Aha! Thanks Daniel. =) I'm such an idiot =oP I totally missed that it
was in my own code - including the path on the top line of the trace
=|. The reason I presumed it wasn't is because I copied+pasted most of
the unicorn config, so I didn't remember *writing* any AR-specific
code. I guess it should work if I take that out!
The line actually turned out to be ActiveRecord::Base.establish_connection
Thanks again! *slaps forehead*
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Unicorn fails to install even though it's already installed and running
@ 2012-12-06 21:54 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2012-12-06 21:54 UTC (permalink / raw)
To: unicorn list
Mac Martine <99miles@gmail.com> wrote:
> Thanks,
>
> I can install other gems just fine in the same way, so it doesn't seem
> like permissions then, right?
>
> And disk space looks ok:
That's good to know.
> Here's the end of mkmf.log
>
> have_macro: checking for SIZEOF_SIZE_T in ruby.h... -------------------- no
>
> "gcc -I. -I/usr/lib/ruby/1.8/x86_64-linux -I. -fno-strict-aliasing -g -g -O2 -fPIC -c conftest.c"
Wait, are you now using a different version of Ruby? Your original
message had 1.9 (via RVM)
> >> ** [out :: mydomain.net] Results logged to /rails_apps/eg/production/shared/bundle/ruby/1.9.1/gems/unicorn-4.4.0/ext/unicorn_http/gem_make.out
> >
> > Maybe gem_make.out will have more info than mkmf.log
Again ^^
However, scrutinizing your original email more:
> >> "gcc -I/home/evergreen/.rvm/rubies/ruby-1.9.3-p194/include/ruby-1.9.1/x86_64-linux -I/home/eg/.rvm/rubies/ruby-1.9.3-p194/include/ruby-1.9.1/ruby/backward -I/home/eg/.rvm/rubies/ruby-1.9.3-p194/include/ruby-1.9.1 -I. -I/home/eg/.rvm/usr/include -O3 -ggdb -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wpointer-arith -Wwrite-strings -Wdeclaration-after-statement -Wimplicit-function-declaration -fPIC -c conftest.c"
I see both /home/evergreen/ and /home/eg/ in there; that looks
suspicious.
Did you perhaps copy anything in .rvm/ between home
directories or rename the home directory at some point after
RVM was installed?
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: SIGSEGV at shutdown (was: Re: your mail)
@ 2013-01-24 10:27 2% ` Charles Hornberger
0 siblings, 0 replies; 200+ results
From: Charles Hornberger @ 2013-01-24 10:27 UTC (permalink / raw)
To: unicorn list
On Thu, Jan 24, 2013 at 10:52 AM, Eric Wong <normalperson@yhbt.net> wrote:
> Charles Hornberger <charles.hornberger@gmail.com> wrote:
>> On Mon, Jan 21, 2013 at 11:28 AM, Eric Wong <normalperson@yhbt.net> wrote:
>> > Charles Hornberger <charles.hornberger@gmail.com> wrote:
>> >> E, [2013-01-18T17:54:21.502915 #59285] ERROR -- : reaped
>> >> #<Process::Status: pid 59288 SIGSEGV (signal 11)> worker=1
>
> <snip>
>
>> >> I, [2013-01-18T17:54:21.605077 #59285] INFO -- : master complete
>> >>
>> >> Just wondering if it's something I should be concerned about? I saw no
>> >> obvious symptoms of problems before or after…
>> >>
>> >> We currently restart unicorn (which is on a freebsd jail) like so:
>> >
>> > A SEGV at shutdown is likely an ordering problem at VM shutdown
>> > (probably GC/finalization handling). It could be specific to the
>> > malloc/pthread implementation on FreeBSD, even.
>> >
>> > Which version of Ruby are you using?
>>
>> 1.9.3p-125
>
> Can you give 1.9.3-p374 a try? There were several GC-related segfault
> fixes along the way (fairly large and drastic changes, even).
We'll be moving to that soon. Since the problem has only occurred
*once* afaik, and there was nothing special that I can think of to do
to reproduce it, we may never know if the change fixed anything. :)
But if I happen to see it again, I will attempt to provide more info.
>
> <snip>
>
> Also, did you manage to get any backtrace from Ruby at all? The Ruby VM
> should show a partial stack trace on SEGV (or at least attempt to, but
> it may not during shutdown). Otherwise, you'd have to get gdb-able core
> dumps the old-fashioned way (ulimit -c unlimited) and show us (or
> ruby-core) the backtrace.
Nothing was emitted at the time -- what I sent before was all that
appeared in stderr.log.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: No middleware without touching RACK_ENV
@ 2013-01-28 14:43 1% ` Lin Jen-Shin (godfat)
2013-01-28 23:21 2% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Lin Jen-Shin (godfat) @ 2013-01-28 14:43 UTC (permalink / raw)
To: unicorn list
On Sat, Jan 26, 2013 at 2:39 AM, Eric Wong <normalperson@yhbt.net> wrote:
> Doesn't Rails favor RAILS_ENV over RACK_ENV? unicorn ignores RAILS_ENV
> (unicorn_rails respects RAILS_ENV, but unicorn_rails isn't recommended
> for modern (Rack-enabled) Rails)
Yes, it is. But it would still look into RACK_ENV if RAILS_ENV is not available.
One solution would be setting RAILS_ENV=development while setting
RACK_ENV=none. I don't really want to do this for several reasons:
* It would be better to keep RAILS_ENV == RACK_ENV to avoid confusion.
* It might be tricky to make RAILS_ENV default to development while setting
RACK_ENV since Rails would be looking into it. So we need to make sure
that Rails is loaded and default to development and then we can setup
RACK_ENV to avoid messing up the original Rails default.
> Is there a benefit which RACK_ENV=development gives you for Rails
> that RAILS_ENV=development does not give you?
The main reason is that I want to avoid confusion. The other reason
is that that's also how Heroku tries to use. I think it's also because of
consistency. (so that we don't have to distinguish RACK_ENV and RAILS_ENV)
The other reason would be making them different might cause some
issues as Rails would also be looking at RACK_ENV.
> Fwiw, the times I worked on Rails, I used customized dev environments
> anyways, so I would use RAILS_ENV=dev_local or RAILS_ENV=dev_$NETWORK
> for switching between a localhost-only vs networked setup.
I would also do this I guess, but since I am not the only one working on
the application, I would avoid making such changes. We have a lot of
configurations depending on the word "development" already.
>> I know this is somehow a spec from Rack, but I guess I
>> don't like this behaviour. One workaround would be
>> providing a `after_app_load' hook, and we add this to
>> the bottom of config.ru:
>>
>> ENV['RACK_ENV_ORIGINAL'] = ENV['RACK_ENV']
>> ENV['RACK_ENV'] = 'none'
>>
>> and add this to the unicorn configuration file:
>>
>> after_app_load do |_|
>> ENV['RACK_ENV'] = ENV['RACK_ENV_ORIGINAL']
>> end
>>
>> This is probably a stupid hack, but I wonder if after_app_load
>> hook would be useful for other cases?
>
> I don't see how this hook has benefits over putting this in config.ru
> (or somewhere else in your app, since your app is already loaded...)
It's better to me because it's clear on the loading order.
The point where to change RACK_ENV=none would be
after Rails loaded (to avoid messing Rails up), and
_before Unicorn inserting middleware_. And then I would
like to switch RACK_ENV=RAILS_ENV to avoid confusion.
It's hard to tell where I can put this code.
> I'd imagine users wanting the same app-wide settings going between
> different application servers, too.
Speaking to this, I might want to complain about the other severs :P
As far as I know, only if you use `rackup' would you get those middleware.
That is, `unicorn' is compatible with `rackup'. That's a good thing.
But it seems to me that both `thin' and `puma' (and maybe including
others, I didn't check) would not do this if you use their command line
tools. They even have different options to enable Rack::CommonLogger.
Actually this is also the reason why sometimes `thin' and `unicorn'
worked differently when people switching from Thin to Unicorn.
And you know a lot of people are using Thin in the first place,
thus sometimes blaming Unicorn works differently.
I am not sure how many people are using `rackup', but according to
my observation, not too much. If people are using Sinatra directly,
eventually they would launch the server via Rack::Handler.get.run,
which does not try to wrap additional middleware as what `rackup'
would do if I read correctly.
So I am not sure if a lot of people would be expecting this actually.
I used to use `rackup' a lot to get consistent behaviour, but it looks to
me not many people even know the existence of rackup :(
> I'm against adding more command-line or config options, especially
> around loading middleware. RACK_ENV is already a source of confusion,
> and Rails also has/(or had?) it's own middleware loading scheme
> independent of config.ru.
Yes, I hate that, too. I can't complain about Rails less :P I don't understand
why people are building those middleware scheme on their own way,
even Sinatra does this. I guess they want more control over middleware,
so that they can reorder, delete, insert more/less middleware somehow.
I guess that makes sense in some way... but they should be really in
Rack instead of each web framework IMHO.
> git format-patch + git send-email is the recommended way if your
> mail client cannot be taught to leave formatting alone. Also this
> is easy for sending multiple patches in the same thread.
>
> git format-patch --stdout with a good mail client which will not wrap
> messages is also great (especially for single patches).
>
> Attachments (generated using git format-patch) for text/x-diff should be
> accepted by Mailman. Attached patches are a last resort, I prefer
> inline patches since I can quote/reply to them inline.
>
> (same patch submission guidelines as git or Linux kernel)
Last time I tried to setup SMTP for git send-email failed.
I'll try again next time. Thanks!
> Commit messages should explain why a change is made/needed, not just
> what it does.
Yeah, I know, but I can understand if none of them is acceptable,
(that's why I didn't bring this up in the first place) and the reason to
build those features might be arguing. So instead of writing them
carefully, I decided to bring this up sooner and then we can work out
the true reason later on :D
Thanks a lot for your time.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 1%]
* Re: No middleware without touching RACK_ENV
2013-01-28 14:43 1% ` Lin Jen-Shin (godfat)
@ 2013-01-28 23:21 2% ` Eric Wong
2013-01-29 2:31 0% ` Lin Jen-Shin (godfat)
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2013-01-28 23:21 UTC (permalink / raw)
To: unicorn list
"Lin Jen-Shin (godfat)" <godfat@godfat.org> wrote:
> On Sat, Jan 26, 2013 at 2:39 AM, Eric Wong <normalperson@yhbt.net> wrote:
> > Doesn't Rails favor RAILS_ENV over RACK_ENV? unicorn ignores RAILS_ENV
> > (unicorn_rails respects RAILS_ENV, but unicorn_rails isn't recommended
> > for modern (Rack-enabled) Rails)
>
> Yes, it is. But it would still look into RACK_ENV if RAILS_ENV is not available.
> One solution would be setting RAILS_ENV=development while setting
> RACK_ENV=none. I don't really want to do this for several reasons:
>
> * It would be better to keep RAILS_ENV == RACK_ENV to avoid confusion.
> * It might be tricky to make RAILS_ENV default to development while setting
> RACK_ENV since Rails would be looking into it. So we need to make sure
> that Rails is loaded and default to development and then we can setup
> RACK_ENV to avoid messing up the original Rails default.
OK, I agree making RAILS_ENV==RACK_ENV in deployments is less confusing.
(unicorn should continue NOT enforcing this internally, though)
> > I'd imagine users wanting the same app-wide settings going between
> > different application servers, too.
>
> Speaking to this, I might want to complain about the other severs :P
> As far as I know, only if you use `rackup' would you get those middleware.
>
> That is, `unicorn' is compatible with `rackup'. That's a good thing.
> But it seems to me that both `thin' and `puma' (and maybe including
> others, I didn't check) would not do this if you use their command line
> tools. They even have different options to enable Rack::CommonLogger.
>
> Actually this is also the reason why sometimes `thin' and `unicorn'
> worked differently when people switching from Thin to Unicorn.
> And you know a lot of people are using Thin in the first place,
> thus sometimes blaming Unicorn works differently.
>
> I am not sure how many people are using `rackup', but according to
> my observation, not too much. If people are using Sinatra directly,
> eventually they would launch the server via Rack::Handler.get.run,
> which does not try to wrap additional middleware as what `rackup'
> would do if I read correctly.
>
> So I am not sure if a lot of people would be expecting this actually.
> I used to use `rackup' a lot to get consistent behaviour, but it looks to
> me not many people even know the existence of rackup :(
*sigh* :<
I kinda wish there was no default middleware, either. But it's
too late to change unicorn defaults.
I think your -N/--no-default-middleware patch can be accepted,
then. after_app_load is more prone to user error/confusion, I think
(I believe RAILS_ENV/RACK_ENV should be frozen for the lifetime of
the process)
> Last time I tried to setup SMTP for git send-email failed.
> I'll try again next time. Thanks!
Can you try send-email again with the --no-default-middleware patch? :)
The git-send-email(1) manpage has several good examples for your email
provider. There's also git-imap-send which you can use to stick email
in your Drafts folder, but I've never needed/used it, though.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: No middleware without touching RACK_ENV
2013-01-28 23:21 2% ` Eric Wong
@ 2013-01-29 2:31 0% ` Lin Jen-Shin (godfat)
0 siblings, 0 replies; 200+ results
From: Lin Jen-Shin (godfat) @ 2013-01-29 2:31 UTC (permalink / raw)
To: unicorn list
On Tue, Jan 29, 2013 at 7:21 AM, Eric Wong <normalperson@yhbt.net> wrote:
> "Lin Jen-Shin (godfat)" <godfat@godfat.org> wrote:
>> Speaking to this, I might want to complain about the other severs :P
>> As far as I know, only if you use `rackup' would you get those middleware.
>>
>> That is, `unicorn' is compatible with `rackup'. That's a good thing.
>> But it seems to me that both `thin' and `puma' (and maybe including
>> others, I didn't check) would not do this if you use their command line
>> tools. They even have different options to enable Rack::CommonLogger.
>>
>> Actually this is also the reason why sometimes `thin' and `unicorn'
>> worked differently when people switching from Thin to Unicorn.
>> And you know a lot of people are using Thin in the first place,
>> thus sometimes blaming Unicorn works differently.
>>
>> I am not sure how many people are using `rackup', but according to
>> my observation, not too much. If people are using Sinatra directly,
>> eventually they would launch the server via Rack::Handler.get.run,
>> which does not try to wrap additional middleware as what `rackup'
>> would do if I read correctly.
>>
>> So I am not sure if a lot of people would be expecting this actually.
>> I used to use `rackup' a lot to get consistent behaviour, but it looks to
>> me not many people even know the existence of rackup :(
>
> *sigh* :<
>
> I kinda wish there was no default middleware, either. But it's
> too late to change unicorn defaults.
>
> I think your -N/--no-default-middleware patch can be accepted,
> then. after_app_load is more prone to user error/confusion, I think
> (I believe RAILS_ENV/RACK_ENV should be frozen for the lifetime of
> the process)
Awesome. Thanks! Hope this would also be helpful for others.
> Can you try send-email again with the --no-default-middleware patch? :)
>
> The git-send-email(1) manpage has several good examples for your email
> provider. There's also git-imap-send which you can use to stick email
> in your Drafts folder, but I've never needed/used it, though.
Just got managed to send a test mail to myself. I realized that my git is
using /usr/bin/perl instead of /usr/local/bin/perl, so that instead of running
`cpan Net::SMTP::SSL` I have to run `sudo -H /usr/bin/cpan Net::SMTP::SSL`
following instructions from this site:
http://kbase.wincent.com/old/knowledge-base/Installing_Net::SMTP::SSL_for_sending_patches_with_Git_over_secure_SMTP.html
I'll send the patch shortly after updating the commit message.
Feel free to edit it as usual :)
And should I send patches for rainbows and zbatery as well?
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Unicorn hangs on POST request
@ 2013-04-03 4:56 2% ` Lin Jen-Shin (godfat)
2013-04-03 11:38 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Lin Jen-Shin (godfat) @ 2013-04-03 4:56 UTC (permalink / raw)
To: unicorn list
On Wed, Apr 3, 2013 at 6:36 AM, Eric Wong <normalperson@yhbt.net> wrote:
> Fwiw, Cool.io works pretty well in my experience.
[...]
> I can also help fix Cool.io bugs since it's written in C, but I can't
> fix EM bugs: C++ is too big for my tiny brain.
Last time I tried Cool.io it would occasionally crash with assertion errors.
It's been a while from now, but I guess since there's no much development
happened for a long time, it won't change too much right now.
I could try to get the assertion errors again and see if anyone wants to step
in, but personally since the author is heading to celluloid-io, I am not too
interested in fixing Cool.io...
Off topic, I have some EM issue with Ruby 2.0, which would hang on our
tests and after interrupting the hung test, the following tests would yield:
"RuntimeError: eventmachine not initialized: evma_signal_loopbreak"
I have no idea what's going on... :( I hope we could leave EM soon...
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Unicorn hangs on POST request
2013-04-03 4:56 2% ` Lin Jen-Shin (godfat)
@ 2013-04-03 11:38 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2013-04-03 11:38 UTC (permalink / raw)
To: unicorn list
"Lin Jen-Shin (godfat)" <godfat@godfat.org> wrote:
> On Wed, Apr 3, 2013 at 6:36 AM, Eric Wong <normalperson@yhbt.net> wrote:
> > Fwiw, Cool.io works pretty well in my experience.
> [...]
> > I can also help fix Cool.io bugs since it's written in C, but I can't
> > fix EM bugs: C++ is too big for my tiny brain.
>
> Last time I tried Cool.io it would occasionally crash with assertion errors.
> It's been a while from now, but I guess since there's no much development
> happened for a long time, it won't change too much right now.
>
> I could try to get the assertion errors again and see if anyone wants to step
> in, but personally since the author is heading to celluloid-io, I am not too
> interested in fixing Cool.io...
I'd be willing to look into this and send a fix to coolio upstream if
you can help me reproduce it. No guarantees there'll be a release even
if I patch it, though, that's up to Tony Arcieri (Coolio maintainer).
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Why doesn't SIGTERM quit gracefully?
@ 2013-04-25 18:09 2% ` Eric Wong
2013-04-25 20:45 0% ` Andreas Falk
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2013-04-25 18:09 UTC (permalink / raw)
To: unicorn list; +Cc: Andreas Falk
Andreas Falk <mail@andreasfalk.se> wrote:
> On Thu, Apr 25, 2013 at 10:51 AM, Eric Wong <normalperson@yhbt.net> wrote:
> > Andreas Falk <mail@andreasfalk.se> wrote:
> >> I'm wondering why SIGINT and SIGTERM both were chosen for the quick
> >> shutdown? I agree with SIGINT but not with SIGTERM. A lot of unix
> >> tools send SIGTERM as default (kill, runit among some) and it seems to
> >> be the standard way of telling a process to quit gracefully but not
> >> among Ruby people (there are a few other ruby processes behaving the
> >> same way). I just think it's weird that the default command will exit
> >> without taking care of their current request.
> >>
> >> Also i'm not on the mailinglist so it would be great if you could cc
> >> mail@andreasfalk.se
> >
> > I think it's weird, too. But that's what nginx does, and I based most
> > of the UI decisions on nginx (so it's easy to reuse nginx scripts
> > with unicorn).
>
> Is it something you'd be willing to change?
If nginx developers are willing to change this, definitely.
Otherwise, at this point, I'm not certain what the ramifications of a
change would be to existing users. Maybe an option would work, but too
many options overwhelms people.
With unicorn, I suppose having the following after_fork hook works:
# Note: after_fork hook is run before the :QUIT handler is installed,
# so we can't use the result of trap(:QUIT) at this point:
after_fork do |server,worker|
trap(:TERM) { Process.kill(:QUIT, $$) }
end
If I had to do this all over again[1], graceful shutdown would be the
_only_ option.
[1] - I haven't released a new web server for 2013... yet :)
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Why doesn't SIGTERM quit gracefully?
2013-04-25 18:09 2% ` Eric Wong
@ 2013-04-25 20:45 0% ` Andreas Falk
0 siblings, 0 replies; 200+ results
From: Andreas Falk @ 2013-04-25 20:45 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn list
On Thu, Apr 25, 2013 at 8:09 PM, Eric Wong <normalperson@yhbt.net> wrote:
> If nginx developers are willing to change this, definitely.
I guess I'll go camp out among some nginx developers then.
> If I had to do this all over again[1], graceful shutdown would be the
> _only_ option.
>
> [1] - I haven't released a new web server for 2013... yet :)
Hehe, i hear you! Maybe a unicorn 5 or do you have any plans for
something completely new :)? I guess i'll have to wait and see.
Thanks for the help!
Andreas
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Increassing timeouts
@ 2013-08-03 6:32 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2013-08-03 6:32 UTC (permalink / raw)
To: unicorn list
Troex Nevelin <list@mrtech.ru> wrote:
> On Jan 15, 2013, at 21:44 , Eric Wong <normalperson@yhbt.net> wrote:
> > But seriously, who will wait 120s for a website to load?
>
> I have similar situation, in my case it's admin interface so admin
> knows that some operations need time to process and will wait.
>
> Right now I'm running two packs of Unicorn instances - one for quick
> and one for long requests. But most of the time long request pool is
> idling and eating memory.
Cranking up the timeout will improve its chances of being swapped out.
If memory usage of an infrequently used instance is a concern.
> My question: is possible to change Unicorn timeout per request from
> Rails app? Middleware hook could be ok too. (not sure if it's possible
> as I think timeout is set in master).
Right, it's currently not possible to influence the master. And I'm
not convinced it's the right way to go, either.
What I've done in the past is to put the client in a periodic refresh
loop (meta refresh or whatever tag it was) and have it poll the app
while it was waiting for a long-running job to finish. I think it was
(or still is) a common thing...
Btw, Rubyforge delayed this email by over 3 weeks? But then again you
responded to a 6 month old thread %x
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* A barrage of unexplained timeouts
@ 2013-08-20 14:47 2% nick
0 siblings, 1 reply; 200+ results
From: nick @ 2013-08-20 14:47 UTC (permalink / raw)
To: mongrel-unicorn
We've been running unicorn-3.6.2 on REE 1.8.7 2011.12 in production for quite some time and we use monit to monitor each unicorn worker. Occasionally, I'll get a notification that a worker has timed-out and has been re-spawned. In all these cases, when I look at the rails logs, I can see the last request that the worker handled, and they all have appeared to complete successfully from the client's perspective (rails and nginx respond with 200), but the unicorn log shows that it was killed due to timeout. This has always been relatively rare and I thought it was a non-problem.
Until today.
Today, for about a 7 minute period, our workers would continually report as having timed-out and would be killed by the master. After re-spawning, the workers would serve a handful of requests and then eventually be killed again.
During this time, our servers (Web, PG DB, and redis) were not under load and IO was normal. After the last monit notification at 8:30, everything went back to normal. I understand why unicorns would timeout if they were waiting (>120 secs) on IO, but there aren't any orphaned requests in the rails log. For each request line, there's a corresponding completion line. No long running queries to blame on PG, either.
I know we're probably due for an upgrade, but I'm hoping to get to the bottom of these unexplained timeouts.
Thanks for your help!
-Nick
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: A barrage of unexplained timeouts
@ 2013-08-20 17:27 2% ` nick
0 siblings, 1 reply; 200+ results
From: nick @ 2013-08-20 17:27 UTC (permalink / raw)
To: unicorn list
"Eric Wong" <normalperson@yhbt.net> said:
> Can you take a look at the nginx error and access logs? From what
> you're saying, there's a chance a request never even got to the Rails
> layer. However, nginx should be logging failed/long-running requests to
> unicorn.
The nginx access logs show frequent 499 responses. The error logs are filled with:
connect() failed (110: Connection timed out) while connecting to upstream
upstream timed out (110: Connection timed out) while reading response header from upstream
What specific pieces of information should I be looking for in the logs?
Thanks Again,
-Nick
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: A barrage of unexplained timeouts
@ 2013-08-20 18:11 2% ` nick
2013-08-20 18:49 2% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: nick @ 2013-08-20 18:11 UTC (permalink / raw)
To: unicorn list
"Eric Wong" <normalperson@yhbt.net> said:
> nick@auger.net wrote:
>> "Eric Wong" <normalperson@yhbt.net> said:
>> > Can you take a look at the nginx error and access logs? From what
>> > you're saying, there's a chance a request never even got to the Rails
>> > layer. However, nginx should be logging failed/long-running requests to
>> > unicorn.
>>
>> The nginx access logs show frequent 499 responses. The error logs are filled
>> with:
>>
>> connect() failed (110: Connection timed out) while connecting to upstream
>> upstream timed out (110: Connection timed out) while reading response header from
>> upstream
>>
>> What specific pieces of information should I be looking for in the logs?
>
> Do you have any other requests in your logs which could be taking
> a long time and hogging workers, but not high enough to trigger the
> unicorn kill timeout.
I don't *think* so. Most requests finish <300ms. We do have some more intensive code-paths, but they're administrative and called much less frequently. Most of these pages complete in <3seconds.
For requests that made it to rails logging, the LAST processed request before the worker timed-out all completed very quickly (and no real pattern in terms of which page may be triggering it.)
> (enable $request_time in nginx access logs if you haven't already)
I'll enable this.
> Is this with Unix or TCP sockets? If it's over a LAN, maybe there's
> still a bad switch/port/cable somewhere (that happens often to me).
TCP sockets, with nginx and unicorn running on the same box.
> With Unix sockets, I don't recall encountering recent problems under
> Linux. Which OS are you running?
Stock RHEL 5, kernel 2.6.18.
Thanks again,
-Nick
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: A barrage of unexplained timeouts
2013-08-20 18:11 2% ` nick
@ 2013-08-20 18:49 2% ` Eric Wong
2013-08-20 20:03 2% ` nick
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2013-08-20 18:49 UTC (permalink / raw)
To: unicorn list
nick@auger.net wrote:
> "Eric Wong" <normalperson@yhbt.net> said:
> >
> > Do you have any other requests in your logs which could be taking
> > a long time and hogging workers, but not high enough to trigger the
> > unicorn kill timeout.
> I don't *think* so. Most requests finish <300ms. We do have some
> more intensive code-paths, but they're administrative and called much
> less frequently. Most of these pages complete in <3seconds.
>
> For requests that made it to rails logging, the LAST processed request
> before the worker timed-out all completed very quickly (and no real
> pattern in terms of which page may be triggering it.)
This is really strange. This was only really bad for a 7s period?
Has it happened again? Anything else going on with the system at that
time? Swapping, particularly...
And if you're inside a VM, maybe your neighbors were hogging things.
Large PUT/POST requests which require filesystem I/O are particularly
sensitive to this.
> > Is this with Unix or TCP sockets? If it's over a LAN, maybe there's
> > still a bad switch/port/cable somewhere (that happens often to me).
>
> TCP sockets, with nginx and unicorn running on the same box.
OK, that probably rules out a bunch of problems.
Just to be thorough, anything interesting in dmesg or syslogs?
> > With Unix sockets, I don't recall encountering recent problems under
> > Linux. Which OS are you running?
>
> Stock RHEL 5, kernel 2.6.18.
RHEL 5.0 or 5.x? I can't remember /that/ far back to 5.0 (I don't think
I even tried it until 5.2), but don't recall anything being obviously
broken in those...
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: A barrage of unexplained timeouts
2013-08-20 18:49 2% ` Eric Wong
@ 2013-08-20 20:03 2% ` nick
2013-08-20 20:42 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: nick @ 2013-08-20 20:03 UTC (permalink / raw)
To: unicorn list
"Eric Wong" <normalperson@yhbt.net> said:
> nick@auger.net wrote:
>> "Eric Wong" <normalperson@yhbt.net> said:
>> >
>> > Do you have any other requests in your logs which could be taking
>> > a long time and hogging workers, but not high enough to trigger the
>> > unicorn kill timeout.
>
>> I don't *think* so. Most requests finish <300ms. We do have some
>> more intensive code-paths, but they're administrative and called much
>> less frequently. Most of these pages complete in <3seconds.
>>
>> For requests that made it to rails logging, the LAST processed request
>> before the worker timed-out all completed very quickly (and no real
>> pattern in terms of which page may be triggering it.)
>
> This is really strange. This was only really bad for a 7s period?
It was a 7 minute period. All of the workers would become busy and exceed their >120s timeout. Master would kill and re-spawn them, they'd start to respond to a handful of requests (anywhere from 5-50) after which they'd become "busy" again, and get force killed by master. This pattern happened 3 times over a 7 minute period.
> Has it happened again?
No
> Anything else going on with the system at that time? Swapping, particularly...
No swap activity or high load. Our munin graphs indicate a peak of web/app server disk latency around that time, although our graphs show many other similar peaks, without incident.
> And if you're inside a VM, maybe your neighbors were hogging things.
> Large PUT/POST requests which require filesystem I/O are particularly
> sensitive to this.
We can't blame virtualization as we're on dedicated hardware.
>> > Is this with Unix or TCP sockets? If it's over a LAN, maybe there's
>> > still a bad switch/port/cable somewhere (that happens often to me).
>>
>> TCP sockets, with nginx and unicorn running on the same box.
>
> OK, that probably rules out a bunch of problems.
> Just to be thorough, anything interesting in dmesg or syslogs?
Nothing interesting in /var/log/messages. dmesg (no timestamps) includes lines like:
TCP: Treason uncloaked! Peer ###.###.###.###:63554/80 shrinks window 1149440448:1149447132. Repaired.
>> > With Unix sockets, I don't recall encountering recent problems under
>> > Linux. Which OS are you running?
>>
>> Stock RHEL 5, kernel 2.6.18.
>
> RHEL 5.0 or 5.x? I can't remember /that/ far back to 5.0 (I don't think
> I even tried it until 5.2), but don't recall anything being obviously
> broken in those...
We're on RHEL 5.6.
Thanks,
-Nick
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: A barrage of unexplained timeouts
2013-08-20 20:03 2% ` nick
@ 2013-08-20 20:42 0% ` Eric Wong
2013-08-20 21:19 0% ` nick
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2013-08-20 20:42 UTC (permalink / raw)
To: unicorn list
nick@auger.net wrote:
> "Eric Wong" <normalperson@yhbt.net> said:
> > This is really strange. This was only really bad for a 7s period?
>
> It was a 7 minute period. All of the workers would become busy and
> exceed their >120s timeout. Master would kill and re-spawn them,
> they'd start to respond to a handful of requests (anywhere from 5-50)
> after which they'd become "busy" again, and get force killed by
> master. This pattern happened 3 times over a 7 minute period.
>
> > Has it happened again?
>
> No
>
> > Anything else going on with the system at that time? Swapping,
> > particularly...
>
> No swap activity or high load. Our munin graphs indicate a peak of
> web/app server disk latency around that time, although our graphs show
> many other similar peaks, without incident.
I'm stumped :<
Do you have any background threads running that could be hanging the
workers? This is Ruby 1.8, after all, so there's more likely to be
some blocking call hanging the entire process. AFAIK, some monitoring
software runs a background thread in the unicorn worker and maybe the
OpenSSL extension doesn't work as well if it encountered network
problems under Ruby 1.8
Otherwise, I don't know...
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: A barrage of unexplained timeouts
2013-08-20 20:42 0% ` Eric Wong
@ 2013-08-20 21:19 0% ` nick
0 siblings, 0 replies; 200+ results
From: nick @ 2013-08-20 21:19 UTC (permalink / raw)
To: unicorn list
"Eric Wong" <normalperson@yhbt.net> said:
> nick@auger.net wrote:
>> "Eric Wong" <normalperson@yhbt.net> said:
>> > This is really strange. This was only really bad for a 7s period?
>>
>> It was a 7 minute period. All of the workers would become busy and
>> exceed their >120s timeout. Master would kill and re-spawn them,
>> they'd start to respond to a handful of requests (anywhere from 5-50)
>> after which they'd become "busy" again, and get force killed by
>> master. This pattern happened 3 times over a 7 minute period.
>>
>> > Has it happened again?
>>
>> No
>>
>> > Anything else going on with the system at that time? Swapping,
>> > particularly...
>>
>> No swap activity or high load. Our munin graphs indicate a peak of
>> web/app server disk latency around that time, although our graphs show
>> many other similar peaks, without incident.
>
> I'm stumped :<
I was afraid you'd say that :(.
> Do you have any background threads running that could be hanging the
> workers? This is Ruby 1.8, after all, so there's more likely to be
> some blocking call hanging the entire process. AFAIK, some monitoring
> software runs a background thread in the unicorn worker and maybe the
> OpenSSL extension doesn't work as well if it encountered network
> problems under Ruby 1.8
We don't explicitly create any threads in our rails code. We do communicate with backgroundrb worker processes, although, none of the strangeness today involved any routes that would hit backgroundrb workers.
Is there any instrumentation that I could add that might help debugging in the future? ($request_time and $upstream_response_time are now in my nginx logs.) We have noticed these "unexplainable timeouts" before, but typically for a single worker. If there's some debugging that could be added I might be able to track it down during these one-off events.
Thoughts?
I absolutely appreciate all your help!
-Nick
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* More unexplained timeouts
@ 2013-09-29 20:13 2% nick
0 siblings, 0 replies; 200+ results
From: nick @ 2013-09-29 20:13 UTC (permalink / raw)
To: mongrel-unicorn
We're still suffering from unexplained workers timing out. We recently upgraded to the latest unicorn 4.6.3 (while still on REE 1.8.7) in the hopes that it would solve our issues. Unfortunately, this seemed to exacerbate the problem, with timeouts happening more frequently, but that could be related to greater precision in timeouts in newer versions of unicorn. (In our unicorn 3.6.2, a timeout set to 120s might not ACTUALLY timeout until 180s or more, thus allowing a bit more time for Ruby to finish whatever it was choking on.)
We dropped the timeout down to 65s (to make sure it was triggered) and then tried to add greater logging (per http://permalink.gmane.org/gmane.comp.lang.ruby.unicorn.general/1269.) The START/FINISH approach confirms it's not an issue with our application code, ie:
HH:MM:SS- S/F[PID]- /PATH
15:21:01- START-25904- /pathA
15:21:01- FINISH-25904- /pathA
15:21:01- START-25904- /pathB
15:21:01- FINISH-25904- /pathB
15:21:01- START-25904- /pathC
15:21:01- FINISH-25904- /pathC
worker=11 PID:25904 timeout (66s > 65s), killing
reaped #<Process::Status: pid=25904,signaled(SIGKILL=9)> worker=11
For each START we always get a corresponding FINISH and then the worker is killed. Additionally, our nginx logs confirm that this last request was sent back to the client. No 'upstream' errors in our nginx log, either.
When we tried the Thread sleep approach, nothing actually appeared in the logs. I imagine this means that ruby or some C extension is misbehaving.
Unfortunately, it's been impossible for us to recreate this in development.
Thoughts?
RHEL 5.6
REE 1.8.7 2011.12
Unicorn 4.6.3
16 unicorn workers on 8 cores
No swap activity, no peaks in load
Again, thanks for all your help!
-Nick
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* pid file handling issue
@ 2013-10-23 22:55 2% Michael Fischer
2013-10-24 0:53 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Michael Fischer @ 2013-10-23 22:55 UTC (permalink / raw)
To: mongrel-unicorn
Hi everyone,
While writing a script to determine the success or failure of a
Unicorn reload attempt (without having to parse a log), I noticed that
Unicorn doesn't preserve the timestamp of its pid file. In other
words, instead of renaming pidfile to pidfile.oldbin (and then back
again if the reload failed), it creates a new pid file for each master
phase change.
This means we cannot simply compare the mtime of the current pidfile
against the time the USR2 signal was given in order to make a
reasonable conclusion.
I tried another method, which was to look at the start time of the
process as reported by ps(1), but on Linux, that time does not come
from the wall clock: it's derived from the number of jiffies since
system boot. So it's not guaranteed to be accurate, especially if the
wall clock was incorrect at system boot.
Are there any other methods anyone can suggest? Otherwise, a change
to Unicorn's behavior with respect to pid file maintenance would be
kindly appreciated.
Best regards,
--Michael
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: pid file handling issue
2013-10-23 22:55 2% pid file handling issue Michael Fischer
@ 2013-10-24 0:53 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2013-10-24 0:53 UTC (permalink / raw)
To: unicorn list
Michael Fischer <mfischer@zendesk.com> wrote:
> Hi everyone,
>
> While writing a script to determine the success or failure of a
> Unicorn reload attempt (without having to parse a log), I noticed that
> Unicorn doesn't preserve the timestamp of its pid file. In other
> words, instead of renaming pidfile to pidfile.oldbin (and then back
> again if the reload failed), it creates a new pid file for each master
> phase change.
>
> This means we cannot simply compare the mtime of the current pidfile
> against the time the USR2 signal was given in order to make a
> reasonable conclusion.
>
> I tried another method, which was to look at the start time of the
> process as reported by ps(1), but on Linux, that time does not come
> from the wall clock: it's derived from the number of jiffies since
> system boot. So it's not guaranteed to be accurate, especially if the
> wall clock was incorrect at system boot.
>
> Are there any other methods anyone can suggest? Otherwise, a change
> to Unicorn's behavior with respect to pid file maintenance would be
> kindly appreciated.
I read and stash the value of the pid file before issuing any USR2.
Later, you can issue "kill -0 $old_pid" after sending SIGQUIT
to ensure it's dead.
Checking the mtime of the pidfile is really bizarre...
OTOH, there's times when users accidentally remove a pid
file and regenerate by hand it from ps(1), too...
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: [PATCH] construct listener_fds Hash in 1.8 compatible way
@ 2013-11-01 19:00 2% ` Ernest W. Durbin III
0 siblings, 0 replies; 200+ results
From: Ernest W. Durbin III @ 2013-11-01 19:00 UTC (permalink / raw)
To: unicorn list
On Fri, Nov 1, 2013 at 2:54 PM, Eric Wong <normalperson@yhbt.net> wrote:
> "Ernest W. Durbin III" <ewdurbin@gmail.com> wrote:
>> On Fri, Nov 1, 2013 at 12:50 PM, Eric Wong <normalperson@yhbt.net> wrote:
>> > "Ernest W. Durbin III" <ewdurbin@gmail.com> wrote:
>> >> This renables the ability for Ruby 1.8 environments to perform reexecs
>> >
>> > Is this for Ruby 1.8.6? I've only tested on 1.8.7,
>> > I haven't had a 1.8.6 installation in a while.
>> >
>>
>> I'll admit that the reason I need it is for a Ruby 1.8.6 environment...
>
> OK (wow!). I've been pondering dropping 1.8 entirely, but I think I'll
> keep it for now since CentOS 6.x still uses it
>
> Anyways, was that the only 1.8.6-incompatible thing we did?
That's all I've found so far! Happy to be dropping thin, and I promise
we're on course to get past 1.8.6 at some point... But I'm just the
Ops guy :)
>
>> However, Ruby 1.8.7 does not have support for that Hash constructor
>> either it simply fails silently and in a bug prone manner.
>>
>> ```
>> [ernestd@ewd3do ~]$ ruby --version
>> ruby 1.8.7 (2011-06-30 patchlevel 352) [i386-linux]
>> [ernestd@ewd3do ~]$ irb
>> irb(main):001:0> thing = Hash[ [ 'foo', 'bar' ], ['fizz', 'buzz'] ]
>> => {["foo", "bar"]=>["fizz", "buzz"]}
>
> Actually, on 1.8.7, unicorn does the following (note the extra outer array)
>
> irb(main):001:0> thing = Hash[ [ [ 'foo', 'bar' ], ['fizz', 'buzz'] ] ]
> => {"fizz"=>"buzz", "foo"=>"bar"}
>
> so it was fine
Interesting. Sorry I missed that in the initial irb tests.
Funnily enough, the List of List's Constructor type isn't documented for 1.8.7
Thanks Again!
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* [PATCH] stream_input: avoid IO#close on client disconnect
2013-11-07 16:48 0% ` Eric Wong
@ 2013-11-07 20:22 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2013-11-07 20:22 UTC (permalink / raw)
To: Andrew Hobson; +Cc: mongrel-unicorn
Eric Wong <normalperson@yhbt.net> wrote:
> Andrew Hobson <ahobson@gmail.com> wrote:
> > I applied that one line patch a day and a half ago and we haven't seen
> > the error in the field (yet). I am optimistic you have elegantly fixed
> > the problem.
> >
> > If we do see an error, I will send another email to the list.
> >
> > Thanks again for your help,
>
> No problem, I'll push that out later today.
Pushed, thanks again:
>From f4005d5efc608e7d75371f0d0527041facd33f89 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Thu, 7 Nov 2013 20:10:01 +0000
Subject: [PATCH] stream_input: avoid IO#close on client disconnect
This can avoid IOError from being seen by the application, and also
reduces points where IO#close may be called. This is a good thing
if we eventually port this code into a low-level server like
cmogstored where per-client memory space is defined by FD number of
a client.
Reported-by: Andrew Hobson <ahobson@gmail.com>
---
lib/unicorn/stream_input.rb | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/lib/unicorn/stream_input.rb b/lib/unicorn/stream_input.rb
index c8a4240..9278f47 100644
--- a/lib/unicorn/stream_input.rb
+++ b/lib/unicorn/stream_input.rb
@@ -139,10 +139,7 @@ private
# we do support clients that shutdown(SHUT_WR) after the
# _entire_ request has been sent, and those will not have
# raised EOFError on us.
- if @socket
- @socket.shutdown
- @socket.close
- end
+ @socket.shutdown if @socket
ensure
raise Unicorn::ClientShutdown, "bytes_read=#{@bytes_read}", []
end
--
1.8.4.483.g7fe67e6.dirty
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 2%]
* Re: Handling closed clients
[not found] ` <m21u2sjpc9.fsf@macdaddy.atl.damballa>
@ 2013-11-07 16:48 0% ` Eric Wong
2013-11-07 20:22 2% ` [PATCH] stream_input: avoid IO#close on client disconnect Eric Wong
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2013-11-07 16:48 UTC (permalink / raw)
To: Andrew Hobson; +Cc: mongrel-unicorn
Andrew Hobson <ahobson@gmail.com> wrote:
> Eric Wong <normalperson@yhbt.net> writes:
>
> > (Please don't cull Cc:, I'm assuming you're not subscribed to the
> > mailing list since we don't require subscriptions)
>
> Sorry, that was unintentional.
No worries, and it is good to also send a copy to each recipient in the
thread in case Rubyforge is down (like it is right now). If it stays
down, I'll have to find/make a replacement myself.
...And move to something more decentralized and resilient to downtime
while I'm at it.
> > With my proposed patch to eliminate IO#close from StreamInput,
> > this test is no longer an accurate representation of unicorn behavior.
>
> I applied that one line patch a day and a half ago and we haven't seen
> the error in the field (yet). I am optimistic you have elegantly fixed
> the problem.
>
> If we do see an error, I will send another email to the list.
>
> Thanks again for your help,
No problem, I'll push that out later today.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: What does it mean for the unicorn process to be bound to a terminal?
@ 2013-11-26 19:21 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2013-11-26 19:21 UTC (permalink / raw)
To: unicorn list; +Cc: Rodrigo Rosenfeld Rosas
Rodrigo Rosenfeld Rosas <rr.rosas@gmail.com> wrote:
> I see. If I understood correctly this only happens in foreground
> mode, so something like that would help me to understand the
> sentence:
> "...If your unicorn process is bound to an interactive terminal
> (running in the foreground), you can skip this step."
I used "not daemonized", since it's possible for a background process
to have a controlling terminal:
unicorn ... & # runs background, still attached to terminal
fg # foreground again
Pushed as fa17da92aa4e76d5fd63cb9b74d6884d611ec899
(also added a link to the init.sh example)
---------------------------8<----------------------------
Subject: [PATCH] doc: clarify SIGNALS and reference init example
"interactive terminal" needed clarification.
While we're at it, link to the init.sh example since it may
be shared with nginx.
Reported-by: Rodrigo Rosenfeld Rosas
ref: <5294E9D4.5030608@gmail.com>
---
SIGNALS | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/SIGNALS b/SIGNALS
index 8851775..48c651e 100644
--- a/SIGNALS
+++ b/SIGNALS
@@ -7,6 +7,9 @@ signal handling matches the behavior of {nginx}[http://nginx.net/] so it
should be possible to easily share process management scripts between
Unicorn and nginx.
+One example init script is distributed with unicorn:
+http://unicorn.bogomips.org/examples/init.sh
+
=== Master Process
* HUP - reloads config file and gracefully restart all workers.
@@ -101,8 +104,8 @@ The procedure is exactly like that of nginx:
3. You can now send WINCH to the old master process so only the new workers
serve requests. If your unicorn process is bound to an interactive
- terminal, you can skip this step. Step 5 will be more difficult but
- you can also skip it if your process is not daemonized.
+ terminal (not daemonized), you can skip this step. Step 5 will be more
+ difficult but you can also skip it if your process is not daemonized.
4. You should now ensure that everything is running correctly with the
new workers as the old workers die off.
---------------------------8<----------------------------
> I believe it would help the documentation if you added a link to
> init.sh in the SIGNALS page:
>
> http://bogomips.org/unicorn.git/tree/examples/init.sh
> Anyway, even that script won't take care of making sure that the
> reload action will rollback in case of an unsuccessful deploy.
>
> I realize it's not easy to detect if the deploy was successful and
> that it's very application dependent, but some generic procedures
> would be helpful for a basic Rails application. Unfortunately I
> don't know what would be a good strategy, that's why I asked :P
Right. Different apps require different validation, especially
those intended for GUI web browsers.
> Thank you for explaining the interactive terminal binding question :)
No problem!
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 2%]
* [PATCH] rework master-to-worker signaling to use a pipe
@ 2013-12-09 9:52 1% Eric Wong
2013-12-10 0:22 0% ` Sam Saffron
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2013-12-09 9:52 UTC (permalink / raw)
To: mongrel-unicorn
Signaling using normal kill(2) is preserved, but the master now
prefers to signal workers using a pipe rather than kill(2).
Non-graceful signals (:TERM/:KILL) are still sent using kill(2),
as they ask for immediate shutdown.
This change is necessary to avoid triggering the ubf (unblocking
function) for rb_thread_call_without_gvl (and similar) functions
extensions. Most notably, this fixes compatibility with newer
versions of the 'pg' gem which will cancel a running DB query if
signaled[1].
This also has the nice side-effect of allowing a premature
master death (assuming preload_app didn't cause the master to
spawn off rogue child daemons).
Note: users should also refrain from using "killall" if using the
'pg' gem or something like it.
Unfortunately, this increases FD usage in the master as the writable
end of the pipe is preserved in the master. This limit the number
of worker processes the master may run to the open file limit of the
master process. Increasing the open file limit of the master
process may be needed. However, the FD use on the workers is
reduced by one as the internal self-pipe is no longer used. Thus,
overall pipe allocation for the kernel remains unchanged.
[1] - pg is correct to cancel a query, as it cannot know if
the signal was for a) graceful unicorn shutdown or
b) oh-noes-I-started-a-bad-query-ABORT-ABORT-ABORT!!
---
Pushed to master on git://bogomips.org/unicorn.git
commit 6f6e4115b4bb03e5e7c55def91527799190566f2
SIGNALS | 6 ++++
lib/unicorn.rb | 5 +++
lib/unicorn/http_server.rb | 85 +++++++++++++++++++++++-----------------------
lib/unicorn/worker.rb | 64 ++++++++++++++++++++++++++++++++++
4 files changed, 118 insertions(+), 42 deletions(-)
diff --git a/SIGNALS b/SIGNALS
index 48c651e..ef0b0d9 100644
--- a/SIGNALS
+++ b/SIGNALS
@@ -45,6 +45,10 @@ http://unicorn.bogomips.org/examples/init.sh
=== Worker Processes
+Note: as of unicorn 4.8, the master uses a pipe to signal workers
+instead of kill(2) for most cases. Using signals still (and works and
+remains supported for external tools/libraries), however.
+
Sending signals directly to the worker processes should not normally be
needed. If the master process is running, any exited worker will be
automatically respawned.
@@ -52,6 +56,8 @@ automatically respawned.
* INT/TERM - Quick shutdown, immediately exit.
Unless WINCH has been sent to the master (or the master is killed),
the master process will respawn a worker to replace this one.
+ Immediate shutdown is still triggered using kill(2) and not the
+ internal pipe as of unicorn 4.8
* QUIT - Gracefully exit after finishing the current request.
Unless WINCH has been sent to the master (or the master is killed),
diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 2535159..638b846 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -97,6 +97,11 @@ module Unicorn
logger.error "#{prefix}: #{message} (#{exc.class})"
exc.backtrace.each { |line| logger.error(line) }
end
+
+ # remove this when we only support Ruby >= 2.0
+ def self.pipe # :nodoc:
+ Kgio::Pipe.new.each { |io| io.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) }
+ end
# :startdoc:
end
# :enddoc:
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index f15c8a7..ae8ad13 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -42,16 +42,8 @@ class Unicorn::HttpServer
# it to wake up the master from IO.select in exactly the same manner
# djb describes in http://cr.yp.to/docs/selfpipe.html
#
- # * The workers immediately close the pipe they inherit from the
- # master and replace it with a new pipe after forking. This new
- # pipe is also used to wakeup from IO.select from inside (worker)
- # signal handlers. However, workers *close* the pipe descriptors in
- # the signal handlers to raise EBADF in IO.select instead of writing
- # like we do in the master. We cannot easily use the reader set for
- # IO.select because LISTENERS is already that set, and it's extra
- # work (and cycles) to distinguish the pipe FD from the reader set
- # once IO.select returns. So we're lazy and just close the pipe when
- # a (rare) signal arrives in the worker and reinitialize the pipe later.
+ # * The workers immediately close the pipe they inherit. See the
+ # Unicorn::Worker class for the pipe workers use.
SELF_PIPE = []
# signal queue used for self-piping
@@ -127,7 +119,7 @@ class Unicorn::HttpServer
inherit_listeners!
# this pipe is used to wake us up from select(2) in #join when signals
# are trapped. See trap_deferred.
- init_self_pipe!
+ SELF_PIPE.replace(Unicorn.pipe)
# setup signal handlers before writing pid file in case people get
# trigger happy and send signals as soon as the pid file exists.
@@ -306,14 +298,14 @@ class Unicorn::HttpServer
logger.info "master reopening logs..."
Unicorn::Util.reopen_logs
logger.info "master done reopening logs"
- kill_each_worker(:USR1)
+ soft_kill_each_worker(:USR1)
when :USR2 # exec binary, stay alive in case something went wrong
reexec
when :WINCH
if Unicorn::Configurator::RACKUP[:daemonized]
respawn = false
logger.info "gracefully stopping all workers"
- kill_each_worker(:QUIT)
+ soft_kill_each_worker(:QUIT)
self.worker_processes = 0
else
logger.info "SIGWINCH ignored because we're not daemonized"
@@ -345,7 +337,11 @@ class Unicorn::HttpServer
self.listeners = []
limit = Time.now + timeout
until WORKERS.empty? || Time.now > limit
- kill_each_worker(graceful ? :QUIT : :TERM)
+ if graceful
+ soft_kill_each_worker(:QUIT)
+ else
+ kill_each_worker(:TERM)
+ end
sleep(0.1)
reap_all_workers
end
@@ -498,6 +494,7 @@ class Unicorn::HttpServer
end
def after_fork_internal
+ SELF_PIPE.each { |io| io.close }.clear # this is master-only, now
@ready_pipe.close if @ready_pipe
Unicorn::Configurator::RACKUP.clear
@ready_pipe = @init_listeners = @before_exec = @before_fork = nil
@@ -517,6 +514,7 @@ class Unicorn::HttpServer
before_fork.call(self, worker)
if pid = fork
WORKERS[pid] = worker
+ worker.atfork_parent
else
after_fork_internal
worker_loop(worker)
@@ -531,9 +529,7 @@ class Unicorn::HttpServer
def maintain_worker_count
(off = WORKERS.size - worker_processes) == 0 and return
off < 0 and return spawn_missing_workers
- WORKERS.dup.each_pair { |wpid,w|
- w.nr >= worker_processes and kill_worker(:QUIT, wpid) rescue nil
- }
+ WORKERS.each_value { |w| w.nr >= worker_processes and w.soft_kill(:QUIT) }
end
# if we get any error, try to write something back to the client
@@ -600,6 +596,7 @@ class Unicorn::HttpServer
# traps for USR1, USR2, and HUP may be set in the after_fork Proc
# by the user.
def init_worker_process(worker)
+ worker.atfork_child
# we'll re-trap :QUIT later for graceful shutdown iff we accept clients
EXIT_SIGS.each { |sig| trap(sig) { exit!(0) } }
exit!(0) if (SIG_QUEUE & EXIT_SIGS)[0]
@@ -608,23 +605,27 @@ class Unicorn::HttpServer
SIG_QUEUE.clear
proc_name "worker[#{worker.nr}]"
START_CTX.clear
- init_self_pipe!
WORKERS.clear
+
+ after_fork.call(self, worker) # can drop perms and create listeners
LISTENERS.each { |sock| sock.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) }
- after_fork.call(self, worker) # can drop perms
+
worker.user(*user) if user.kind_of?(Array) && ! worker.switched
self.timeout /= 2.0 # halve it for select()
@config = nil
build_app! unless preload_app
ssl_enable!
@after_fork = @listener_opts = @orig_app = nil
+ readers = LISTENERS.dup
+ readers << worker
+ trap(:QUIT) { readers.each { |io| io.close }.replace([false]) }
+ readers
end
def reopen_worker_logs(worker_nr)
logger.info "worker=#{worker_nr} reopening logs..."
Unicorn::Util.reopen_logs
logger.info "worker=#{worker_nr} done reopening logs"
- init_self_pipe!
rescue => e
logger.error(e) rescue nil
exit!(77) # EX_NOPERM in sysexits.h
@@ -635,22 +636,24 @@ class Unicorn::HttpServer
# given a INT, QUIT, or TERM signal)
def worker_loop(worker)
ppid = master_pid
- init_worker_process(worker)
+ readers = init_worker_process(worker)
nr = 0 # this becomes negative if we need to reopen logs
- l = LISTENERS.dup
- ready = l.dup
- # closing anything we IO.select on will raise EBADF
- trap(:USR1) { nr = -65536; SELF_PIPE[0].close rescue nil }
- trap(:QUIT) { worker = nil; LISTENERS.each { |s| s.close rescue nil }.clear }
- logger.info "worker=#{worker.nr} ready"
+ # this only works immediately if the master sent us the signal
+ # (which is the normal case)
+ trap(:USR1) { nr = -65536 }
+
+ ready = readers.dup
+ @logger.info "worker=#{worker.nr} ready"
begin
nr < 0 and reopen_worker_logs(worker.nr)
nr = 0
-
worker.tick = Time.now.to_i
- while sock = ready.shift
+ tmp = ready.dup
+ while sock = tmp.shift
+ # Unicorn::Worker#kgio_tryaccept is not like accept(2) at all,
+ # but that will return false
if client = sock.kgio_tryaccept
process_client(client)
nr += 1
@@ -663,8 +666,8 @@ class Unicorn::HttpServer
# we're probably reasonably busy, so avoid calling select()
# and do a speculative non-blocking accept() on ready listeners
# before we sleep again in select().
- unless nr == 0 # (nr < 0) => reopen logs (unlikely)
- ready = l.dup
+ unless nr == 0
+ tmp = ready.dup
redo
end
@@ -672,11 +675,11 @@ class Unicorn::HttpServer
# timeout used so we can detect parent death:
worker.tick = Time.now.to_i
- ret = IO.select(l, nil, SELF_PIPE, @timeout) and ready = ret[0]
+ ret = IO.select(readers, nil, nil, @timeout) and ready = ret[0]
rescue => e
- redo if nr < 0 && (Errno::EBADF === e || IOError === e) # reopen logs
- Unicorn.log_error(@logger, "listen loop error", e) if worker
- end while worker
+ redo if nr < 0
+ Unicorn.log_error(@logger, "listen loop error", e) if readers[0]
+ end while readers[0]
end
# delivers a signal to a worker and fails gracefully if the worker
@@ -692,6 +695,10 @@ class Unicorn::HttpServer
WORKERS.keys.each { |wpid| kill_worker(signal, wpid) }
end
+ def soft_kill_each_worker(signal)
+ WORKERS.each_value { |worker| worker.soft_kill(signal) }
+ end
+
# unlinks a PID file at given +path+ if it contains the current PID
# still potentially racy without locking the directory (which is
# non-portable and may interact badly with other programs), but the
@@ -720,7 +727,7 @@ class Unicorn::HttpServer
config[:listeners].replace(@init_listeners)
config.reload
config.commit!(self)
- kill_each_worker(:QUIT)
+ soft_kill_each_worker(:QUIT)
Unicorn::Util.reopen_logs
self.app = orig_app
build_app! if preload_app
@@ -756,12 +763,6 @@ class Unicorn::HttpServer
io.sync = true
end
- def init_self_pipe!
- SELF_PIPE.each { |io| io.close rescue nil }
- SELF_PIPE.replace(Kgio::Pipe.new)
- SELF_PIPE.each { |io| io.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) }
- end
-
def inherit_listeners!
# inherit sockets from parents, they need to be plain Socket objects
# before they become Kgio::UNIXServer or Kgio::TCPServer
diff --git a/lib/unicorn/worker.rb b/lib/unicorn/worker.rb
index 1fb6a4a..e74a1c9 100644
--- a/lib/unicorn/worker.rb
+++ b/lib/unicorn/worker.rb
@@ -12,6 +12,7 @@ class Unicorn::Worker
# :stopdoc:
attr_accessor :nr, :switched
attr_writer :tmp
+ attr_reader :to_io # IO.select-compatible
PER_DROP = Raindrops::PAGE_SIZE / Raindrops::SIZE
DROPS = []
@@ -23,6 +24,66 @@ class Unicorn::Worker
@raindrop[@offset] = 0
@nr = nr
@tmp = @switched = false
+ @to_io, @master = Unicorn.pipe
+ end
+
+ def atfork_child # :nodoc:
+ # we _must_ close in child, parent just holds this open to signal
+ @master = @master.close
+ end
+
+ # master fakes SIGQUIT using this
+ def quit # :nodoc:
+ @master = @master.close if @master
+ end
+
+ # parent does not read
+ def atfork_parent # :nodoc:
+ @to_io = @to_io.close
+ end
+
+ # call a signal handler immediately without triggering EINTR
+ # We do not use the more obvious Process.kill(sig, $$) here since
+ # that signal delivery may be deferred. We want to avoid signal delivery
+ # while the Rack app.call is running because some database drivers
+ # (e.g. ruby-pg) may cancel pending requests.
+ def fake_sig(sig) # :nodoc:
+ old_cb = trap(sig, "IGNORE")
+ old_cb.call
+ ensure
+ trap(sig, old_cb)
+ end
+
+ # master sends fake signals to children
+ def soft_kill(sig) # :nodoc:
+ case sig
+ when Integer
+ signum = sig
+ else
+ signum = Signal.list[sig.to_s] or
+ raise ArgumentError, "BUG: bad signal: #{sig.inspect}"
+ end
+ # writing and reading 4 bytes on a pipe is atomic on all POSIX platforms
+ # Do not care in the odd case the buffer is full, here.
+ @master.kgio_trywrite([signum].pack('l'))
+ rescue Errno::EPIPE
+ # worker will be reaped soon
+ end
+
+ # this only runs when the Rack app.call is not running
+ # act like a listener
+ def kgio_tryaccept # :nodoc:
+ case buf = @to_io.kgio_tryread(4)
+ when String
+ # unpack the buffer and trigger the signal handler
+ signum = buf.unpack('l')
+ fake_sig(signum[0])
+ # keep looping, more signals may be queued
+ when nil # EOF: master died, but we are at a safe place to exit
+ fake_sig(:QUIT)
+ when :wait_readable # keep waiting
+ return false
+ end while true # loop, as multiple signals may be sent
end
# worker objects may be compared to just plain Integers
@@ -49,8 +110,11 @@ class Unicorn::Worker
end
end
+ # called in both the master (reaping worker) and worker (SIGQUIT handler)
def close # :nodoc:
@tmp.close if @tmp
+ @master.close if @master
+ @to_io.close if @to_io
end
# :startdoc:
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 1%]
* Re: [PATCH] rework master-to-worker signaling to use a pipe
2013-12-09 9:52 1% [PATCH] rework master-to-worker signaling to use a pipe Eric Wong
@ 2013-12-10 0:22 0% ` Sam Saffron
0 siblings, 0 replies; 200+ results
From: Sam Saffron @ 2013-12-10 0:22 UTC (permalink / raw)
To: unicorn list
Eric,
Your work on unicorn is exemplary and responsiveness second to none. I
did want to take a full circle back though on the issue.
After discussing with tmm1 and ko1 the conclusion is that pg is using
ubfs incorrectly. They are just there to unblock io, not as a
termination hook, once io is unblocked you still need to be able to
re-acquire. pg was using ubfs as termination hooks. See full
discussion at: https://groups.google.com/forum/#!topic/ruby-pg/5_ylGmog1S4
The end result is that I suggested Lars open a ticket for redmine if
we need a ubfs to act as termination hooks, the change to pg was
reverted.
Bottom line is that your change is not really required.
Thanks you so much for being super responsive here. Sorry if I caused
you to go on a tangent you did need to go on.
Sam
On Mon, Dec 9, 2013 at 8:52 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Signaling using normal kill(2) is preserved, but the master now
> prefers to signal workers using a pipe rather than kill(2).
> Non-graceful signals (:TERM/:KILL) are still sent using kill(2),
> as they ask for immediate shutdown.
>
> This change is necessary to avoid triggering the ubf (unblocking
> function) for rb_thread_call_without_gvl (and similar) functions
> extensions. Most notably, this fixes compatibility with newer
> versions of the 'pg' gem which will cancel a running DB query if
> signaled[1].
>
> This also has the nice side-effect of allowing a premature
> master death (assuming preload_app didn't cause the master to
> spawn off rogue child daemons).
>
> Note: users should also refrain from using "killall" if using the
> 'pg' gem or something like it.
>
> Unfortunately, this increases FD usage in the master as the writable
> end of the pipe is preserved in the master. This limit the number
> of worker processes the master may run to the open file limit of the
> master process. Increasing the open file limit of the master
> process may be needed. However, the FD use on the workers is
> reduced by one as the internal self-pipe is no longer used. Thus,
> overall pipe allocation for the kernel remains unchanged.
>
> [1] - pg is correct to cancel a query, as it cannot know if
> the signal was for a) graceful unicorn shutdown or
> b) oh-noes-I-started-a-bad-query-ABORT-ABORT-ABORT!!
> ---
> Pushed to master on git://bogomips.org/unicorn.git
> commit 6f6e4115b4bb03e5e7c55def91527799190566f2
>
> SIGNALS | 6 ++++
> lib/unicorn.rb | 5 +++
> lib/unicorn/http_server.rb | 85 +++++++++++++++++++++++-----------------------
> lib/unicorn/worker.rb | 64 ++++++++++++++++++++++++++++++++++
> 4 files changed, 118 insertions(+), 42 deletions(-)
>
> diff --git a/SIGNALS b/SIGNALS
> index 48c651e..ef0b0d9 100644
> --- a/SIGNALS
> +++ b/SIGNALS
> @@ -45,6 +45,10 @@ http://unicorn.bogomips.org/examples/init.sh
>
> === Worker Processes
>
> +Note: as of unicorn 4.8, the master uses a pipe to signal workers
> +instead of kill(2) for most cases. Using signals still (and works and
> +remains supported for external tools/libraries), however.
> +
> Sending signals directly to the worker processes should not normally be
> needed. If the master process is running, any exited worker will be
> automatically respawned.
> @@ -52,6 +56,8 @@ automatically respawned.
> * INT/TERM - Quick shutdown, immediately exit.
> Unless WINCH has been sent to the master (or the master is killed),
> the master process will respawn a worker to replace this one.
> + Immediate shutdown is still triggered using kill(2) and not the
> + internal pipe as of unicorn 4.8
>
> * QUIT - Gracefully exit after finishing the current request.
> Unless WINCH has been sent to the master (or the master is killed),
> diff --git a/lib/unicorn.rb b/lib/unicorn.rb
> index 2535159..638b846 100644
> --- a/lib/unicorn.rb
> +++ b/lib/unicorn.rb
> @@ -97,6 +97,11 @@ module Unicorn
> logger.error "#{prefix}: #{message} (#{exc.class})"
> exc.backtrace.each { |line| logger.error(line) }
> end
> +
> + # remove this when we only support Ruby >= 2.0
> + def self.pipe # :nodoc:
> + Kgio::Pipe.new.each { |io| io.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) }
> + end
> # :startdoc:
> end
> # :enddoc:
> diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
> index f15c8a7..ae8ad13 100644
> --- a/lib/unicorn/http_server.rb
> +++ b/lib/unicorn/http_server.rb
> @@ -42,16 +42,8 @@ class Unicorn::HttpServer
> # it to wake up the master from IO.select in exactly the same manner
> # djb describes in http://cr.yp.to/docs/selfpipe.html
> #
> - # * The workers immediately close the pipe they inherit from the
> - # master and replace it with a new pipe after forking. This new
> - # pipe is also used to wakeup from IO.select from inside (worker)
> - # signal handlers. However, workers *close* the pipe descriptors in
> - # the signal handlers to raise EBADF in IO.select instead of writing
> - # like we do in the master. We cannot easily use the reader set for
> - # IO.select because LISTENERS is already that set, and it's extra
> - # work (and cycles) to distinguish the pipe FD from the reader set
> - # once IO.select returns. So we're lazy and just close the pipe when
> - # a (rare) signal arrives in the worker and reinitialize the pipe later.
> + # * The workers immediately close the pipe they inherit. See the
> + # Unicorn::Worker class for the pipe workers use.
> SELF_PIPE = []
>
> # signal queue used for self-piping
> @@ -127,7 +119,7 @@ class Unicorn::HttpServer
> inherit_listeners!
> # this pipe is used to wake us up from select(2) in #join when signals
> # are trapped. See trap_deferred.
> - init_self_pipe!
> + SELF_PIPE.replace(Unicorn.pipe)
>
> # setup signal handlers before writing pid file in case people get
> # trigger happy and send signals as soon as the pid file exists.
> @@ -306,14 +298,14 @@ class Unicorn::HttpServer
> logger.info "master reopening logs..."
> Unicorn::Util.reopen_logs
> logger.info "master done reopening logs"
> - kill_each_worker(:USR1)
> + soft_kill_each_worker(:USR1)
> when :USR2 # exec binary, stay alive in case something went wrong
> reexec
> when :WINCH
> if Unicorn::Configurator::RACKUP[:daemonized]
> respawn = false
> logger.info "gracefully stopping all workers"
> - kill_each_worker(:QUIT)
> + soft_kill_each_worker(:QUIT)
> self.worker_processes = 0
> else
> logger.info "SIGWINCH ignored because we're not daemonized"
> @@ -345,7 +337,11 @@ class Unicorn::HttpServer
> self.listeners = []
> limit = Time.now + timeout
> until WORKERS.empty? || Time.now > limit
> - kill_each_worker(graceful ? :QUIT : :TERM)
> + if graceful
> + soft_kill_each_worker(:QUIT)
> + else
> + kill_each_worker(:TERM)
> + end
> sleep(0.1)
> reap_all_workers
> end
> @@ -498,6 +494,7 @@ class Unicorn::HttpServer
> end
>
> def after_fork_internal
> + SELF_PIPE.each { |io| io.close }.clear # this is master-only, now
> @ready_pipe.close if @ready_pipe
> Unicorn::Configurator::RACKUP.clear
> @ready_pipe = @init_listeners = @before_exec = @before_fork = nil
> @@ -517,6 +514,7 @@ class Unicorn::HttpServer
> before_fork.call(self, worker)
> if pid = fork
> WORKERS[pid] = worker
> + worker.atfork_parent
> else
> after_fork_internal
> worker_loop(worker)
> @@ -531,9 +529,7 @@ class Unicorn::HttpServer
> def maintain_worker_count
> (off = WORKERS.size - worker_processes) == 0 and return
> off < 0 and return spawn_missing_workers
> - WORKERS.dup.each_pair { |wpid,w|
> - w.nr >= worker_processes and kill_worker(:QUIT, wpid) rescue nil
> - }
> + WORKERS.each_value { |w| w.nr >= worker_processes and w.soft_kill(:QUIT) }
> end
>
> # if we get any error, try to write something back to the client
> @@ -600,6 +596,7 @@ class Unicorn::HttpServer
> # traps for USR1, USR2, and HUP may be set in the after_fork Proc
> # by the user.
> def init_worker_process(worker)
> + worker.atfork_child
> # we'll re-trap :QUIT later for graceful shutdown iff we accept clients
> EXIT_SIGS.each { |sig| trap(sig) { exit!(0) } }
> exit!(0) if (SIG_QUEUE & EXIT_SIGS)[0]
> @@ -608,23 +605,27 @@ class Unicorn::HttpServer
> SIG_QUEUE.clear
> proc_name "worker[#{worker.nr}]"
> START_CTX.clear
> - init_self_pipe!
> WORKERS.clear
> +
> + after_fork.call(self, worker) # can drop perms and create listeners
> LISTENERS.each { |sock| sock.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) }
> - after_fork.call(self, worker) # can drop perms
> +
> worker.user(*user) if user.kind_of?(Array) && ! worker.switched
> self.timeout /= 2.0 # halve it for select()
> @config = nil
> build_app! unless preload_app
> ssl_enable!
> @after_fork = @listener_opts = @orig_app = nil
> + readers = LISTENERS.dup
> + readers << worker
> + trap(:QUIT) { readers.each { |io| io.close }.replace([false]) }
> + readers
> end
>
> def reopen_worker_logs(worker_nr)
> logger.info "worker=#{worker_nr} reopening logs..."
> Unicorn::Util.reopen_logs
> logger.info "worker=#{worker_nr} done reopening logs"
> - init_self_pipe!
> rescue => e
> logger.error(e) rescue nil
> exit!(77) # EX_NOPERM in sysexits.h
> @@ -635,22 +636,24 @@ class Unicorn::HttpServer
> # given a INT, QUIT, or TERM signal)
> def worker_loop(worker)
> ppid = master_pid
> - init_worker_process(worker)
> + readers = init_worker_process(worker)
> nr = 0 # this becomes negative if we need to reopen logs
> - l = LISTENERS.dup
> - ready = l.dup
>
> - # closing anything we IO.select on will raise EBADF
> - trap(:USR1) { nr = -65536; SELF_PIPE[0].close rescue nil }
> - trap(:QUIT) { worker = nil; LISTENERS.each { |s| s.close rescue nil }.clear }
> - logger.info "worker=#{worker.nr} ready"
> + # this only works immediately if the master sent us the signal
> + # (which is the normal case)
> + trap(:USR1) { nr = -65536 }
> +
> + ready = readers.dup
> + @logger.info "worker=#{worker.nr} ready"
>
> begin
> nr < 0 and reopen_worker_logs(worker.nr)
> nr = 0
> -
> worker.tick = Time.now.to_i
> - while sock = ready.shift
> + tmp = ready.dup
> + while sock = tmp.shift
> + # Unicorn::Worker#kgio_tryaccept is not like accept(2) at all,
> + # but that will return false
> if client = sock.kgio_tryaccept
> process_client(client)
> nr += 1
> @@ -663,8 +666,8 @@ class Unicorn::HttpServer
> # we're probably reasonably busy, so avoid calling select()
> # and do a speculative non-blocking accept() on ready listeners
> # before we sleep again in select().
> - unless nr == 0 # (nr < 0) => reopen logs (unlikely)
> - ready = l.dup
> + unless nr == 0
> + tmp = ready.dup
> redo
> end
>
> @@ -672,11 +675,11 @@ class Unicorn::HttpServer
>
> # timeout used so we can detect parent death:
> worker.tick = Time.now.to_i
> - ret = IO.select(l, nil, SELF_PIPE, @timeout) and ready = ret[0]
> + ret = IO.select(readers, nil, nil, @timeout) and ready = ret[0]
> rescue => e
> - redo if nr < 0 && (Errno::EBADF === e || IOError === e) # reopen logs
> - Unicorn.log_error(@logger, "listen loop error", e) if worker
> - end while worker
> + redo if nr < 0
> + Unicorn.log_error(@logger, "listen loop error", e) if readers[0]
> + end while readers[0]
> end
>
> # delivers a signal to a worker and fails gracefully if the worker
> @@ -692,6 +695,10 @@ class Unicorn::HttpServer
> WORKERS.keys.each { |wpid| kill_worker(signal, wpid) }
> end
>
> + def soft_kill_each_worker(signal)
> + WORKERS.each_value { |worker| worker.soft_kill(signal) }
> + end
> +
> # unlinks a PID file at given +path+ if it contains the current PID
> # still potentially racy without locking the directory (which is
> # non-portable and may interact badly with other programs), but the
> @@ -720,7 +727,7 @@ class Unicorn::HttpServer
> config[:listeners].replace(@init_listeners)
> config.reload
> config.commit!(self)
> - kill_each_worker(:QUIT)
> + soft_kill_each_worker(:QUIT)
> Unicorn::Util.reopen_logs
> self.app = orig_app
> build_app! if preload_app
> @@ -756,12 +763,6 @@ class Unicorn::HttpServer
> io.sync = true
> end
>
> - def init_self_pipe!
> - SELF_PIPE.each { |io| io.close rescue nil }
> - SELF_PIPE.replace(Kgio::Pipe.new)
> - SELF_PIPE.each { |io| io.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) }
> - end
> -
> def inherit_listeners!
> # inherit sockets from parents, they need to be plain Socket objects
> # before they become Kgio::UNIXServer or Kgio::TCPServer
> diff --git a/lib/unicorn/worker.rb b/lib/unicorn/worker.rb
> index 1fb6a4a..e74a1c9 100644
> --- a/lib/unicorn/worker.rb
> +++ b/lib/unicorn/worker.rb
> @@ -12,6 +12,7 @@ class Unicorn::Worker
> # :stopdoc:
> attr_accessor :nr, :switched
> attr_writer :tmp
> + attr_reader :to_io # IO.select-compatible
>
> PER_DROP = Raindrops::PAGE_SIZE / Raindrops::SIZE
> DROPS = []
> @@ -23,6 +24,66 @@ class Unicorn::Worker
> @raindrop[@offset] = 0
> @nr = nr
> @tmp = @switched = false
> + @to_io, @master = Unicorn.pipe
> + end
> +
> + def atfork_child # :nodoc:
> + # we _must_ close in child, parent just holds this open to signal
> + @master = @master.close
> + end
> +
> + # master fakes SIGQUIT using this
> + def quit # :nodoc:
> + @master = @master.close if @master
> + end
> +
> + # parent does not read
> + def atfork_parent # :nodoc:
> + @to_io = @to_io.close
> + end
> +
> + # call a signal handler immediately without triggering EINTR
> + # We do not use the more obvious Process.kill(sig, $$) here since
> + # that signal delivery may be deferred. We want to avoid signal delivery
> + # while the Rack app.call is running because some database drivers
> + # (e.g. ruby-pg) may cancel pending requests.
> + def fake_sig(sig) # :nodoc:
> + old_cb = trap(sig, "IGNORE")
> + old_cb.call
> + ensure
> + trap(sig, old_cb)
> + end
> +
> + # master sends fake signals to children
> + def soft_kill(sig) # :nodoc:
> + case sig
> + when Integer
> + signum = sig
> + else
> + signum = Signal.list[sig.to_s] or
> + raise ArgumentError, "BUG: bad signal: #{sig.inspect}"
> + end
> + # writing and reading 4 bytes on a pipe is atomic on all POSIX platforms
> + # Do not care in the odd case the buffer is full, here.
> + @master.kgio_trywrite([signum].pack('l'))
> + rescue Errno::EPIPE
> + # worker will be reaped soon
> + end
> +
> + # this only runs when the Rack app.call is not running
> + # act like a listener
> + def kgio_tryaccept # :nodoc:
> + case buf = @to_io.kgio_tryread(4)
> + when String
> + # unpack the buffer and trigger the signal handler
> + signum = buf.unpack('l')
> + fake_sig(signum[0])
> + # keep looping, more signals may be queued
> + when nil # EOF: master died, but we are at a safe place to exit
> + fake_sig(:QUIT)
> + when :wait_readable # keep waiting
> + return false
> + end while true # loop, as multiple signals may be sent
> end
>
> # worker objects may be compared to just plain Integers
> @@ -49,8 +110,11 @@ class Unicorn::Worker
> end
> end
>
> + # called in both the master (reaping worker) and worker (SIGQUIT handler)
> def close # :nodoc:
> @tmp.close if @tmp
> + @master.close if @master
> + @to_io.close if @to_io
> end
>
> # :startdoc:
> --
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* [ANN] unicorn 4.8.0 - big internal changes, but compatible
@ 2014-01-11 7:40 2% Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2014-01-11 7:40 UTC (permalink / raw)
To: mongrel-unicorn
Changes:
This release contains fairly major internal workings of master-to-worker
notifications. The master process no longer sends signals to workers
for most tasks. This works around some compatibility issues with some
versions of the "pg" gem (and potentially any other code which may not
handle EINTR properly). One extra benefit is it also helps stray
workers notice a rare, unexpected master death more easily. Workers
continue to (and will always) accept existing signals for compatibility
with tools/scripts which may signal workers.
PID file are always written early (even on upgrade) again to avoid
breaking strange monitoring setups which use PID files. Keep in mind we
have always discouraged monitoring based on PID files as they are
fragile.
We now avoid bubbling IOError to the Rack app on premature client
disconnects when streaming the input body. This is usually not a
problem with nginx, but may be on some LAN setups without nginx).
Thanks to Sam Saffron, Jimmy Soho, Rodrigo Rosenfeld Rosas,
Michael Fischer, and Andrew Hobson for their help with this release.
Note: the unicorn mailing list will be moved/changed soon due to the
RubyForge shutdown. unicorn will always rely only on Free Software.
There will never be any sign-up requirements nor terms-of-service to
agree to when communicating with us.
* http://unicorn.bogomips.org/
* mongrel-unicorn@rubyforge.org
* git://bogomips.org/unicorn.git
* http://unicorn.bogomips.org/NEWS.atom.xml
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Different behavior with pid files and SIGUSR2
@ 2014-01-28 20:03 2% ` Eric Wong
2014-01-28 20:08 0% ` Michael Graff
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2014-01-28 20:03 UTC (permalink / raw)
To: unicorn list; +Cc: Michael Graff
Michael Graff <explorer@flame.org> wrote:
> What I appear to be seeing now, with 4.7.0, is:
>
> (1) The PID file is removed.
> (2) The new instance spins up, but until it starts its first worker, no .pid file is created.
>
> I don't mind the new behavior particularly, but it did surprise me a bit. Is this new behavior working as intended?
Yes in 4.7.0, but Jimmy convinced me to change it so the PID is written
early again in 4.8.0 (also pushing out 4.8.1 soon)
ref: http://mid.gmane.org/CAHStS5gFYcPBDxkVizAHrOeDKAkjT69kruFdgaY0CbB+vLbK8Q@mail.gmail.com
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Different behavior with pid files and SIGUSR2
2014-01-28 20:03 2% ` Eric Wong
@ 2014-01-28 20:08 0% ` Michael Graff
0 siblings, 0 replies; 200+ results
From: Michael Graff @ 2014-01-28 20:08 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn list
Good deal. This was breaking my restart script in a non-obvious way. Granted, my restart script should be smarter...
What is the preferred way to start-on-server-reboot and upgrade-in-place these days? I have an older linux-expecting init.d-style script I've been using for years, so it's likely time to consider alternatives.
--Michael
On Jan 28, 2014, at 2:03 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Michael Graff <explorer@flame.org> wrote:
>> What I appear to be seeing now, with 4.7.0, is:
>>
>> (1) The PID file is removed.
>> (2) The new instance spins up, but until it starts its first worker, no .pid file is created.
>>
>> I don't mind the new behavior particularly, but it did surprise me a bit. Is this new behavior working as intended?
>
> Yes in 4.7.0, but Jimmy convinced me to change it so the PID is written
> early again in 4.8.0 (also pushing out 4.8.1 soon)
>
> ref: http://mid.gmane.org/CAHStS5gFYcPBDxkVizAHrOeDKAkjT69kruFdgaY0CbB+vLbK8Q@mail.gmail.com
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Stucks DB connections
@ 2014-02-24 12:33 2% Ilya Bazylchuk
2014-02-24 18:25 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Ilya Bazylchuk @ 2014-02-24 12:33 UTC (permalink / raw)
To: mongrel-unicorn
Hey Guys,
We have problems with DB connection in unicorn. When workers work some hours we see in Postgres activity stucks connections with query "COMMIT". After restart unicorn it work fine some time, then connections stucks again.
database.yml
adapter: postgis
postgis_extension: true
schema_search_path: public,postgis
encoding: utf8
pool: 5
checkout_timeout: 10
unicorn 4.8.2
pg 0.17.1
rails 3.2.17
unicorn use with nginx
Can you help with something?
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 2%]
* Re: Stucks DB connections
2014-02-24 12:33 2% Stucks DB connections Ilya Bazylchuk
@ 2014-02-24 18:25 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2014-02-24 18:25 UTC (permalink / raw)
To: unicorn list; +Cc: Ilya Bazylchuk
Ilya Bazylchuk <ilya.bazylchuk@gmail.com> wrote:
> Hey Guys,
>
> We have problems with DB connection in unicorn. When workers work some
> hours we see in Postgres activity stucks connections with query
> "COMMIT". After restart unicorn it work fine some time, then
> connections stucks again.
>
> database.yml
>
> adapter: postgis
> postgis_extension: true
> schema_search_path: public,postgis
> encoding: utf8
> pool: 5
> checkout_timeout: 10
>
> unicorn 4.8.2
> pg 0.17.1
> rails 3.2.17
I'm not familiar with postgis. Is there any sort of idle timeout on the
database side? Is this high or low traffic when things get stuck?
I think the postgis/pg adapter guys might be able to help, too, since
unicorn doesn't do anything with the DB connections itself.
Some generic, DB-independent thoughts/questions:
Sometimes, tiny socket buffer sizes with the DB connection might hit this.
Any non-standard kernel/socket knobs or tuning?
Also, if running Linux, which kernel version are you running? If it's
3.7, make sure commit 8fb74b9fb2b182d54beee592350d9ea1f325917a
(mm: compaction: partially revert capture of suitable high-order page)
got backported.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: [PATCH] tests: switch to minitest
@ 2014-04-26 6:07 2% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2014-04-26 6:07 UTC (permalink / raw)
To: Ken Dreyer; +Cc: unicorn-public, mongrel-unicorn
Ken Dreyer <ktdreyer@ktdreyer.com> wrote:
> Ruby 1.9+ uses Minitest as the backend for Test::Unit. As of Minitest 5,
> the shim has lost some backwards compatibility. It is time to make the
> jump to minitest.
>
> Adjust the unicorn test suite to support Minitest 5's syntax.
Thank you very much for taking a look at this. I was going to do it
myself over the summer.
I ran into some problems with our tests forking + minitest/autorun
causing some problems due to at_exit usage.
We probably need to workaround, I already did something for yahns
which we can reuse[1] in unicorn.
> Minitest versions 4 and below do not support the newer Minitest::Test
> class that arrived in version 5. For that case, use the
> MiniTest::Unit::TestCase class as a fallback.
For yahns, I did the following:
----------- git://yhbt.net/yahns -- test/helper.rb ------------------
gem 'minitest'
begin # favor minitest 5
require 'minitest'
Testcase = Minitest::Test
mtobj = Minitest
rescue NameError, LoadError # but support minitest 4
require 'minitest/unit'
Testcase = Minitest::Unit::TestCase
mtobj = MiniTest::Unit.new
end
# Not using minitest/autorun because that doesn't guard against redundant
# extra runs with fork. We cannot use exit! in the tests either
# (since users/apps hosted on yahns _should_ expect exit, not exit!).
TSTART_PID = $$
at_exit do
# skipping @@after_run stuff in minitest since we don't need it
case $!
when nil, SystemExit
exit(mtobj.run(ARGV)) if $$ == TSTART_PID
end
end
-------------------------------- 8< -------------------------------
I'd appreciate a second opinion on what I did with yahns above,
but it should work with unicorn.
Can you take a look at integrating+fixing those cases which fork?
Thanks again.
> Please keep me in the CC as I'm not subscribed to the unicorn email list.
No problem. Also keeping unicorn-public@bogomips.org Cc-ed since that's
the new public-inbox[2] address and reply-all will be the norm there.
Hopefully public-inbox can encourage more drive-by contributors like
you :)
[1] - Fwiw, I give myself permission to relicense any yahns GPLv3+
test code I wrote to (GPLv2+ || Ruby 1.8 license) for unicorn.
[2] - http://public-inbox.org/
^ permalink raw reply [relevance 2%]
* Re: [ANN] unicorn 4.8.3 - the end of an era
@ 2014-05-07 21:25 3% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2014-05-07 21:25 UTC (permalink / raw)
To: Michael Fischer
Cc: Eric Wong, Jérémy Lecour, Liste Unicorn, unicorn-public
Michael Fischer <mfischer@zendesk.com> wrote:
> Is there some compelling reason why the mailing list simply cannot be
> moved to another provider? IMHO your users and fellow developers
> shouldn't have to do anything other than change the submission
> address.
We would need to migrate again if/when that provider goes dead or
service starts deteriorating. Ease-of-migration and being forkable
again in the future was the top priority.
If I'm hit by a bus or start allowing too much spam, it should be
trivially easy to migrate the project[1] and all its archives and
infrastructure.
> I respect your desire to power the communication platform with free
> software (and I'm sure this can still be done with Mailman or
> whatever), but keep in mind the practical reality of our time, where
> most of us these days are now comfortably using Webmail or or POP/IMAP
> against a remote server that's not under the user's control and have
> no desire to implement yet another communication conduit.
I will probably take the addresses of active subscribers who've posted
here[2] imported into the new delivery system, even.
It would be great to be able to make the list of ML subscribers public,
too, to ensure forkability. I'm not sure how the lurkers will react to
that, though...
[1] of course, whoever takes over may not be a Free Software zealot
like myself.
[2] those addresses are already public, but lurkers will probably
have to resubscribe (or use ssoma or the Atom feed).
^ permalink raw reply [relevance 3%]
* Re: Log reopening broken on Rails 4 with config.autoflush_log = false?
@ 2014-07-04 9:13 2% ` Cedric Maion
0 siblings, 0 replies; 200+ results
From: Cedric Maion @ 2014-07-04 9:13 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn-public
Hi Eric,
Thanks for your answer (and update to the FAQ ;) ).
I'm playing with autoflush_log = true and see if I find a performance
impact (I'm logging approx 20KB per sec).
sys.vm.dirty* sysctls are already changed, but with low ratios as those
systems are also writing large files (and I don't want those pages in
memory for too long).
If a performance impact can be measured, I believe that I will turn
autoflush_log to false again and modify Unicorn::Util#is_log? to accept
!fp.sync ;) (I'm willing to accept occasional log lines corruption esp
during rotation)... or just restart unicorn after rotation instead.
Or, thinking out loud... keep autoflush_log to false and have unicorn
flush logs itself regularly, outside the HTTP request path (and when
reopening logs), to avoid blocking on IO during a log flush in the
middle of a request? (I need to refresh myself on the sizes of the ruby
& system IO buffers though, not sure if it's worth the pain).
Cedric
On Thu, Jul 03, 2014 at 05:24:10PM +0000, Eric Wong wrote:
>
> Cedric Maion <cedric@maion.com> wrote:
> > On a Rails 4.0 app with 'config.autoflush_log = false', it seems to me
> > that telling unicorn to reopen log files fails (e.g., after a log
> > rotation and USR1 signal, Rails/unicorn still writes to the old rotated
> > file instead of reopening a new one).
> >
> > Without this config option, logs does properly get reopened.
> >
> > Is this something known?
>
> Yes, log buffering via File#sync=true is incompatible with reopening in
> multiprocess servers. File#sync=true does not guarantee writes are
> atomic on line boundaries, so it's dangerous to assume they're logs.
>
> Even without reopening, enabling this option on a multiprocess server
> might corrupt logs (depending on the buffering implementation)
> because the synchronization is within each process rather for the
> entire OS.
>
> Looking at the rails source:
> ==> railties/lib/rails/application/bootstrap.rb <==
> f = File.open path, 'a'
> f.binmode
> f.sync = config.autoflush_log # if true make sure every write flushes
>
> You're not likely to notice any performance difference unless you're
> logging excessively. Keep in mind the Linux kernel also does buffering
> and you can tune that via the sys.vm.dirty_* sysctls.
^ permalink raw reply [relevance 2%]
* Re: Old unicorn workers still receives requests?
@ 2014-07-24 17:04 2% ` Eric Wong
2014-07-24 17:22 0% ` Bráulio Bhavamitra
0 siblings, 1 reply; 200+ results
From: Eric Wong @ 2014-07-24 17:04 UTC (permalink / raw)
To: Bráulio Bhavamitra; +Cc: unicorn-public
Bráulio Bhavamitra <braulio@eita.org.br> wrote:
> Hello all,
>
> I would like to know if old unicorn still receives and process requests in
> a QUIT signal is sent to the old master.
During heavy traffic, yes, there's a chance due to the optimization
in worker_loop (lib/unicorn/http_server.rb):
# make the following bet: if we accepted clients this round,
# we're probably reasonably busy, so avoid calling select()
# and do a speculative non-blocking accept() on ready listeners
# before we sleep again in select().
unless nr == 0
tmp = ready.dup
redo
end
Note, "receives" requests isn't correct, workers "pull" requests from
a central queue in the kernel.
^ permalink raw reply [relevance 2%]
* Re: Old unicorn workers still receives requests?
2014-07-24 17:04 2% ` Eric Wong
@ 2014-07-24 17:22 0% ` Bráulio Bhavamitra
0 siblings, 0 replies; 200+ results
From: Bráulio Bhavamitra @ 2014-07-24 17:22 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn-public
Thanks Eric! It is not clear to me if old workers have a lower priority to
pull requets than new workers.
cheers,
br=C3=A1ulio
On Thu, Jul 24, 2014 at 2:04 PM, Eric Wong <e@80x24.org> wrote:
> Br=C3=A1ulio Bhavamitra <braulio@eita.org.br> wrote:
> > Hello all,
> >
> > I would like to know if old unicorn still receives and process requests
> in
> > a QUIT signal is sent to the old master.
>
> During heavy traffic, yes, there's a chance due to the optimization
> in worker_loop (lib/unicorn/http_server.rb):
>
> # make the following bet: if we accepted clients this round,
> # we're probably reasonably busy, so avoid calling select()
> # and do a speculative non-blocking accept() on ready listeners
> # before we sleep again in select().
> unless nr =3D=3D 0
> tmp =3D ready.dup
> redo
> end
>
> Note, "receives" requests isn't correct, workers "pull" requests from
> a central queue in the kernel.
>
--=20
"Lute pela sua ideologia. Seja um com sua ideologia. Viva pela sua
ideologia. Morra por sua ideologia" P.R. Sarkar
EITA - Educa=C3=A7=C3=A3o, Informa=C3=A7=C3=A3o e Tecnologias para Autogest=
=C3=A3o
http://cirandas.net/brauliobo
http://eita.org.br
"Paramapurusha =C3=A9 meu pai e Parama Prakriti =C3=A9 minha m=C3=A3e. O un=
iverso =C3=A9 meu
lar e todos n=C3=B3s somos cidad=C3=A3os deste cosmo. Este universo =C3=A9 =
a imagina=C3=A7=C3=A3o da
Mente Macroc=C3=B3smica, e todas as entidades est=C3=A3o sendo criadas, pre=
servadas e
destru=C3=ADdas nas fases de extrovers=C3=A3o e introvers=C3=A3o do fluxo i=
maginativo
c=C3=B3smico. No =C3=A2mbito pessoal, quando uma pessoa imagina algo em sua=
mente,
naquele momento, essa pessoa =C3=A9 a =C3=BAnica propriet=C3=A1ria daquilo =
que ela
imagina, e ningu=C3=A9m mais. Quando um ser humano criado mentalmente camin=
ha
por um milharal tamb=C3=A9m imaginado, a pessoa imaginada n=C3=A3o =C3=A9 a=
propriedade
desse milharal, pois ele pertence ao indiv=C3=ADduo que o est=C3=A1 imagina=
ndo. Este
universo foi criado na imagina=C3=A7=C3=A3o de Brahma, a Entidade Suprema, =
por isso
a propriedade deste universo =C3=A9 de Brahma, e n=C3=A3o dos microcosmos q=
ue tamb=C3=A9m
foram criados pela imagina=C3=A7=C3=A3o de Brahma. Nenhuma propriedade dest=
e mundo,
mut=C3=A1vel ou imut=C3=A1vel, pertence a um indiv=C3=ADduo em particular; =
tudo =C3=A9 o
patrim=C3=B4nio comum de todos."
Restante do texto em
http://cirandas.net/brauliobo/blog/a-problematica-de-hoje-em-dia
^ permalink raw reply [relevance 0%]
* Re: Please move to github
@ 2014-08-02 19:07 1% ` Gary Grossman
2014-08-02 20:15 0% ` Eric Wong
0 siblings, 1 reply; 200+ results
From: Gary Grossman @ 2014-08-02 19:07 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn-public, michael
Hi Eric,
Thanks for your reply and for reviewing the patch!
>Right, the Rack spec does not dictate this. Doing this out-of-spec has
>the ability to break existing apps as well as compatibility with other
>app servers.
It's true, my patch is too naive since it's a pretty drastic change
in behavior not behind any kind of switch.
>What do other app servers do?
I did a little survey. ASCII-8BIT is kind of the de facto standard
even if it's not mandated by the Rack specification. Phusion
Passenger, Thin and WEBrick all send mostly ASCII-8BIT strings in
the env.
>My main concern is having more different behavior between various Rack
>servers servers, making it harder to switch between them.
Very valid; Rack wouldn't be much of a standard if there were a bunch
of variants in use.
>Another concern is breaking apps which are already working around this
>but work with non-UTF-8 encodings.
We'd pretty much need to introduce some kind of configuration
switch, at least for the short term and maybe for the long term.
The hope would be that it could become the default setting.
Apps that don't use UTF8 should be able to set their desired default
external encoding appropriately.
>The rack-devel mailing list had a discussion on this in September 2010
>and a decision was never reached. You can search the archives at:
>http://groups.google.com/group/rack-devel
I came across this thread but didn't realize that was the last word
so far when it came to Rack and encodings.
This might be one of those instances where it would be helpful for
implementation to lead specification. Unicorn is one of the leading
servers of its genre, if not the leader. If you supported a switch
that made the encoding regime more sane, I think other popular servers
like Thin and Passenger would swiftly follow and it might re-energize
the discussion about getting encodings into the Rack spec once and
for all, and give a base for experimentation and iteration for
getting the encodings in the spec right.
There's a lot of developer pain here. Many apps probably are serving
up encoding-related 500 errors without knowing it. There are
stories of developers adding "# encoding" everywhere, setting
the external/internal encoding, and then "things are fine until it
blows up somewhere else." I heard recently that a very large company
has stuck with Ruby 1.8.7, probably to avoid these encoding issues
among other things. It would be nice to improve the situation.
>Disclaimer: I am not an encoding expert, so for that reason I prefer
>to let other Rack folks make the decision.
I'm not an encoding expert either! Most people aren't... which is
why it'd be nice if they didn't have to know so much about it when
they write a Rack app!
>Do you have performance measurements for doing this as pure-Ruby
>middleware vs your patch?
I don't have measurements currently but I'll get some.
Our app is several years old and so there's a lot of stuff in
request.env by the time we get around to forcing everything to
UTF8 encoding. I wouldn't be surprised if the hit on
every single request is small but significant for us.
>So it should be best if there were a way to do this for all Rack
>servers.
Thanks again for reviewing the patch. I'll work on a new patch that
incorporates your comments and has a switch for enabling/disabling
the functionality, and I'll try to follow roughly what the spec
group in 2010 thought would make sense in terms of encodings for
the various strings in the env. And I'll see if I can ask the
Rack folks to chime in.
Gary
^ permalink raw reply [relevance 1%]
* Re: Please move to github
2014-08-02 19:07 1% ` Gary Grossman
@ 2014-08-02 20:15 0% ` Eric Wong
0 siblings, 0 replies; 200+ results
From: Eric Wong @ 2014-08-02 20:15 UTC (permalink / raw)
To: Gary Grossman; +Cc: unicorn-public, michael
Gary Grossman <gary.grossman@gmail.com> wrote:
> We'd pretty much need to introduce some kind of configuration
> switch, at least for the short term and maybe for the long term.
> The hope would be that it could become the default setting.
> Apps that don't use UTF8 should be able to set their desired default
> external encoding appropriately.
If possible, I would like to avoid an option and rely on
Encoding.default_external in a new major version. Too many ways to set
the same thing is confusing and requires extra documentation overhead.
> >The rack-devel mailing list had a discussion on this in September 2010
> >and a decision was never reached. You can search the archives at:
> >http://groups.google.com/group/rack-devel
>
> I came across this thread but didn't realize that was the last word
> so far when it came to Rack and encodings.
>
> This might be one of those instances where it would be helpful for
> implementation to lead specification. Unicorn is one of the leading
> servers of its genre, if not the leader. If you supported a switch
> that made the encoding regime more sane, I think other popular servers
> like Thin and Passenger would swiftly follow and it might re-energize
> the discussion about getting encodings into the Rack spec once and
> for all, and give a base for experimentation and iteration for
> getting the encodings in the spec right.
I might start with WEBrick (or the Rack/WEBrick handler). WEBrick is
distributed with Ruby and maintained by the core team. It's not used in
production much, but it the reference implementation which is usable
from all Ruby implementations.
naruse (from that rack-devel thread) is also active in Ruby core and
is very knowledgeable in these areas.
> Thanks again for reviewing the patch. I'll work on a new patch that
> incorporates your comments and has a switch for enabling/disabling
> the functionality, and I'll try to follow roughly what the spec
> group in 2010 thought would make sense in terms of encodings for
> the various strings in the env. And I'll see if I can ask the
> Rack folks to chime in.
Definitely get other Rack folks to chime in, even if it is a
unicorn-only change. This has been a problem for years already,
so taking more time to get things right won't hurt.
^ permalink raw reply [relevance 0%]
Results 1-200 of ~290 next (newer) | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2009-07-01 22:58 1% [ANN] unicorn 0.9.0 (experimental release) Eric Wong
2009-07-06 8:21 Unicorn Paul Sponagl
2009-07-06 23:16 ` Unicorn Eric Wong
2009-07-07 8:23 ` Unicorn Paul Sponagl
2009-07-07 20:48 2% ` Unicorn Eric Wong
[not found] ` <CE7F2AC4-4088-44ED-A5FF-2A41FE2B61EF@sponagl.de>
[not found] ` <20090708220413.GA32416@dcvr.yhbt.net>
2009-07-09 8:06 ` Unicorn Paul Sponagl
2009-07-09 10:10 2% ` Unicorn Eric Wong
[not found] <8b73aaca0909042020w73fb03dfpf6c77c85c1c486ad@mail.gmail.com>
2009-09-05 3:21 2% ` Pidfiles and cwd? Chris Wanstrath
2009-09-05 4:28 0% ` Eric Wong
2009-09-19 23:07 2% GitHub on Unicorn Chris Wanstrath
2009-09-20 0:00 0% ` Eric Wong
2009-10-02 23:54 2% 0.93.0 gem bug report: permissions snafu Jay Reitz
2009-10-08 0:31 Unicorn - "Still a couple bugs to knock out." Eric Wong
[not found] ` <b89b94f00910072029m5c7265ban2f98e4346265e534@mail.gmail.com>
2009-10-08 4:47 0% ` Eric Wong
2009-10-09 19:42 2% Our Unicorn Setup Chris Wanstrath
2009-10-09 20:30 2% ` Eric Wong
2009-10-09 21:25 0% ` Chris Wanstrath
2009-10-09 21:03 0% ` Dusty Doris
2009-10-13 16:57 Unicorn Nginx Issue Matt Mongeau
2009-10-13 17:20 ` Brian Ketelsen
2009-10-13 17:27 ` Matt Mongeau
2009-10-13 17:34 ` Matt Mongeau
2009-10-13 18:53 ` Eric Wong
2009-10-13 19:03 ` Matt Mongeau
2009-10-13 19:43 2% ` Eric Wong
2009-10-13 20:11 0% ` Matt Mongeau
[not found] <d779a0c60910230001r5f7ecea3l4904d48a08d2130f@mail.gmail.com>
2009-10-23 14:38 ` Truncated request bodies Vadim Spivak
2009-10-24 22:05 ` Vadim Spivak
2009-10-25 0:04 2% ` Eric Wong
2009-11-03 9:44 workers does not seems to exit when served one client HaiMing Yin
2009-11-03 16:56 ` Eric Wong
2009-11-04 5:33 2% ` HaiMing Yin
2009-11-19 7:49 RE:Nginx Conf huet bartels
2009-11-22 22:53 ` Nginx Conf Eric Wong
2009-11-23 3:57 ` Dylan Stamat
2009-11-24 7:19 2% ` Eric Wong
2009-11-26 4:50 feature request - when_ready() hook Suraj Kurapati
2009-11-26 6:05 ` Eric Wong
2009-11-26 19:05 ` Suraj Kurapati
2009-11-26 19:53 ` Eric Wong
2009-11-30 23:47 2% ` Suraj Kurapati
2009-12-07 10:01 2% [ANN] unicorn 0.95.2 - small HTTP parser fixes Eric Wong
2009-12-09 21:05 Expect 100-continue errors Jan Dvorak
2009-12-09 22:16 2% ` Eric Wong
2009-12-09 23:46 0% ` Jan Dvorak
2009-12-10 22:30 2% PATH_INFO spec (with regard to ";") Eric Wong
2009-12-23 2:20 3% "unicorn -D" always returns 0 "success" (even when failed to load) Iñaki Baz Castillo
2009-12-23 7:26 0% ` Eric Wong
2009-12-25 23:58 2% ` Iñaki Baz Castillo
2009-12-23 14:29 2% How to trap USR2 for custom ussage ? Iñaki Baz Castillo
2009-12-23 20:47 2% ` Eric Wong
2009-12-24 9:40 2% ` Iñaki Baz Castillo
2009-12-24 19:41 0% ` Eric Wong
2009-12-25 22:16 2% ` Iñaki Baz Castillo
2009-12-26 4:29 "unicorn -D" always returns 0 "success" (even when failed to load) Iñaki Baz Castillo
2009-12-26 6:16 ` Eric Wong
2009-12-26 15:47 1% ` Iñaki Baz Castillo
2009-12-26 18:23 ` Iñaki Baz Castillo
2009-12-27 1:31 ` Eric Wong
2009-12-27 3:06 4% ` Iñaki Baz Castillo
2009-12-28 3:29 0% ` Eric Wong
2009-12-28 10:39 0% ` Iñaki Baz Castillo
2010-01-02 16:42 Raised exceptions are not logged when USR2 Iñaki Baz Castillo
2010-01-02 21:27 ` Iñaki Baz Castillo
2010-01-02 21:33 2% ` Iñaki Baz Castillo
2010-01-03 0:04 0% ` Eric Wong
2010-01-03 16:22 0% ` Iñaki Baz Castillo
2010-01-02 18:29 2% unicorn/reexec/bundler/cap deploy issue skaar
2010-01-02 22:59 2% skaar
2010-01-03 0:25 0% ` Eric Wong
2010-01-03 17:08 2% ` skaar
2010-01-07 15:35 Using a worker for a different purpose Iñaki Baz Castillo
2010-01-07 20:26 ` Eric Wong
2010-01-07 20:44 3% ` Iñaki Baz Castillo
2010-01-07 20:53 0% ` Eric Wong
2010-01-07 20:58 0% ` Iñaki Baz Castillo
2010-01-07 16:05 1% Strange Thread related errors Michael Guterl
2010-01-07 20:13 0% ` Eric Wong
2010-01-07 21:32 0% ` Michael Guterl
2010-01-09 22:16 X-Forwarded-Proto / X_FORWARDED_PROTO skaar
2010-01-09 22:39 ` Eric Wong
2010-01-09 23:29 ` skaar
2010-01-09 23:45 ` Iñaki Baz Castillo
2010-01-10 6:12 ` Eric Wong
2010-01-10 6:32 ` Eric Wong
2010-01-10 11:43 2% ` John-Paul Bader
2010-01-11 22:27 1% Unicorn 0.96 doesn't play nice with Rails 2.3.5 Peter Kieltyka
2010-01-12 3:42 0% ` Eric Wong
2010-01-12 14:36 0% ` Peter Kieltyka
2010-01-19 21:50 0% ` Iñaki Baz Castillo
2010-01-20 8:54 2% Suggestion for reload action (USR2) Iñaki Baz Castillo
2010-02-04 8:16 "upstream timed out" after upgrades John-Paul Bader
2010-02-04 8:48 1% ` John-Paul Bader
2010-02-04 9:28 ` John-Paul Bader
2010-02-04 10:11 ` Eric Wong
2010-02-04 12:26 2% ` John-Paul Bader
2010-02-04 14:22 2% ` John-Paul Bader
2010-02-04 15:11 0% ` John-Paul Bader
2010-02-04 19:42 ` Eric Wong
2010-02-04 20:01 ` John-Paul Bader
2010-02-04 20:14 ` Eric Wong
2010-02-04 20:27 ` John-Paul Bader
2010-02-04 21:52 2% ` Eric Wong
2010-02-14 19:01 Nginx Sock And Rails Envinroment Error Alex Barlow
2010-02-15 7:30 ` Eric Wong
2010-02-15 10:38 ` Alex Barlow
2010-02-15 16:19 2% ` Eric Wong
2010-02-15 22:14 0% ` Alex Barlow
2010-03-18 21:12 Issue with unicorn not starting via bluepill? James Cox
2010-03-19 8:26 ` Eric Wong
2010-03-19 8:57 ` Eric Wong
2010-03-23 18:54 ` Eric Wong
2010-03-24 19:14 2% ` James Cox
2010-04-26 8:18 2% Shared memory between workers Iñaki Baz Castillo
2010-04-26 19:03 0% ` Eric Wong
2010-05-06 20:29 Garbage collection outside of request cycle? Luke Melia
2010-05-06 20:57 ` Eric Wong
2010-05-06 23:17 ` Luke Melia
2010-05-13 20:29 3% ` Luke Melia
2010-05-14 19:02 0% ` Eric Wong
2010-05-25 18:53 Forking off the unicorn master process to create a background worker Russell Branca
2010-05-26 21:05 ` Eric Wong
2010-06-15 17:55 2% ` Russell Branca
2010-06-15 22:14 0% ` Eric Wong
2010-06-15 22:51 0% ` Russell Branca
2010-06-16 0:06 0% ` Eric Wong
2010-06-03 17:37 Fwd: Support for Soft Timeout in Unicorn Eric Wong
2010-06-03 18:22 ` Eric Wong
2010-06-03 18:32 ` Pierre Baillet
2010-06-03 18:47 ` Eric Wong
2010-06-03 19:38 ` Chris Wanstrath
2010-06-03 19:40 ` Pierre Baillet
2010-06-09 13:17 ` Pierre Baillet
2010-06-11 1:56 2% ` Eric Wong
2010-06-04 1:58 1% unicorn_rails cleanup (possible fix for Rails3) pushed Eric Wong
2010-06-04 2:13 0% ` Michael Guterl
2010-06-04 2:48 0% ` Eric Wong
2010-06-08 9:51 [ANN] unicorn 0.990.0 - inching towards 1.0 Eric Wong
2010-06-09 8:39 ` Alexander Simonov
2010-06-09 18:22 ` Eric Wong
2010-06-10 5:26 ` Alexander Simonov
2010-06-10 7:38 2% ` Eric Wong
2010-06-10 8:46 0% ` Alexander Simonov
2010-06-11 1:27 0% ` Eric Wong
2010-06-11 18:00 0% ` Alexander Simonov
2010-06-08 12:24 unicorn_rails cleanup (possible fix for Rails3) pushed Hongli Lai
2010-06-08 19:20 ` Eric Wong
2010-06-08 19:25 ` Hongli Lai
2010-06-08 20:55 2% ` Eric Wong
2010-06-14 19:46 Unicorn future plans Eric Wong
2010-06-14 20:58 ` John-Paul Bader
2010-06-14 22:10 3% ` Eric Wong
2010-06-15 7:09 3% ` John-Paul Bader
2010-06-23 5:57 Purpose of "Status" header in HTTP responses? Craig Davey
2010-06-23 9:07 ` Eric Wong
2010-06-24 14:56 2% ` Purpose of "Status" " Craig Davey
2010-07-13 4:25 2% SIGWINCH Lawrence Pit
2010-07-13 4:34 0% ` SIGWINCH Eric Wong
2010-08-17 23:38 3% javascript disappears Jimmy Soho
2010-08-18 1:14 0% ` Eric Wong
2010-08-18 2:23 ` Jimmy Soho
2010-08-18 2:45 ` Eric Wong
2010-08-18 3:39 ` Jimmy Soho
2010-08-18 7:14 ` Eric Wong
2010-08-18 13:20 2% ` Jimmy Soho
2010-08-18 23:34 0% ` Eric Wong
2010-09-15 6:40 2% Unicorn fails to restart gracefully on capistrano deploy Eirik Dentz Sinclair
2010-09-15 21:52 0% ` Eric Wong
2010-10-02 0:57 2% Unicorn doesn't reload the app after the HUP signal Rubén Dávila
2010-10-02 8:13 0% ` Eric Wong
2010-10-02 16:38 Problem with binding UNIX listeners before checking PID Jordan Ritter
2010-10-04 4:17 ` Eric Wong
2010-10-04 4:22 4% ` [PATCH] avoid unlinking actively listening sockets Eric Wong
2010-11-20 2:47 [ANN] unicorn 3.0.0 - disable rewindable input! Eric Wong
2010-11-20 17:50 ` Michael Guterl
2010-11-21 4:05 ` Eric Wong
2010-11-21 16:29 ` Jeremy Evans
2010-11-21 23:49 ` Eric Wong
2010-11-22 1:21 2% ` Jeremy Evans
2010-12-01 11:59 Unicorn and HAProxy, 500 Internal errors after checks Pierre
2010-12-01 16:52 ` Eric Wong
2010-12-01 17:14 2% ` Pierre
2011-02-16 19:11 2% license is NOT changing Eric Wong
2011-03-08 4:40 [RFC/PATCH] Bundler/Sandbox documentation updates Eric Wong
2011-03-09 10:39 2% ` Lawrence Pit
2011-03-10 0:52 0% ` Eric Wong
2011-04-07 21:11 1% Unicorn error Ernesto Rocha
2011-04-07 22:04 0% ` Clifton King
2011-05-21 3:05 leading/trailing linear whitespace in headers Eric Wong
2011-05-21 5:14 ` Lawrence Pit
2011-05-21 5:57 2% ` Eric Wong
2011-05-27 7:32 0% ` Lawrence Pit
2011-05-30 7:02 2% ` Lawrence Pit
2011-05-31 4:49 0% ` Eric Wong
2011-05-31 9:02 workers not utilizing multiple CPUs Nate Clark
2011-05-31 12:10 ` Lawrence Pit
2011-05-31 12:20 ` Nate Clark
2011-05-31 14:07 ` Clifton King
2011-05-31 15:48 ` Eric Wong
2011-05-31 15:55 ` Clifton King
[not found] ` <BANLkTikC5D+0pUDRDgvQbzu=dpwmdKNY=A@mail.gmail.com>
2011-06-01 6:51 2% ` Nate Clark
2011-06-20 23:40 problem setting multiple cookies Jason Su
2011-06-21 0:06 ` Eric Wong
2011-06-21 0:25 2% ` Jason Su
2011-06-21 0:46 ` Eric Wong
2011-06-21 1:34 ` Jason Su
2011-06-21 1:35 2% ` Jason Su
2011-06-21 2:15 ` Eric Wong
2011-06-21 2:48 2% ` Jason Su
2011-06-21 3:15 0% ` Eric Wong
2011-06-21 3:38 2% ` Jason Su
2011-06-21 18:29 0% ` Eric Wong
2011-06-22 1:01 ` Jason Su
2011-06-22 1:59 ` Eric Wong
2011-06-22 4:18 1% ` Jason Su
2011-06-22 6:02 0% ` Eric Wong
2011-07-11 16:07 2% Unicorn vs Apache Matt Smith
2011-07-11 18:45 0% ` Eric Wong
2011-08-02 20:09 2% Strange quit behavior James Cox
2011-08-02 20:34 0% ` Alex Sharp
2011-08-02 21:54 0% ` Eric Wong
2011-08-05 4:09 ` Alex Sharp
2011-08-05 4:12 ` Alex Sharp
2011-08-05 8:07 ` Eric Wong
2011-08-17 4:26 1% ` Alex Sharp
2011-08-17 9:22 ` Eric Wong
2011-08-17 20:13 ` Eric Wong
2011-08-18 23:13 2% ` Alex Sharp
2011-08-19 1:53 ` Eric Wong
2011-08-19 9:42 ` Alex Sharp
2011-08-23 2:59 ` Alex Sharp
2011-08-23 7:12 ` Eric Wong
2011-08-23 16:49 2% ` Alex Sharp
2011-08-02 21:53 0% ` Eric Wong
2011-08-02 22:46 0% ` James Cox
2011-08-03 16:41 2% Unicorn in the jail Tatsuya Ono
2011-08-28 13:35 Unicorn for Rails development mode? Jay Levitt
2011-08-28 23:14 2% ` Eric Wong
2011-09-08 9:04 1% Sending ABRT to timeout-errant process before KILL J. Austin Hughey
2011-10-04 7:10 Peformance up - using OobGC & GC.disable secondlife
2011-10-04 22:53 ` Eric Wong
2011-10-06 6:22 ` secondlife
2011-10-10 17:05 ` Tatsuya Ono
2011-10-10 21:53 ` Eric Wong
2011-10-10 23:03 ` Tatsuya Ono
2011-10-12 16:00 2% ` Tatsuya Ono
2011-11-21 23:14 Should USR2 always work? Laurens Pit
2011-11-22 1:16 ` Eric Wong
2011-11-22 1:55 ` Lawrence Pit
2011-11-22 3:00 ` Eric Wong
2011-11-23 5:31 3% ` Lawrence Pit
2011-11-30 15:24 2% COW friendly GC and PostgreSQL Laas Toom
2011-11-30 20:54 0% ` Jeremy Evans
2011-12-08 18:19 Master repeatedly killing workers due to timeouts Jonathan del Strother
2011-12-08 20:23 ` Eric Wong
2011-12-08 22:24 2% ` Jonathan del Strother
2012-01-31 18:39 FreeBSD jail and unicorn Eric Wong
2012-01-31 18:50 2% ` Charles Hornberger
2012-01-31 19:05 Eric Wong
[not found] ` <CB4EBD2A.7DF%philipp.bruell@skrill.com>
2012-02-01 18:14 0% ` Eric Wong
[not found] ` <CB5014D7.892%philipp.bruell@skrill.com>
2012-02-02 19:31 0% ` Eric Wong
2012-04-12 17:36 2% Background jobs with #fork paddor
2012-04-12 20:39 0% ` Eric Wong
2012-04-12 22:41 ` Patrik Wenger
2012-04-12 23:04 3% ` Eric Wong
2012-04-27 14:36 app error: Socket is not connected (Errno::ENOTCONN) Joel Nimety
2012-04-27 18:59 ` Eric Wong
2012-04-27 19:12 ` Joel Nimety
2012-04-27 19:32 ` Eric Wong
2012-04-27 19:45 ` Matt Smith
2012-04-27 19:47 2% ` Matt Smith
2012-04-28 2:46 2% unicorn + sleep = long wake up loading time adam k
2012-04-28 4:40 0% ` Eric Wong
2012-06-25 13:02 Address already in use Manuel Palenciano Guerrero
2012-06-25 20:28 ` Eric Wong
2012-06-25 21:03 1% ` Manuel Palenciano Guerrero
2012-07-17 0:33 Any signal other than -9 causes full CPU utilization by master unicorn process on FreeBSD Mark Mccraw
2012-07-17 2:05 ` Eric Wong
2012-07-17 11:56 ` Mark Mccraw
2012-07-17 21:23 2% ` Mark Mccraw
2012-07-17 22:17 0% ` Eric Wong
2012-08-07 7:26 1% SIGUSR2 from symlinked working directory Danial Pearce
[not found] <mailman.0.1345509654.31187.mongrel-unicorn@rubyforge.org>
2012-08-21 0:44 ` Unused Unicorn processes Konstantin Gredeskoul
2012-08-21 9:11 ` Eric Wong
2012-08-22 18:16 2% ` Konstantin Gredeskoul
2012-10-09 0:39 Is a client uploading a file a slow client from unicorn's point of view? Jimmy Soho
2012-10-09 1:58 ` Eric Wong
2012-10-09 6:31 ` Laas Toom
2012-10-09 20:03 ` Eric Wong
2012-10-09 23:06 ` Laas Toom
2012-10-09 23:54 2% ` Eric Wong
2012-10-10 6:59 2% ` Laas Toom
2012-10-29 17:44 Combating nginx 499 HTTP responses during flash traffic scenario Tom Burns
2012-10-29 18:45 ` Eric Wong
2012-10-29 21:53 ` Eric Wong
2012-10-30 20:40 ` Tom Burns
2012-10-30 21:37 2% ` Eric Wong
2012-11-02 17:59 ` Tom Burns
2012-11-02 19:38 ` Eric Wong
2012-11-03 22:45 ` Tom Burns
2012-11-05 11:48 ` Eric Wong
2012-11-06 3:16 ` Tom Burns
2012-11-06 21:23 ` Eric Wong
2012-11-29 15:52 1% ` Tom Burns
2012-11-29 20:41 0% ` Eric Wong
2012-11-07 16:42 2% select(): Interrupted system call from curb when stopping unicorn Graham Bleach
2012-11-07 21:11 0% ` Eric Wong
2012-11-13 0:34 2% Testing Unicorn on Rubinius mike
2012-11-28 13:32 When a client terminates a connection Andrew Stewart
2012-11-29 17:09 ` Iñaki Baz Castillo
2012-11-29 23:04 ` Eric Wong
2012-11-30 9:15 2% ` Andrew Stewart
2012-11-30 19:26 0% ` Eric Wong
2012-12-03 14:42 0% ` Andrew Stewart
2012-11-30 17:20 2% using unicorn with logging-rails gem Yoav Aner
2012-11-30 19:15 0% ` Eric Wong
2012-11-30 19:59 0% ` Yoav Aner
[not found] ` <CA+mA6PgTGcyLy2Dua9a6gDWhRwsQLtHEyk17fhBE8DW1TkO5EA@mail.gmail.com>
2012-11-30 20:05 ` Eric Wong
2012-11-30 20:17 3% ` Yoav Aner
2012-11-30 20:55 0% ` Eric Wong
[not found] <CA+c-+BYTxv+5mQ2MKeKVYEoxgPbi2QjwY6Oak-R_g65GhixEzg@mail.gmail.com>
2012-12-05 5:19 ` Fwd: Issue starting unicorn with non-ActiveRecord Rails app Peter Hall
2012-12-05 7:48 ` Daniel Condomitti
2012-12-05 9:56 2% ` Peter Hall
2012-12-06 19:48 Unicorn fails to install even though it's already installed and running Mac Martine
2012-12-06 20:23 ` Eric Wong
2012-12-06 21:11 ` Mac Martine
2012-12-06 21:54 2% ` Eric Wong
2013-01-14 15:42 Increassing timeouts Manuel Palenciano Guerrero
2013-01-15 19:44 ` Eric Wong
2013-07-09 15:15 ` Troex Nevelin
2013-08-03 6:32 2% ` Eric Wong
2013-01-24 8:42 SIGSEGV at shutdown (was: Re: your mail) Charles Hornberger
2013-01-24 9:52 ` Eric Wong
2013-01-24 10:27 2% ` Charles Hornberger
2013-01-25 10:52 No middleware without touching RACK_ENV Lin Jen-Shin (godfat)
2013-01-25 18:39 ` Eric Wong
2013-01-28 14:43 1% ` Lin Jen-Shin (godfat)
2013-01-28 23:21 2% ` Eric Wong
2013-01-29 2:31 0% ` Lin Jen-Shin (godfat)
2013-03-09 16:02 Unicorn hangs on POST request Tom Pesman
2013-03-09 21:02 ` Eric Wong
2013-03-10 16:22 ` Tom Pesman
2013-03-11 19:49 ` Eric Wong
2013-03-11 22:20 ` Tom Pesman
2013-03-11 23:01 ` Eric Wong
2013-04-02 11:55 ` Tom Pesman
2013-04-02 17:24 ` Eric Wong
2013-04-02 20:25 ` Tom Pesman
2013-04-02 22:36 ` Eric Wong
2013-04-03 4:56 2% ` Lin Jen-Shin (godfat)
2013-04-03 11:38 0% ` Eric Wong
2013-04-25 8:02 Why doesn't SIGTERM quit gracefully? Andreas Falk
2013-04-25 8:51 ` Eric Wong
2013-04-25 11:02 ` Andreas Falk
2013-04-25 18:09 2% ` Eric Wong
2013-04-25 20:45 0% ` Andreas Falk
2013-08-20 14:47 2% A barrage of unexplained timeouts nick
2013-08-20 16:37 ` Eric Wong
2013-08-20 17:27 2% ` nick
2013-08-20 17:40 ` Eric Wong
2013-08-20 18:11 2% ` nick
2013-08-20 18:49 2% ` Eric Wong
2013-08-20 20:03 2% ` nick
2013-08-20 20:42 0% ` Eric Wong
2013-08-20 21:19 0% ` nick
2013-09-29 20:13 2% More " nick
2013-10-23 22:55 2% pid file handling issue Michael Fischer
2013-10-24 0:53 0% ` Eric Wong
2013-11-01 14:12 [PATCH] construct listener_fds Hash in 1.8 compatible way Ernest W. Durbin III
2013-11-01 16:50 ` Eric Wong
2013-11-01 17:04 ` Ernest W. Durbin III
2013-11-01 18:54 ` Eric Wong
2013-11-01 19:00 2% ` Ernest W. Durbin III
2013-11-05 14:46 Handling closed clients Andrew Hobson
2013-11-05 17:20 ` Eric Wong
[not found] ` <m2iow6k7nk.fsf@macdaddy.atl.damballa>
2013-11-05 20:51 ` Eric Wong
[not found] ` <m21u2sjpc9.fsf@macdaddy.atl.damballa>
2013-11-07 16:48 0% ` Eric Wong
2013-11-07 20:22 2% ` [PATCH] stream_input: avoid IO#close on client disconnect Eric Wong
2013-11-26 13:30 What does it mean for the unicorn process to be bound to a terminal? Rodrigo Rosenfeld Rosas
2013-11-26 18:04 ` Eric Wong
2013-11-26 18:35 ` Rodrigo Rosenfeld Rosas
2013-11-26 19:21 2% ` Eric Wong
2013-12-09 9:52 1% [PATCH] rework master-to-worker signaling to use a pipe Eric Wong
2013-12-10 0:22 0% ` Sam Saffron
2014-01-11 7:40 2% [ANN] unicorn 4.8.0 - big internal changes, but compatible Eric Wong
2014-01-28 19:49 Different behavior with pid files and SIGUSR2 Michael Graff
2014-01-28 20:03 2% ` Eric Wong
2014-01-28 20:08 0% ` Michael Graff
2014-02-24 12:33 2% Stucks DB connections Ilya Bazylchuk
2014-02-24 18:25 0% ` Eric Wong
2014-04-26 3:20 [PATCH] tests: switch to minitest Ken Dreyer
2014-04-26 6:07 2% ` Eric Wong
2014-05-07 8:05 [ANN] unicorn 4.8.3 - the end of an era Eric Wong
2014-05-07 9:30 ` Jérémy Lecour
2014-05-07 9:46 ` Eric Wong
2014-05-07 20:33 ` Michael Fischer
2014-05-07 21:25 3% ` Eric Wong
2014-07-03 14:40 Log reopening broken on Rails 4 with config.autoflush_log = false? Cedric Maion
2014-07-03 17:24 ` Eric Wong
2014-07-04 9:13 2% ` Cedric Maion
2014-07-24 16:27 Old unicorn workers still receives requests? Bráulio Bhavamitra
2014-07-24 17:04 2% ` Eric Wong
2014-07-24 17:22 0% ` Bráulio Bhavamitra
2014-08-02 7:51 Please move to github Gary Grossman
2014-08-02 8:50 ` Eric Wong
2014-08-02 19:07 1% ` Gary Grossman
2014-08-02 20:15 0% ` Eric Wong
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).