unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
* Unicorn / Daemontools
@ 2011-04-25 16:37 Curtis j Schofield
  2011-04-25 18:34 ` Eric Wong
  2011-04-25 22:07 ` snacktime
  0 siblings, 2 replies; 7+ messages in thread
From: Curtis j Schofield @ 2011-04-25 16:37 UTC (permalink / raw)
  To: mongrel-unicorn

Hi  -

I tried to get Daemontools and Unicorn working together a while back -
there are issues on USR2 restart because of the pid
change - I'm hoping someone  in the community will have some
understanding of this issue

I documented my experience and eventual defeat here :
http://log.robotarmyma.de/post/2053448029/daemontools-ubuntu-rvm-bundler-unicorn-install


Any help would be received graciously.

-- 
make haste slowly \
festina lente  \
-
mobile  +1_415_632_6001
curtis.schofield@gmail.com
http://robotarmyma.de
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Unicorn / Daemontools
  2011-04-25 16:37 Unicorn / Daemontools Curtis j Schofield
@ 2011-04-25 18:34 ` Eric Wong
  2011-04-25 19:03   ` Curtis j Schofield
  2011-04-25 22:07 ` snacktime
  1 sibling, 1 reply; 7+ messages in thread
From: Eric Wong @ 2011-04-25 18:34 UTC (permalink / raw)
  To: unicorn list

Curtis j Schofield <curtis.schofield@gmail.com> wrote:
> I tried to get Daemontools and Unicorn working together a while back -
> there are issues on USR2 restart because of the pid
> change - I'm hoping someone  in the community will have some
> understanding of this issue

I have no experience with daemontools directly, but I expect any
solutions people found to make daemontools work with nginx USR2
would also work for unicorn since they have the same process/signal
model.

Maybe a hacky way would be to have daemontools run this wrapper
script instead of Unicorn:

-------------------------------- 8< -------------------------
#!/bin/sh
set -e

# this script is totally untested and written by someone who's never
# used daemontools

# start Unicorn in the background:
bundle exec unicorn -D ...

# forward signals received in this wrapper to the Unicorn process:
for sig in HUP USR1 USR2 QUIT TERM QUIT
do
	trap 'kill -'$sig' $(cat $UNICORN_PID_FILE)' $sig
done

# loop forever while Unicorn has its pid file
# a smarter, Linux-only version of this would use inotify instead
while test -f $UNICORN_PID_FILE
do
	sleep 1
done

-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Unicorn / Daemontools
  2011-04-25 18:34 ` Eric Wong
@ 2011-04-25 19:03   ` Curtis j Schofield
  2011-04-25 21:01     ` Eric Wong
  0 siblings, 1 reply; 7+ messages in thread
From: Curtis j Schofield @ 2011-04-25 19:03 UTC (permalink / raw)
  To: unicorn list

On Mon, Apr 25, 2011 at 11:34 AM, Eric Wong <normalperson@yhbt.net> wrote:
> Curtis j Schofield <curtis.schofield@gmail.com> wrote:
>> I tried to get Daemontools and Unicorn working together a while back -
>> there are issues on USR2 restart because of the pid
>> change - I'm hoping someone  in the community will have some
>> understanding of this issue
>
> I have no experience with daemontools directly, but I expect any
> solutions people found to make daemontools work with nginx USR2
> would also work for unicorn since they have the same process/signal
> model.
>
> Maybe a hacky way would be to have daemontools run this wrapper
> script instead of Unicorn:
>
> -------------------------------- 8< -------------------------
> #!/bin/sh
> set -e
>
> # this script is totally untested and written by someone who's never
> # used daemontools
>
> # start Unicorn in the background:
> bundle exec unicorn -D ...
>
> # forward signals received in this wrapper to the Unicorn process:
> for sig in HUP USR1 USR2 QUIT TERM QUIT
> do
>        trap 'kill -'$sig' $(cat $UNICORN_PID_FILE)' $sig
> done
>
> # loop forever while Unicorn has its pid file
> # a smarter, Linux-only version of this would use inotify instead
> while test -f $UNICORN_PID_FILE
> do
>        sleep 1
> done
>


 Eric! This is fascinating!


 If i run a wrapper with unicorn as a child process - i can detect the
absence of the wrapper - and
i can detect the absence of the pid file - but i cannot detect a hard
fail in the unicorn process unless i have some kind of pipe
open to the child and register that failure.

Is my understanding correct?

Daemontools is design to take a process that is running in the
foreground and monitor it - keep it alive - the breakdown in unicorn
is when a USR2 arrives at unicorn - and the pid switch occurs - daemon
tools believes it has lost the process.

I see this as a design flaw in all pid based monitoring solutions - in
my experiences.



> --
> Eric Wong
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>



-- 
make haste slowly \
festina lente  \
-
mobile  +1_415_632_6001
curtis.schofield@gmail.com
http://robotarmyma.de
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Unicorn / Daemontools
  2011-04-25 19:03   ` Curtis j Schofield
@ 2011-04-25 21:01     ` Eric Wong
  2011-04-25 21:58       ` Curtis j Schofield
  0 siblings, 1 reply; 7+ messages in thread
From: Eric Wong @ 2011-04-25 21:01 UTC (permalink / raw)
  To: unicorn list

Curtis j Schofield <curtis.schofield@gmail.com> wrote:
> On Mon, Apr 25, 2011 at 11:34 AM, Eric Wong <normalperson@yhbt.net> wrote:

<snip>

> > while test -f $UNICORN_PID_FILE
> > do
> >        sleep 1
> > done
> 
>  If i run a wrapper with unicorn as a child process - i can detect the
> absence of the wrapper - and
> i can detect the absence of the pid file - but i cannot detect a hard
> fail in the unicorn process unless i have some kind of pipe
> open to the child and register that failure.
> 
> Is my understanding correct?

Yes, but the absence of the pid file usually means the unicorn process
failed hard.  Only unicorn error logs can tell you that definitively,
though.  I would probably make the wrapper script "exit 1" after the
while loop in my script...

> Daemontools is design to take a process that is running in the
> foreground and monitor it - keep it alive - the breakdown in unicorn
> is when a USR2 arrives at unicorn - and the pid switch occurs - daemon
> tools believes it has lost the process.
> 
> I see this as a design flaw in all pid based monitoring solutions - in
> my experiences.

Yes, it's a problem with the monitoring solution trying to do the job of
the unicorn master process.  Unicorn is designed to use the master
process and there's no other way to do what Unicorn does without it.

The unicorn master process itself should be very robust and never fail
during normal operation (upgrades may break it if things go really
wrong, but you already pay attention to what you're upgrading, right?).

-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Unicorn / Daemontools
  2011-04-25 21:01     ` Eric Wong
@ 2011-04-25 21:58       ` Curtis j Schofield
  0 siblings, 0 replies; 7+ messages in thread
From: Curtis j Schofield @ 2011-04-25 21:58 UTC (permalink / raw)
  To: unicorn list

>
> Yes, it's a problem with the monitoring solution trying to do the job of
> the unicorn master process.  Unicorn is designed to use the master
> process and there's no other way to do what Unicorn does without it.
>
> The unicorn master process itself should be very robust and never fail
> during normal operation (upgrades may break it if things go really
> wrong, but you already pay attention to what you're upgrading, right?).
>


I'm not concerned about the unicorn process failing - more i use daemon tools on
boot and to make sure the process comes alive properly at boot and
because daemon tools
has a simple and intelligent interface that doesn't have me writing
blue-pill conf files or monit
conf files.

That said - it may be enough to monitor that wrapper script in this case.



-- 
make haste slowly \
festina lente  \
-
mobile  +1_415_632_6001
curtis.schofield@gmail.com
http://robotarmyma.de
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Unicorn / Daemontools
  2011-04-25 16:37 Unicorn / Daemontools Curtis j Schofield
  2011-04-25 18:34 ` Eric Wong
@ 2011-04-25 22:07 ` snacktime
  2011-04-25 23:18   ` Curtis j Schofield
  1 sibling, 1 reply; 7+ messages in thread
From: snacktime @ 2011-04-25 22:07 UTC (permalink / raw)
  To: unicorn list

There are USR2 patches for daemontools, we used them for unicorn on a
large production app worked just fine.  Can't remember the url but
google should show it.

Chris Ochs

On Mon, Apr 25, 2011 at 9:37 AM, Curtis j Schofield
<curtis.schofield@gmail.com> wrote:
> Hi  -
>
> I tried to get Daemontools and Unicorn working together a while back -
> there are issues on USR2 restart because of the pid
> change - I'm hoping someone  in the community will have some
> understanding of this issue
>
> I documented my experience and eventual defeat here :
> http://log.robotarmyma.de/post/2053448029/daemontools-ubuntu-rvm-bundler-unicorn-install
>
>
> Any help would be received graciously.
>
> --
> make haste slowly \
> festina lente  \
> -
> mobile  +1_415_632_6001
> curtis.schofield@gmail.com
> http://robotarmyma.de
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Unicorn / Daemontools
  2011-04-25 22:07 ` snacktime
@ 2011-04-25 23:18   ` Curtis j Schofield
  0 siblings, 0 replies; 7+ messages in thread
From: Curtis j Schofield @ 2011-04-25 23:18 UTC (permalink / raw)
  To: unicorn list

I read one possible patch - it doesn't do pid handling - just lets svc
send different signals

My understanding of how unicorn responds to USR2 is that it changes it's pid and
respawns - this would mean that Daemontools is now monitoring the
wrong pid and will
think the process went down.



On Mon, Apr 25, 2011 at 3:07 PM, snacktime <snacktime@gmail.com> wrote:
> There are USR2 patches for daemontools, we used them for unicorn on a
> large production app worked just fine.  Can't remember the url but
> google should show it.
>
> Chris Ochs
>
> On Mon, Apr 25, 2011 at 9:37 AM, Curtis j Schofield
> <curtis.schofield@gmail.com> wrote:
>> Hi  -
>>
>> I tried to get Daemontools and Unicorn working together a while back -
>> there are issues on USR2 restart because of the pid
>> change - I'm hoping someone  in the community will have some
>> understanding of this issue
>>
>> I documented my experience and eventual defeat here :
>> http://log.robotarmyma.de/post/2053448029/daemontools-ubuntu-rvm-bundler-unicorn-install
>>
>>
>> Any help would be received graciously.
>>
>> --
>> make haste slowly \
>> festina lente  \
>> -
>> mobile  +1_415_632_6001
>> curtis.schofield@gmail.com
>> http://robotarmyma.de
>> _______________________________________________
>> Unicorn mailing list - mongrel-unicorn@rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>> Do not quote signatures (like this one) or top post when replying
>>
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>



-- 
make haste slowly \
festina lente  \
-
mobile  +1_415_632_6001
curtis.schofield@gmail.com
http://robotarmyma.de
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2011-04-25 23:32 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-04-25 16:37 Unicorn / Daemontools Curtis j Schofield
2011-04-25 18:34 ` Eric Wong
2011-04-25 19:03   ` Curtis j Schofield
2011-04-25 21:01     ` Eric Wong
2011-04-25 21:58       ` Curtis j Schofield
2011-04-25 22:07 ` snacktime
2011-04-25 23:18   ` Curtis j Schofield

Code repositories for project(s) associated with this public inbox

	https://yhbt.net/unicorn.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).