Re: pid file deleted briefly when doing hot restart

[Eric Wong <normalperson@yhbt.net>]

Petteri Räty <betelgeuse@gentoo.org> wrote:
> On 27.11.2012 4.02, Eric Wong wrote:
> > Petteri Räty <betelgeuse@gentoo.org> wrote:
> >> On 27.11.2012 2.35, Eric Wong wrote:
> >>>>
> >>>> nginx does not explicitly unlink the old pid file before it renames it
> >>>> out of the way so yes matching nginx in that regard changes the behavior
> >>>> exactly how I originally asked but you were against that. Maybe the
> >>>> point is moot though.
> >>>
> >>> Ah, I thought you wanted the pid file to be replaced without
> >>> a window where the file is non-existent (or empty).
> >>
> >> That would be ideal but I meant this thread to be explicitly about the
> >> unlink and resulting couple seconds window. Now that I have spend some
> >> thinking about the issue here's an approach using hard links that can be
> >> used to replace the pid without window of non-existance:
> > 
> > Yes, this is ideal, but unfortunately, the window of non-existence is
> > necessary for compatibility with some existing (nginx-based) scripts.
> 
> Can you point out what those scripts are?

(Revisiting old archives ...)

OK, I misremembered the situation...

TL;DR: there's too much historical baggage, as unicorn inherits traits
  from both mongrel and nginx.  A change to fix one pid file monitoring
  solution will likely break something else.  Of course PID files
  are broken _anyways_, so relying on them for monitoring is a
  terrible idea, *especially* when it's something like nginx/unicorn
  where a master process gets replaced.

1) Before unicorn, there was Mongrel: it was important for somebody to
   create the PID file early in the process, before binding the listen
   socket.  I seem to recall this being done to make some other health
   monitor work.

2) nginx prevents PID file clobbering by attempting to bind the listen
   socket before writing the pid file.  This is the best way to prevent
   PID clobbering for a fresh process.

Unfortunately, unicorn needed to inherit trait (1) from Mongrel to
ease migration for Mongrel users.

However, I also decided PID file clobber prevention in nginx was useful,
so unicorn will not overwrite a PID file of a valid process.
Unfortunately, the unicorn implementation of PID file clobber prevention
is racy as it cannot copy nginx behavior here while preserving Mongrel
compatibility.

Thus, unicorn unlinks the PID file before writing a new one.
unicorn could rename the PID file out of the way before writing
a new one (as nginx does) and not suffer bad consequences, but
the file still disappears momentarily.

There could be a minor tweak to allow clobbering of an existing
PID file iff it matches an expected value (of the original process),
but I don't think its worth the effort to implement at this point;
as it will still be racy.

Really, PID files are horrible, and MORE horrible than usual
because we support process upgrade/backout via fork+exec().
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying