Re: pid file deleted briefly when doing hot restart

[Eric Wong <normalperson@yhbt.net>]

Petteri Räty <betelgeuse@gentoo.org> wrote:
> On 26.11.2012 20.24, Eric Wong wrote:
> > 
> >> The use case here is that with health monitors wouldn't have a window
> >> where a pid file does not exist. With a hot restart it should always be
> >> possible to have a pid file that points to either the old or the new master.
> > 
> > Then, doesn't nginx have the same problem?
> > 
> 
> nginx doens't have the same problem as I show later. Even if it I
> restart/reload nginx very infrequently. Unicorn has to be hot restarted
> every time we change our application code and happens frequently.
> 
> >>> I think unicorn differs a bit from nginx here:
> >>>
> >>> nginx uses rename() to clear the way for a new pid file.  Like unicorn,
> >>> this still leaves a window where no pid file exists.
> >>>
> >>
> >> Looking at the inotify log it seems the reason pid file does not exist
> >> is an explicit delete and not due to rename. It happens a couple seconds
> >> earlier also so the window is possible to hit even with a periodic poller.
> > 
> > Is matching nginx rename behavior enough to solve the problem?
> > 
> > Matching nginx behavior can become the default if it solves your problem.
> > 
> 
> nginx does not explicitly unlink the old pid file before it renames it
> out of the way so yes matching nginx in that regard changes the behavior
> exactly how I originally asked but you were against that. Maybe the
> point is moot though.

Ah, I thought you wanted the pid file to be replaced without
a window where the file is non-existent (or empty).

> This is from a combo of USR2, WINCH, QUIT sent from htop to the master
> process:
> 
> 2012-11-27 01:44:25 +0200
> [:moved_from, :move]
> "nginx.pid"
> 2012-11-27 01:44:25 +0200
> [:moved_to, :move]
> "nginx.pid.oldbin"
> 2012-11-27 01:44:25 +0200

OK, this is the window where file does not exist at all.

unicorn has the same problem here, but obviously unicorn is slower than
nginx.

> [:create]
> "nginx.pid"
> 2012-11-27 01:44:25 +0200
> [:open]
> "nginx.pid"

This part of nginx makes me uncomfortable since nginx.pid is empty.

> 2012-11-27 01:44:25 +0200
> [:modify]
> "nginx.pid"
> 2012-11-27 01:44:25 +0200
> [:close_write, :close]
> "nginx.pid"
> 2012-11-27 01:45:31 +0200
> [:delete]
> "nginx.pid.oldbin"
> 
> The window here is much smaller than for the current unicorn behavior.

A small window is still a window.

Can't you make your health monitor check the state of the listening
ports as well?  There's no point where a listening port will be
unavailable.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying