zero downtime deploys

zero downtime deploys

[Suraj Kurapati]

Hello,

I tried using preload_app with the "zero downtime deploys" trick of
sending SIGQUIT to the old master in before_fork(), as described in:

  http://unicorn.bogomips.org/Unicorn/Configurator.html
  http://github.com/blog/517-unicorn

This significantly reduced, but did not eliminate, the transition time
when the new workers take over (step #2 below):

1. old master (and its workers) is killed in before_fork()
2. workers re-establish DB connections in after_fork()
3. workers are ready to work, at the bottom of after_fork()

Why do we kill the old master in before_fork() when the new workers
are really ready to work much later?  Shouldn't we kill the old master
at the *bottom* of after_fork() --- when the new workers are really
ready to work?

Thanks for your consideration.

Re: zero downtime deploys

[Eric Wong]

Suraj Kurapati <sunaku@gmail.com> wrote:
> Hello,
> 
> I tried using preload_app with the "zero downtime deploys" trick of
> sending SIGQUIT to the old master in before_fork(), as described in:
> 
>   http://unicorn.bogomips.org/Unicorn/Configurator.html
>   http://github.com/blog/517-unicorn
> 
> This significantly reduced, but did not eliminate, the transition time
> when the new workers take over (step #2 below):
> 
> 1. old master (and its workers) is killed in before_fork()
> 2. workers re-establish DB connections in after_fork()
> 3. workers are ready to work, at the bottom of after_fork()
> 
> Why do we kill the old master in before_fork() when the new workers
> are really ready to work much later?  Shouldn't we kill the old master
> at the *bottom* of after_fork() --- when the new workers are really
> ready to work?

Hi Suraj,

It depends on your setup, mainly memory constraints.  I know some
deployments run very close to the memory limit of the box and those
examples are geared for that.  We should clarify that in the
documentation.  I also know some setups that always startup with
worker_processes=1 and then SIGTTIN through an external script
gradually.

Obviously I would always try to avoid maintenance during peak traffic
because performance inevitably suffers somewhat (and human error is
always possible :)

-- 
Eric Wong

Re: zero downtime deploys

[Suraj Kurapati]

On Fri, Nov 13, 2009 at 3:12 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Suraj Kurapati <sunaku@gmail.com> wrote:
>> Why do we kill the old master in before_fork() when the new workers
>> are really ready to work much later?
>
> It depends on your setup, mainly memory constraints.  I know some
> deployments run very close to the memory limit of the box and those
> examples are geared for that.

Ah, that makes more sense.  Thanks.
_______________________________________________
mongrel-unicorn mailing list
mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn