'before_fork' is executed in parallel by two processes ¿?

'before_fork' is executed in parallel by two processes ¿?

[Iñaki Baz Castillo]

Hi, by adding `echo $$ >> /tmp/kk` into 'before_fork' I've realized that two 
pids are printed to the file when starting Unicorn.
This means that 'before_fork' is being runned by two different processes in 
parallel. Is it the expected behaviour? perhaps something very wrong in my 
configuration?

Thanks.

-- 
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: 'before_fork' is executed in parallel by two processes ¿?

[Iñaki Baz Castillo]

El Martes, 5 de Enero de 2010, Iñaki Baz Castillo escribió:
> Hi, by adding `echo $$ >> /tmp/kk` into 'before_fork' I've realized that
>  two pids are printed to the file when starting Unicorn.
> This means that 'before_fork' is being runned by two different processes in
> parallel. Is it the expected behaviour? perhaps something very wrong in my
> configuration?

I've totally lost. before_fork is executed in parallel by the number of 
workers set in the "worker_processes". I've tested the following:

  before_fork do |server, worker|
    `echo "before_fork $$" >> /tmp/kk`
  end

  after_fork do |server, worker|
    `echo "after_fork  $$" >> /tmp/kk`
  end

And this is the file output:

  before_fork 3447
  after_fork  3456
  before_fork 3454
  before_fork 3463
  after_fork  3464
  after_fork  3472


Could I know what is happening please? This really breaks the concept I had.
In fact I use before_fork to check database connections and so, but I've 
realized that such check if done N (number of workers).

Thanks for any help.




-- 
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: 'before_fork' is executed in parallel by two processes ¿?

[Iñaki Baz Castillo]

El Martes, 5 de Enero de 2010, Iñaki Baz Castillo escribió:
>   before_fork do |server, worker|
>     `echo "before_fork $$" >> /tmp/kk`
>   end


> And this is the file output:
> 
>   before_fork 3447
>   after_fork  3456
>   before_fork 3454
>   before_fork 3463
>   after_fork  3464
>   after_fork  3472


Hummm, this could be something strange related to how `` works...

If I do the following:

  before_fork do |server, worker|
    `echo "before_fork $$" >> /tmp/kk`
     File.open("/tmp/kk", "a") do |f|
       f.puts "$$=#{$$}"
     end
  end

then I get:

  before_fork 11989
  $$=11977
  before_fork 11994
  $$=11977
  before_fork 12000
  $$=11977

So I'm totally lost. However if I test my databases in "before_fork" then I 
see N number of DB connections (N = workers num).





-- 
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: 'before_fork' is executed in parallel by two processes ¿?

[Eric Wong]

Iñaki Baz Castillo <ibc@aliax.net> wrote:
> El Martes, 5 de Enero de 2010, Iñaki Baz Castillo escribió:
> >   before_fork do |server, worker|
> >     `echo "before_fork $$" >> /tmp/kk`
> >   end
> 
> 
> > And this is the file output:
> > 
> >   before_fork 3447
> >   after_fork  3456
> >   before_fork 3454
> >   before_fork 3463
> >   after_fork  3464
> >   after_fork  3472
> 
> 
> Hummm, this could be something strange related to how `` works...
> 
> If I do the following:
> 
>   before_fork do |server, worker|
>     `echo "before_fork $$" >> /tmp/kk`
>      File.open("/tmp/kk", "a") do |f|
>        f.puts "$$=#{$$}"
>      end
>   end

Odd, I'm confused by the way `` is working there too...  I wonder if
this is a Ruby bug...  Anyhow, in your use case system() or your
File.open block should be the way to go.  Don't use `` when system()
will do.

> then I get:
> 
>   before_fork 11989
>   $$=11977
>   before_fork 11994
>   $$=11977
>   before_fork 12000
>   $$=11977
> 
> So I'm totally lost. However if I test my databases in "before_fork" then I 
> see N number of DB connections (N = workers num).

How are you testing the database?  `` or via Ruby code?

-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: 'before_fork' is executed in parallel by two processes ¿?

[Iñaki Baz Castillo]

El Martes, 5 de Enero de 2010, Eric Wong escribió:
> > So I'm totally lost. However if I test my databases in "before_fork" then
> > I  see N number of DB connections (N = workers num).
> 
> How are you testing the database?  `` or via Ruby code?

I use Ruby Sequel ORM.
I've also added the following code to before_fork:

  File.open("/tmp/lalala", "ab") do |f|
    f.puts "My PID is #{$$}"
  end

I confirm that there are two lines printed in that file when Unicorn starts 
(being 2 the number of workers). Is it the expected behavior?

Regards.

-- 
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: 'before_fork' is executed in parallel by two processes ¿?

[Eric Wong]

Iñaki Baz Castillo <ibc@aliax.net> wrote:
> El Martes, 5 de Enero de 2010, Eric Wong escribió:
> > > So I'm totally lost. However if I test my databases in "before_fork" then
> > > I  see N number of DB connections (N = workers num).
> > 
> > How are you testing the database?  `` or via Ruby code?
> 
> I use Ruby Sequel ORM.
> I've also added the following code to before_fork:
> 
>   File.open("/tmp/lalala", "ab") do |f|
>     f.puts "My PID is #{$$}"
>   end
> 
> I confirm that there are two lines printed in that file when Unicorn starts 
> (being 2 the number of workers). Is it the expected behavior?

Yes, before_fork and after_fork are both called for every worker forked.

-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: 'before_fork' is executed in parallel by two processes ¿?

[Iñaki Baz Castillo]

El Martes, 5 de Enero de 2010, Eric Wong escribió:

> Yes, before_fork and after_fork are both called for every worker forked.

So if I just want to test a DB connection then I do better wrtitting such code 
out of before_fork in the config file, right? (at least it's the workaround 
that works for me).

Thanks. 


-- 
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: 'before_fork' is executed in parallel by two processes ¿?

[Eric Wong]

Iñaki Baz Castillo <ibc@aliax.net> wrote:
> El Martes, 5 de Enero de 2010, Eric Wong escribió:
> 
> > Yes, before_fork and after_fork are both called for every worker forked.
> 
> So if I just want to test a DB connection then I do better wrtitting such code 
> out of before_fork in the config file, right? (at least it's the workaround 
> that works for me).

You can also do it on a certain worker only:

  before_fork do |server, worker|
    if worker.nr == 0
      ...
    end
  end

-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: 'before_fork' is executed in parallel by two processes ¿?

[Iñaki Baz Castillo]

El Miércoles, 6 de Enero de 2010, Eric Wong escribió:
> Iñaki Baz Castillo <ibc@aliax.net> wrote:
> > El Martes, 5 de Enero de 2010, Eric Wong escribió:
> > > Yes, before_fork and after_fork are both called for every worker
> > > forked.
> >
> > So if I just want to test a DB connection then I do better wrtitting such
> > code out of before_fork in the config file, right? (at least it's the
> > workaround that works for me).
> 
> You can also do it on a certain worker only:
> 
>   before_fork do |server, worker|
>     if worker.nr == 0
>       ...
>     end
>   end

So if I'm not wrong when before_fork block is runned the $stderr is already 
redirected to the IO set in "stderr_path", rigth?
Then any error in the config file into before_fork would not vi raised to the 
terminal screen but to the stderr (if it has been redirected).

Then I see no advantage on using what you suggest ("if worker.nr == 0 ...") 
over adding such code at the top of the config file. Do I miss something?

Thanks a lot.



-- 
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: 'before_fork' is executed in parallel by two processes ¿?

[Eric Wong]

Iñaki Baz Castillo <ibc@aliax.net> wrote:
> El Miércoles, 6 de Enero de 2010, Eric Wong escribió:
> > Iñaki Baz Castillo <ibc@aliax.net> wrote:
> > > El Martes, 5 de Enero de 2010, Eric Wong escribió:
> > > > Yes, before_fork and after_fork are both called for every worker
> > > > forked.
> > >
> > > So if I just want to test a DB connection then I do better wrtitting such
> > > code out of before_fork in the config file, right? (at least it's the
> > > workaround that works for me).
> > 
> > You can also do it on a certain worker only:
> > 
> >   before_fork do |server, worker|
> >     if worker.nr == 0
> >       ...
> >     end
> >   end
> 
> So if I'm not wrong when before_fork block is runned the $stderr is already 
> redirected to the IO set in "stderr_path", rigth?

Yes, stderr is redirected before any before_fork hooks run.

> Then any error in the config file into before_fork would not vi raised to the 
> terminal screen but to the stderr (if it has been redirected).

If it's a Ruby syntax error, then it would've been sent to the
terminal at startup.  Otherwise it'll be redirected.

> Then I see no advantage on using what you suggest ("if worker.nr == 0 ...") 
> over adding such code at the top of the config file. Do I miss something?

I depends on whether you use preload_app or not, if you use preload_app,
then you'll get to have access to any objects your app creates at
initialization.

-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: 'before_fork' is executed in parallel by two processes ¿?

[Iñaki Baz Castillo]

El Martes, 5 de Enero de 2010, Iñaki Baz Castillo escribió:
> I confirm that there are two lines printed in that file when Unicorn starts 
> (being 2 the number of workers). Is it the expected behavior?

Simpler example:

-----------------
worker_processes 3

before_fork do |server, worker|
  puts "*** I'my PID #{$$}"
end
----------------


Then run Unicorn in foreground and I see:

  *** I'm PID 8791
  *** I'm PID 8791
  *** I'm PID 8791

8791 is the PID of the master process, so yes, it seems that workers don't 
exist yet in before_fork (as obviously expected). However the block into 
before_fork is executed "worker_processes" times by master process.

Do I miss something?

-- 
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: 'before_fork' is executed in parallel by two processes ¿?

[Iñaki Baz Castillo]

El Martes, 5 de Enero de 2010, Eric Wong escribió:
> >   before_fork do |server, worker|
> >     `echo "before_fork $$" >> /tmp/kk`
> >      File.open("/tmp/kk", "a") do |f|
> >        f.puts "$$=#{$$}"
> >      end
> >   end
> 
> Odd, I'm confused by the way `` is working there too...  I wonder if
> this is a Ruby bug...  Anyhow, in your use case system() or your
> File.open block should be the way to go.  Don't use `` when system()
> will do.

I think that the above makse sense since `` creates a new process (with its 
own pid). So each time it's called (once per worker) it will be a different 
process. 


-- 
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying