unicorn/reexec/bundler/cap deploy issue

unicorn/reexec/bundler/cap deploy issue

[skaar]

Hi Eric,

we've run into an issue with Unicorn's reexec, when the rails app is
using Bundler and we deploy into dated release directories. Still not
sure what the correct fix is for this, but what happens is that bundler
modifies (acutally prefixes) PATH/RUBYOPT and when you reexec unicorn
these are passed along to the new master process - and Bundler will
again append to the PATH/RUBYOBT.

This becomes more of a problem when you deploy with capistrano and
include SIGUSR2 restart of your unicorn. And effectively it will attempt
to load all Bundler vendor/bundler_gems/environment.rb for all deploys
since unicorn was first started - and if you have, say, a keep only 5
releases strategy, you will soon get failures when a bundler
environment.rb file in RUBYOPT has been shifted out.

I'm still not sure what the right solution is (to modify unicorn, the
unicorn.rb config file or to address it in bundler), but the following
patch does it brute force in unicorn and introduce the assumption that
a reexec should take the same PATH/RUBYOPT variables as the initial
invokation of the unicorn process (using a similar strategy as with
PWD):


>From 3d6ca163723148668d69115ebaf00cb814db8f49 Mon Sep 17 00:00:00 2001
From: skaar <skaar@waste.org>
Date: Sat, 2 Jan 2010 12:29:38 -0500
Subject: [PATCH] filter PATH/RUBYOPT when rexec, and give new master same values as the
 original master process

---
 lib/unicorn.rb |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 225e00a..6b3e334 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -111,6 +111,8 @@ module Unicorn
             Dir.pwd
           end
         }.call,
+      :path => ENV['PATH'],
+      :rybyopt => ENV['RUBYOPT'],
       0 => $0.dup,
     }
 
@@ -488,6 +490,8 @@ module Unicorn
       self.reexec_pid = fork do
         listener_fds = LISTENERS.map { |sock| sock.fileno }
         ENV['UNICORN_FD'] = listener_fds.join(',')
+        ENV['PATH'] = START_CTX[:path]
+        ENV['RUBYOPT'] = START_CTX[:rubyopt]
         Dir.chdir(START_CTX[:cwd])
         cmd = [ START_CTX[0] ].concat(START_CTX[:argv])
 
-- 
1.6.6.rc3



-- 
/skaar

skaar@waste.org
where in the W.A.S.T.E is the wisdom
s_u_b_s_t_r_u_c_t_i_o_n
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

unicorn/reexec/bundler/cap deploy issue

[skaar]

Hi Eric,

we've run into an issue with Unicorn's reexec, when the rails app is
using Bundler and we deploy into dated release directories. Still not
sure what the correct fix is for this, but what happens is that bundler
modifies (acutally prefixes) PATH/RUBYOPT and when you reexec unicorn
these are passed along to the new master process - and Bundler will
again append to the PATH/RUBYOBT.

This becomes more of a problem when you deploy with capistrano and
include SIGUSR2 restart of your unicorn. And effectively it will attempt
to load all Bundler vendor/bundler_gems/environment.rb for all deploys
since unicorn was first started - and if you have, say, a keep only 5
releases strategy, you will soon get failures when a bundler
environment.rb file in RUBYOPT has been shifted out.

I'm still not sure what the right solution is (to modify unicorn, the
unicorn.rb config file or to address it in bundler), but the following
patch does it brute force in unicorn and introduce the assumption that
a reexec should take the same PATH/RUBYOPT variables as the initial
invokation of the unicorn process (using a similar strategy as with
PWD):


>From 3d6ca163723148668d69115ebaf00cb814db8f49 Mon Sep 17 00:00:00 2001
From: skaar <skaar@waste.org>
Date: Sat, 2 Jan 2010 12:29:38 -0500
Subject: [PATCH] filter PATH/RUBYOPT when rexec, and give new master same values as the
 original master process

---
 lib/unicorn.rb |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 225e00a..6b3e334 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -111,6 +111,8 @@ module Unicorn
             Dir.pwd
           end
         }.call,
+      :path => ENV['PATH'],
+      :rybyopt => ENV['RUBYOPT'],
       0 => $0.dup,
     }
 
@@ -488,6 +490,8 @@ module Unicorn
       self.reexec_pid = fork do
         listener_fds = LISTENERS.map { |sock| sock.fileno }
         ENV['UNICORN_FD'] = listener_fds.join(',')
+        ENV['PATH'] = START_CTX[:path]
+        ENV['RUBYOPT'] = START_CTX[:rubyopt]
         Dir.chdir(START_CTX[:cwd])
         cmd = [ START_CTX[0] ].concat(START_CTX[:argv])
 
-- 
1.6.6.rc3


-- 
/skaar

skaar@waste.org
where in the W.A.S.T.E is the wisdom
s_u_b_s_t_r_u_c_t_i_o_n
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: unicorn/reexec/bundler/cap deploy issue

[Eric Wong]

skaar <skaar@waste.org> wrote:
> Hi Eric,
> 
> we've run into an issue with Unicorn's reexec, when the rails app is
> using Bundler and we deploy into dated release directories. Still not
> sure what the correct fix is for this, but what happens is that bundler
> modifies (acutally prefixes) PATH/RUBYOPT and when you reexec unicorn
> these are passed along to the new master process - and Bundler will
> again append to the PATH/RUBYOBT.
> 
> This becomes more of a problem when you deploy with capistrano and
> include SIGUSR2 restart of your unicorn. And effectively it will attempt
> to load all Bundler vendor/bundler_gems/environment.rb for all deploys
> since unicorn was first started - and if you have, say, a keep only 5
> releases strategy, you will soon get failures when a bundler
> environment.rb file in RUBYOPT has been shifted out.
> 
> I'm still not sure what the right solution is (to modify unicorn, the
> unicorn.rb config file or to address it in bundler),

Hi skaar,

Does having this in the Unicorn config file work for you?

  stash_env = %w(PATH RUBYOPT).map { |x| [ x, ENV[x] ] }
  before_exec do |_|
    stash_env.each { |(k,v)| ENV[k] = v }
  end

It's more generic and less surprising for people that (may) want
to change environment variables on upgrades:

> but the following
> patch does it brute force in unicorn and introduce the assumption that
> a reexec should take the same PATH/RUBYOPT variables as the initial
> invokation of the unicorn process (using a similar strategy as with
> PWD):

Early versions of Unicorn actually captured the entire ENV at startup
and restored it before exec.  I felt it was too heavy-handed since I
figured any process that changed its environment variables has good
reason to do so, so I've been trying to leave as much alone as
possible...

-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: unicorn/reexec/bundler/cap deploy issue

[skaar]

> Hi skaar,
> 
> Does having this in the Unicorn config file work for you?
> 
>   stash_env = %w(PATH RUBYOPT).map { |x| [ x, ENV[x] ] }
>   before_exec do |_|
>     stash_env.each { |(k,v)| ENV[k] = v }
>   end
> 
> It's more generic and less surprising for people that (may) want
> to change environment variables on upgrades:
 
yep, that does the job just fine - thanks.

> Early versions of Unicorn actually captured the entire ENV at startup
> and restored it before exec.  I felt it was too heavy-handed since I
> figured any process that changed its environment variables has good
> reason to do so, so I've been trying to leave as much alone as
> possible...

the approach of just stashing what you need from ENV as above is by far
a better approach than just forcing in the reexec code.

thanks again
-- 
/skaar

skaar@waste.org
where in the W.A.S.T.E is the wisdom
s_u_b_s_t_r_u_c_t_i_o_n
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying