Re: [ANN] unicorn 1.0.1 and 1.1.2 - fix rollbacks

[Lawrence Pit <lawrence.pit@gmail.com>]

>> It appears unicorn rolls back by itself when it can't start the new master. Nice!
>>
>> Jamie also mentions to use the shared vendor/bundler_gems path.  Though I do use that in all my projects I'm not so sure it's wise to do when you can't afford any downtime.  I'll have to wait for unicorn v1.1.3 before I can test whether rolling back will indeed continue with unicorn v1.1.2 instead of v1.1.3 ;o
>>     
>
> Hi Lawrence, why does the shared vendor/bundler_gems cause you downtime? From not re-bundling during rollback?
>
> FWIW I made that recommendation because I ran into issues with unicorns not restarting correctly after running `cap deploy:cleanup`, since the `bundle exec` launches a binary with a path like /app/releases/201007125236/bin/unicorn, which goes missing after N deploys. I haven't tested if this applies to more recent versions
>
> Shared bundles are also significantly faster -- `bundle check || bundle install` is ~1s for me vs. several minutes to totally rebundle
>   

Hi Jamie, I see what you mean. I haven't tested this yet, so I don't 
know for sure if it could cause downtime. I also always use bundle check 
|| bundle install to get that time benefit.

My worry was indeed that it might pick a newer unicorn to restart 
workers for the old master. I guess that can/should be prevented by 
having an on_rollback in my capistrano bundler task.


What is the exact command you use to start unicorn?


What I see is this, I start with:

$ su -l deployer -c "cd /app/current ; bundle exec unicorn_rails -D -c 
/app/current/config/unicorn.rb"

My after_fork and before_exec methods output the ENV vars to the unicorn 
log. I see for example:

AFTER_FORK: VERSION 1
PATH=/app/releases/20100714054705/vendor/bundler_gems/bin:....
GEM_HOME=/app/releases/20100714054705/vendor/bundler_gems
GEM_PATH=/app/releases/20100714054705/vendor/bundler_gems:/app/releases/20100714054705/vendor/bundler_gems/bundler/gems
BUNDLE_GEMFILE=/app/releases/20100714054705/Gemfile

Note that /app/releases/20100714054705/vendor/bundler_gems is a symlink 
to /app/releases/shared/vendor/bundler_gems

When I deploy a new version, symlink /app/current to the new release 
directory, and send USR2 :

executing 
["/app/releases/20100714054705/vendor/bundler_gems/bin/unicorn_rails", 
"-D", "-E", "staging", "-c", "/app/current/config/unicorn.rb"] (in 
/app/releases/20100714055624)
BEFORE_EXEC: VERSION 1
PATH=/app/releases/20100714054705/vendor/bundler_gems/bin:....
GEM_HOME=/app/releases/20100714054705/vendor/bundler_gems
GEM_PATH=/app/releases/20100714054705/vendor/bundler_gems:/app/releases/20100714054705/vendor/bundler_gems/bundler/gems
BUNDLE_GEMFILE=/app/releases/20100714054705/Gemfile
I, [2010-07-15T23:31:47.765995 #23084]  INFO -- : inherited 
addr=/tmp/.unicorn_sock fd=3
I, [2010-07-15T23:31:47.766646 #23084]  INFO -- : Refreshing Gem list
/app/releases/20100714055624/.bundle/environment.rb:175: warning: 
already initialized constant ENV_LOADED
/app/releases/20100714055624/.bundle/environment.rb:176: warning: 
already initialized constant LOCKED_BY
/app/releases/20100714055624/.bundle/environment.rb:177: warning: 
already initialized constant FINGERPRINT
/app/releases/20100714055624/.bundle/environment.rb:178: warning: 
already initialized constant HOME
/app/releases/20100714055624/.bundle/environment.rb:179: warning: 
already initialized constant AUTOREQUIRES
/app/releases/20100714055624/.bundle/environment.rb:181: warning: 
already initialized constant SPECS
AFTER_FORK: VERSION 2
PATH=/app/releases/20100714054705/vendor/bundler_gems/bin:....
GEM_HOME=/app/releases/20100714055624/vendor/bundler_gems
GEM_PATH=/app/releases/20100714055624/vendor/bundler_gems:/app/releases/20100714055624/vendor/bundler_gems/bundler/gems
BUNDLE_GEMFILE=/app/releases/20100714054705/Gemfile


What you see here is that the new worker does have correct GEM_HOME and 
GEM_PATH values, but the PATH and BUNDLE_GEMFILE values are pointing to 
the old dir.

Are those bundler warnings something to worry about?

The BUNDLE_GEMFILE value is worrying I think. I haven't tested this, but 
I'm pretty sure when Bundler.setup is called within your app it will 
actually setup using the old Gemfile then. So that would mean you need 
to reset it somehow?  I don't see how at the moment...  should unicorn 
provide a method similar to +working_directory+ to help ensure the 
application always uses the "current" Gemfile (eg 
"/app/current/Gemfile") ?  Sound kind of strange unicorn should provide 
such a method, is there another way?

What about that PATH value?  What happens if 10 deployments later the 
dir /app/releases/20100714054705 is pruned? I tried. After removing that 
dir I could still send a HUP, but when I send a USR2 I get this:

/app/releases/20100714054705/vendor/bundler_gems/gems/unicorn-1.1.2/lib/unicorn.rb:573:in 
`exec': No such file or directory - 
app/releases/20100714054705/vendor/bundler_gems/bin/unicorn_rails 
(Errno::ENOENT)
        from 
/app/releases/20100714054705/vendor/bundler_gems/gems/unicorn-1.1.2/lib/unicorn.rb:573:in 
`reexec'


 > since the `bundle exec` launches a binary with a path like 
/app/releases/201007125236/bin/unicorn,
 > which goes missing after N deploys


That's what I'm seeing. How did using shared vendor/bundler_gems help 
you?  Because I'm starting with bundle exec using shared 
vendor/bundler_gems as well.




Cheers,
Lawrence
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying