Unicorn tempfile chown problem with Ruby 1.9

Unicorn tempfile chown problem with Ruby 1.9

[Andreas Neuhaus]

Hi everybody,

I ran into a problem with unicorn today. Using Ruby 1.9.2-preview3
(compiled by rvm) and a unicorn config that has a "user" statement to
let workers run under a different user, unicorn fails to start any
worker process:

/usr/local/rvm/gems/ruby-1.9.2-preview3/gems/unicorn-1.0.0/lib/unicorn.rb:189:in
`chown': No such file or directory - /tmp/0.8216141026704238
(Errno::ENOENT)
	from /usr/local/rvm/gems/ruby-1.9.2-preview3/gems/unicorn-1.0.0/lib/unicorn.rb:189:in
`user'
	from /usr/local/rvm/gems/ruby-1.9.2-preview3/gems/unicorn-1.0.0/lib/unicorn.rb:664:in
`init_worker_process'
	from /usr/local/rvm/gems/ruby-1.9.2-preview3/gems/unicorn-1.0.0/lib/unicorn.rb:681:in
`worker_loop'
	from /usr/local/rvm/gems/ruby-1.9.2-preview3/gems/unicorn-1.0.0/lib/unicorn.rb:598:in
`block (2 levels) in spawn_missing_workers'

>From what I see, Worker#user tries to chown the unlinked worker
tempfile (in unicorn.rb:189), which fails with Ruby 1.9.2, but seems
to work fine with Ruby 1.8.7. The problem happened to me on an Ubuntu
server, but I was able to reproduce it locally on OSX.

How to reproduce:
- create a simple rack app, like
class Hello; def call (env); [200, { 'Content-Type' => 'text/plain' },
['Hello!']]; end; end
run Hello.new
- create a unicorn.conf.rb with a user statement that switches to the
current user
user 'andy'
- run "unicorn -c unicorn.conf.rb" on Ruby 1.9.2
- watch workers failing to start

I'm not that deep into unicorn's internals, but since the worker
tempfile is unlinked anyway, does it actually make sense to still call
chown on it? Removing tmp.chown in Worker#user solved this for me.

Please cc replies since I'm not subscribed to the list, thanks.

regards,
Andreas Neuhaus
http://zargony.com/
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: Unicorn tempfile chown problem with Ruby 1.9

[Eric Wong]

Andreas Neuhaus <zargony@gmail.com> wrote:
> Hi everybody,
> 
> I ran into a problem with unicorn today. Using Ruby 1.9.2-preview3
> (compiled by rvm) and a unicorn config that has a "user" statement to
> let workers run under a different user, unicorn fails to start any
> worker process:
> 
> /usr/local/rvm/gems/ruby-1.9.2-preview3/gems/unicorn-1.0.0/lib/unicorn.rb:189:in
> `chown': No such file or directory - /tmp/0.8216141026704238
> (Errno::ENOENT)
> 	from /usr/local/rvm/gems/ruby-1.9.2-preview3/gems/unicorn-1.0.0/lib/unicorn.rb:189:in
> `user'
> 	from /usr/local/rvm/gems/ruby-1.9.2-preview3/gems/unicorn-1.0.0/lib/unicorn.rb:664:in
> `init_worker_process'
> 	from /usr/local/rvm/gems/ruby-1.9.2-preview3/gems/unicorn-1.0.0/lib/unicorn.rb:681:in
> `worker_loop'
> 	from /usr/local/rvm/gems/ruby-1.9.2-preview3/gems/unicorn-1.0.0/lib/unicorn.rb:598:in
> `block (2 levels) in spawn_missing_workers'
> 
> >From what I see, Worker#user tries to chown the unlinked worker
> tempfile (in unicorn.rb:189), which fails with Ruby 1.9.2, but seems
> to work fine with Ruby 1.8.7. The problem happened to me on an Ubuntu
> server, but I was able to reproduce it locally on OSX.
> 
> How to reproduce:
> - create a simple rack app, like
> class Hello; def call (env); [200, { 'Content-Type' => 'text/plain' },
> ['Hello!']]; end; end
> run Hello.new
> - create a unicorn.conf.rb with a user statement that switches to the
> current user
> user 'andy'
> - run "unicorn -c unicorn.conf.rb" on Ruby 1.9.2
> - watch workers failing to start
> 
> I'm not that deep into unicorn's internals, but since the worker
> tempfile is unlinked anyway, does it actually make sense to still call
> chown on it? Removing tmp.chown in Worker#user solved this for me.

Hi Andreas,

This is a bug in Ruby and I've filed a report and patch in Redmine:

 http://redmine.ruby-lang.org/issues/show/3451

Unicorn needs to fchmod(2) it as the regular user for the heartbeat
functionality to work (master process continues to run as root
if started as root).

Is Unicorn actually able to serve requests when you removed tmp.chown?
Under Linux, fchmod() doesn't work without ownership of the file.

Btw, for Unicorn 2.0, the fchmod()-based heartbeat implementation will
be a fallback only.  I'll be relying on the mmap() + atomic counters code
of Raindrops[1][2].  mmap() + atomic counters was my first choice
anyways, but I didn't want to write more C at the time.

> Please cc replies since I'm not subscribed to the list, thanks.

No problem :>


[1] - The mmap()/atomic counters parts of Raindrops should be portable
      to modern POSIX-ish systems.  I'll probably use libatomic-ops or
      similar if gcc isn't available.

[2] - http://raindrops.bogomips.org/

-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: Unicorn tempfile chown problem with Ruby 1.9

[Andreas Neuhaus]

Hi Eric,

>> /usr/local/rvm/gems/ruby-1.9.2-preview3/gems/unicorn-1.0.0/lib/unicorn.rb:189:in
>> `chown': No such file or directory - /tmp/0.8216141026704238
>> (Errno::ENOENT)

> This is a bug in Ruby and I've filed a report and patch in Redmine:
>  http://redmine.ruby-lang.org/issues/show/3451

Thanks. I started to watch that ticket to see where it goes. In the
meantime, I tried to workaround the problem by telling unicorn to
chown the tempfile before it's unlinked (which looks like it works
fine for now). http://gist.github.com/444901

> Unicorn needs to fchmod(2) it as the regular user for the heartbeat
> functionality to work (master process continues to run as root
> if started as root).
> Is Unicorn actually able to serve requests when you removed tmp.chown?
> Under Linux, fchmod() doesn't work without ownership of the file.

You're right, it doesn't work if tmp.chown is removed. For
convenience, I always tried it on my local machine as a user and
chown'ed to my own user to trigger the problem. This way I could test
things without running it as root. When running unicorn with tmp.chown
removed as root, it throws EPERM errors as you expected.

regards,
Andreas Neuhaus
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying