unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* Re: Exception `Errno::EAGAIN'
  2009-11-15 23:21  7% Exception `Errno::EAGAIN' James Turnbull
@ 2009-11-15 23:47  0% ` Eric Wong
  0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2009-11-15 23:47 UTC (permalink / raw)
  To: unicorn list

James Turnbull <james@lovedthanlost.net> wrote:
> I get this repeated error - about once a second:
> 
> Exception `Errno::EAGAIN' at
> /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 -
> Resource temporarily unavailable - accept(2)
> Exception `Errno::EAGAIN' at
> /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 -
> Resource temporarily unavailable - accept(2)
> Exception `Errno::EAGAIN' at
> /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 -
> Resource temporarily unavailable - accept(2)
> 
> Which seems to be related to:
> 
>         ready.each do |sock|
>           begin
>             process_client(sock.accept_nonblock)
>             nr += 1
>             alive.chmod(m = 0 == m ? 1 : 0)
>           rescue Errno::EAGAIN, Errno::ECONNABORTED
>           end
>           break if nr < 0
>         end

Hi James,

This is expected.  The kernel wakes up all the workers when there's
_one_ connection and they all race to accept() one client connection.
One wins and accepts the connection, 3 lose and go back to sleep.  With
lots of worker processes, this can be a thundering herd problem but
that's why we encourage Unicorn as a backend server, not as a frontend
server.

If we only have one listener, we could do a blocking accept() call to
avoid thundering herds under Linux, but testing on a 16-core box, I
don't remember being able to measure a performance improvement.  But
otherwise we have to use select() with multiple listeners.  nginx uses a
variety of non-portable locking mechanisms to implement its accept
mutex, but that's too much work under Ruby for little/no benefit.  In
the nginx case, sometimes even disabling the accept mutex entirely gives
better performance.

> My unicorn.conf is:
> 
> worker_processes 4
> working_directory "/etc/puppet"
> listen '/tmp/puppet.sock', :backlog => 1
> listen 8140, :tcp_nopush => true
> timeout 10
> pid "/tmp/puppet.pid"
> 
>   preload_app true
>   GC.respond_to?(:copy_on_write_friendly=) and
>   GC.copy_on_write_friendly = true
> 
>   before_fork do |server, worker|
>     # the following is recomended for Rails + "preload_app true"
>     # as there's no need for the master process to hold a connection
> #    defined?(ActiveRecord::Base) and
> #      ActiveRecord::Base.connection.disconnect!
> 
>     # the following allows a new master process to incrementally
>     # phase out the old master process with SIGTTOU to avoid a
>     # thundering herd (especially in the "preload_app false" case)
>     # when doing a transparent upgrade.  The last worker spawned
>     # will then kill off the old master process with a SIGQUIT.
>     old_pid = "#{server.config[:pid]}.oldbin"
>     if old_pid != server.pid
>       begin
>         sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
>         Process.kill(sig, File.read(old_pid).to_i)
>       rescue Errno::ENOENT, Errno::ESRCH
>       end
>     end
> 
>     # optionally throttle the master from forking too quickly by
> sleeping
>     sleep 1
>   end
> 
> The platform is Fedora 10, Ruby 1.8.6.
> 
> Oddly, if I comment out the preload I get:
> 
> Exception `Errno::EPERM' at
> /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:639 -
> Operation not permitted - /tmp/0.520680132392046

Do you by any chance do user switching in your application
or config.ru?

Instead, since 0.94.0, I would do this:

	after_fork do |server, worker|
	  worker.user('user', 'group') if Process.euid == 0
	end

That way only the worker process drops permissions.  The master really
doesn't do anything interesting.   Of course *I* would just start
Unicorn as a regular user and forget the complexity of user switching
entirely...

-- 
Eric Wong

^ permalink raw reply	[relevance 0%]

* Exception `Errno::EAGAIN'
@ 2009-11-15 23:21  7% James Turnbull
  2009-11-15 23:47  0% ` Eric Wong
  0 siblings, 1 reply; 3+ results
From: James Turnbull @ 2009-11-15 23:21 UTC (permalink / raw)
  To: unicorn list

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I get this repeated error - about once a second:

Exception `Errno::EAGAIN' at
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 -
Resource temporarily unavailable - accept(2)
Exception `Errno::EAGAIN' at
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 -
Resource temporarily unavailable - accept(2)
Exception `Errno::EAGAIN' at
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 -
Resource temporarily unavailable - accept(2)

Which seems to be related to:

        ready.each do |sock|
          begin
            process_client(sock.accept_nonblock)
            nr += 1
            alive.chmod(m = 0 == m ? 1 : 0)
          rescue Errno::EAGAIN, Errno::ECONNABORTED
          end
          break if nr < 0
        end

My unicorn.conf is:

worker_processes 4
working_directory "/etc/puppet"
listen '/tmp/puppet.sock', :backlog => 1
listen 8140, :tcp_nopush => true
timeout 10
pid "/tmp/puppet.pid"

  preload_app true
  GC.respond_to?(:copy_on_write_friendly=) and
  GC.copy_on_write_friendly = true

  before_fork do |server, worker|
    # the following is recomended for Rails + "preload_app true"
    # as there's no need for the master process to hold a connection
#    defined?(ActiveRecord::Base) and
#      ActiveRecord::Base.connection.disconnect!

    # the following allows a new master process to incrementally
    # phase out the old master process with SIGTTOU to avoid a
    # thundering herd (especially in the "preload_app false" case)
    # when doing a transparent upgrade.  The last worker spawned
    # will then kill off the old master process with a SIGQUIT.
    old_pid = "#{server.config[:pid]}.oldbin"
    if old_pid != server.pid
      begin
        sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
        Process.kill(sig, File.read(old_pid).to_i)
      rescue Errno::ENOENT, Errno::ESRCH
      end
    end

    # optionally throttle the master from forking too quickly by
sleeping
    sleep 1
  end

The platform is Fedora 10, Ruby 1.8.6.

Oddly, if I comment out the preload I get:

Exception `Errno::EPERM' at
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:639 -
Operation not permitted - /tmp/0.520680132392046
Unhandled listen loop exception #<Errno::EPERM: Operation not
permitted - /tmp/0.520680132392046>.
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:639:in `chmod'
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:639:in
`worker_loop'
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:534:in
`spawn_missing_workers'
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:534:in `fork'
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:534:in
`spawn_missing_workers'
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:530:in `each'
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:530:in
`spawn_missing_workers'
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:540:in
`maintain_worker_count'
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:215:in `start'
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:28:in `run'
/usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/bin/unicorn:165
/usr/bin/unicorn:19:in `load'
/usr/bin/unicorn:19

Repeatedly.

Regards

James Turnbull

- --
Author of:
* Pro Linux System Administration (http://tinyurl.com/linuxadmin)
* Pulling Strings with Puppet (http://tinyurl.com/pupbook)
* Pro Nagios 2.0 (http://tinyurl.com/pronagios)
* Hardening Linux (http://tinyurl.com/hardeninglinux)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEVAwUBSwCM/iFa/lDkFHAyAQIiSggA5r3rFPGiY5AeratLgxPvUDE1OjnrhRGn
HxoRAcM901BNvXT5i7fX5fMTayW7vPXiKd0IazottCDgd40TYlFTQZNpU9hUDx+g
LRAKQ5M5gHTkFX6nqxFlys8WIA3xseDBBUBNxGnZ4YGbqUMg7tQ258b41hiU+4pW
fxQgzYN7JjRQ7/u1Px+86aEEdSGHFYNAOSwG15b+iE/7hQbiWrlFIL7p7XcZqT8M
UxDm6eTkhP1kry7zJ4wTxVsreXgCi75AaCJBfIIoKs0NMParJ/rBWB+42ZA+1t5/
aEaKXBOi47Kw3CmbsSh6Bi4zbTyuavCX6/tfOMNXx6jXygvykUzpgw==
=pJbE
-----END PGP SIGNATURE-----

^ permalink raw reply	[relevance 7%]

* [ANN] unicorn 0.95.0 - we <3 Rainbows! and ponies too
@ 2009-11-15 22:34  3% Eric Wong
  0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2009-11-15 22:34 UTC (permalink / raw)
  To: ruby-talk ML, mongrel-unicorn; +Cc: ruby-talk

Unicorn is a HTTP server for Rack applications designed to only serve
fast clients on low-latency, high-bandwidth connections and take
advantage of features in Unix/Unix-like kernels.  Slow clients should
only be served by placing a reverse proxy capable of fully buffering
both the the request and response in between Unicorn and slow clients.

* http://unicorn.bogomips.org/
* mongrel-unicorn@rubyforge.org
* git://git.bogomips.org/unicorn.git

Changes:

Mostly internal cleanups and documentation updates.  Irrelevant
stacktraces from client disconnects/errors while reading
"rack.input" are now cleared to avoid unnecessary noise.  If
user switching in workers is used, ownership of logs is now
preserved when reopening worker logs (send USR1 only to the the
master in this case).  The timeout config no longer affects long
after_fork hooks or application startups.

New features include the addition of the :umask option for the
"listen" config directive and error reporting for non-portable
socket options.

No ponies have ever been harmed in our development.

Eric Wong (28):
      unicorn.1: document RACK_ENV changes in 0.94.0
      HACKING: update with "gmake" in examples
      don't nuke children for long after_fork and app loads
      local.mk.sample: steal some updates from Rainbows!
      Load Unicorn constants when building app
      tee_input: fix RDoc argument definition for tee
      Add FAQ
      FAQ: fix links to Configurator docs
      tee_input: better premature disconnect handling
      tee_input: don't shadow struct members
      raise Unicorn::ClientShutdown if client aborts in TeeInput
      tee_input: fix comment from an intermediate commit
      FAQ: additional notes on getting HTTPS redirects right
      configurator: update RDoc and comments in examples
      bump version to 0.95.0pre
      configurator: listen :umask parameter for UNIX sockets
      preserve user/group ownership when reopening logs
      old_rails/static: avoid freezing strings
      old_rails: autoload Static
      const: no need to freeze HTTP_EXPECT
      test_server: ensure stderr is written to before reading
      tee_input: expand client error handling
      replace "rescue => e" with "rescue Object => e"
      socket_helper: do not hide errors when setting socket options
      socket_helper: RDoc for constants
      ClientShutdown: RDoc
      Rakefile: add raa_update task
      tee_input: client_error always raises

-- 
Eric Wong



^ permalink raw reply	[relevance 3%]

Results 1-3 of 3 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2009-11-15 22:34  3% [ANN] unicorn 0.95.0 - we <3 Rainbows! and ponies too Eric Wong
2009-11-15 23:21  7% Exception `Errno::EAGAIN' James Turnbull
2009-11-15 23:47  0% ` Eric Wong

Code repositories for project(s) associated with this public inbox

	https://yhbt.net/unicorn.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).