unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
* Rails 7 - Unicorn 6.1.0 -- reaped with preload_app
@ 2022-05-06 13:55 Cédric Carrard
  2022-05-07 17:27 ` Eric Wong
  0 siblings, 1 reply; 2+ messages in thread
From: Cédric Carrard @ 2022-05-06 13:55 UTC (permalink / raw)
  To: unicorn-public

Hello

Since the update to rails 7 and ruby 3.1.1 I have errors with unicorn
6.1.0 (we use docker-compose, nginx, sidekiq, postgresql and
unicorn-worker-killer).

1:10280 configurator.rb:49] Unicorn -- reaped #<Process::Status: pid
415 exit 1> worker=15

Here is the unicorn configuration:

require 'semantic_logger'

worker_processes(File.read('/proc/cpuinfo').scan(/^processor\s*:/).size
* (ENV['WORKERS_PER_CPU'] || 2).to_i)
timeout 300
preload_app true
listen(ENV["PORT"] || 3000)
logger SemanticLogger['Unicorn']

before_fork do |server, _worker|
  Signal.trap 'TERM' do
    server.logger.info 'Unicorn master intercepting TERM and sending
myself QUIT instead'
    Process.kill 'QUIT', Process.pid
  end

  defined?(ActiveRecord::Base) && ActiveRecord::Base.connection.disconnect!
end

after_fork do |server, _worker|
  Signal.trap 'TERM' do
    server.logger.info 'Unicorn worker intercepting TERM and doing
nothing. Wait for master to send QUIT'
  end

  SemanticLogger.reopen

  defined?(ActiveRecord::Base) && ActiveRecord::Base.establish_connection
end

If I remove the preload_app it works (but apparently it is recommended
to leave this setting)

Do you have any idea where the problem could be?

Thanks for your help

------------------------
Best regards

Cédric Carrard

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: Rails 7 - Unicorn 6.1.0 -- reaped with preload_app
  2022-05-06 13:55 Rails 7 - Unicorn 6.1.0 -- reaped with preload_app Cédric Carrard
@ 2022-05-07 17:27 ` Eric Wong
  0 siblings, 0 replies; 2+ messages in thread
From: Eric Wong @ 2022-05-07 17:27 UTC (permalink / raw)
  To: cedric.carrard; +Cc: unicorn-public

Cédric Carrard <cedriccarrard@gmail.com> wrote:
> Hello
> 
> Since the update to rails 7 and ruby 3.1.1 I have errors with unicorn
> 6.1.0 (we use docker-compose, nginx, sidekiq, postgresql and
> unicorn-worker-killer).

You might need something in your before_fork / after_fork hooks
for sidekiq or anything else that might make connections...

more below...

> 1:10280 configurator.rb:49] Unicorn -- reaped #<Process::Status: pid
> 415 exit 1> worker=15

Is there anything else?

> Here is the unicorn configuration:
> 
> require 'semantic_logger'
> 
> worker_processes(File.read('/proc/cpuinfo').scan(/^processor\s*:/).size
> * (ENV['WORKERS_PER_CPU'] || 2).to_i)

Sidenote: Etc.nprocessors is available since Ruby 2.2 and more
portable than reading /proc/cpuinfo.

I also recommend starting with 1-2 workers whenever you're
debugging to help narrow things down.  Some (most) FD
sharing problems appear when going to 1 -> 2, and 2 is usually
as good as dozens (until system memory/FD limits are hit)

> timeout 300
> preload_app true
> listen(ENV["PORT"] || 3000)
> logger SemanticLogger['Unicorn']
> 
> before_fork do |server, _worker|
>   Signal.trap 'TERM' do
>     server.logger.info 'Unicorn master intercepting TERM and sending
> myself QUIT instead'
>     Process.kill 'QUIT', Process.pid
>   end
> 
>   defined?(ActiveRecord::Base) && ActiveRecord::Base.connection.disconnect!

OK, I guess AR is for Postgres.  Any other things that need to
be handled here?  (e.g. sidekiq, sqlite, xapian, memcached, redis, ...).

"lsof -p $PID" on both master and any single worker can be used
to spot inadvertant FD sharing for things that may need to be
shared.

> end
> 
> after_fork do |server, _worker|
>   Signal.trap 'TERM' do
>     server.logger.info 'Unicorn worker intercepting TERM and doing
> nothing. Wait for master to send QUIT'
>   end
> 
>   SemanticLogger.reopen

I assume SemanticLogger is just a filesystem-based logger?
If it establishes a connection, maybe it needed to be
disconnected in before_fork.

>   defined?(ActiveRecord::Base) && ActiveRecord::Base.establish_connection
> end
> 
> If I remove the preload_app it works (but apparently it is recommended
> to leave this setting)

Recommendations depend on needs vs willingness to make things work :)
Yes, preload_app saves memory and improves startup time; but it
requires deeper app knowledge to use correctly (as you're
finding out).

> Do you have any idea where the problem could be?

Maybe the above helps, but it's really your codebase nobody in
here knows...

I have no idea what SemanticLogger is, but having fewer dependencies
can help you narrow things down.  In other words, try the normal
logger.

If you don't know the app very well (e.g. inherited from someone
else); I'd start by editing out large sections of the codebase
to narrow things down.

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2022-05-07 17:27 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-06 13:55 Rails 7 - Unicorn 6.1.0 -- reaped with preload_app Cédric Carrard
2022-05-07 17:27 ` Eric Wong

Code repositories for project(s) associated with this public inbox

	https://yhbt.net/unicorn.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).