unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: subashkc1 <subashkc1@protonmail.com>
Cc: unicorn-public@yhbt.net
Subject: Re: unicorn worker being killed issue
Date: Sun, 7 May 2023 23:13:08 +0000	[thread overview]
Message-ID: <20230507231308.M295161@dcvr> (raw)
In-Reply-To: <w7BWN8IxeMKEPgZbrFk9i2_stHxvb4D-iE4-mpBKvyeLAZvEWS71ki3dsk4qYfjjqezWKdeNcRe8hGX0v96H8TszyPPNI1ktxN6Jwo61-X8=@protonmail.com>

subashkc1 <subashkc1@protonmail.com> wrote:
> I have a weird issue that I'm having trouble debugging. I've
> created a sample bare bones rails app where there issue is
> reproducible https://github.com/subashkc/sidekiq_webui_issue.

That's too big and having https://enterprise.contribsys.com/ as
a source in your Gemfile means I'm probably not able/allowed
to inspect those gems.

Some general questions:

* how are you starting unicorn and what is the config file?
  (mainly, is `preload_app' enabled? and any hooks?)

  `preload_app true' is the most common source of problems if
  the rest of your application (including all dependencies)
  isn't equipped to handle fork.

* What are the RACK_ENV / RAILS_ENV?

`unicorn_rails' is actually pointless nowadays, and the plain
`unicorn' command is highly recommended.

Use a single worker for debugging, especially if you want to
strace/truss the process.

Fwiw, I haven't touched Rails roughly a decade (and barely at
that).  My only web apps for the past 15 years or so are on bare
Rack and meant to be consumed via curl or w3m (nothing involving
JavaScript or fancy browsers).

Maybe others watching this mailbox can chime in....

> After running the rails app, and load up the sidekiq web UI,
> if I click on any tabs e.g. busy/queues then the request is
> served right away with a 200 but it seems there is a 2nd
> request (can't tell where from) which eventually times out and
> kills the unicorn worker.

I suspect a 2nd request comes from some JS code.  Check for a
network debugging tool included with your browser if it has one.

Or use a single unicorn worker, then strace/truss that worker.
Something like `strace -p $PID_OF_WORKER -f -v -s5000 -o /tmp/dump`
(assuming Linux)

Are you able to reproduce the issue with JS disabled?
Preferably just via curl to minimize surprises.

> I originally opened the issue with sidekiq repo but Mike
> (sidekiq owner) wasn't able to find anything wrong with
> sidekiq code and he suggested it seemed unicorn was double
> reading the request and the 2nd one dies after 60s timeout.

Double-reading a request is impossible unless there's something
very broken in Ruby or your networking stack.  There'd be a
hundreds of reports about it if that were true if Ruby or
unicorn were double-reading requests.

Are you going through some firewalls, or caching/scanning
proxies which might send a request twice?  I've seen that before.

I have seen some folks accidentally enable multiple loggers,
so it looks like a single request is happening multiple times.

strace is usually my tool for debugging stuff like this; along
with sprinkling `warn "#{__LINE__} ..."' statements throughout.
And then perhaps commenting out lots of code and disabling
as many dependencies as possible.

  reply	other threads:[~2023-05-07 23:13 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-05-06 22:02 unicorn worker being killed issue subashkc1
2023-05-07 23:13 ` Eric Wong [this message]
2023-05-10 21:34   ` subashkc1
2023-05-11 19:08     ` Eric Wong
2023-05-12  6:41       ` subashkc1
2023-05-12  8:05         ` Eric Wong
2023-05-13  0:10           ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information: https://yhbt.net/unicorn/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230507231308.M295161@dcvr \
    --to=e@80x24.org \
    --cc=subashkc1@protonmail.com \
    --cc=unicorn-public@yhbt.net \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).