Re: What would happen if a worker thread died?

[Eric Wong <e@80x24.org>]

"Lin Jen-Shin (godfat)" <godfat@godfat.org> wrote:
> During the experiments, we found that whenever a worker
> thread died due to LoadError raised from the application,
> which is not a StandardError therefore was not rescued at all,
> crashing the worker thread (assumed, not verified).

Ugh, I guess since it happened in a thread, the error message got
swallowed unless you were running in $DEBUG.

Loading code after the server is ready and serving requests is a bad
idea.  It leads to really nasty thread-safety problems as well as
invalidating the method/constant caches.

> When this happened, the client just hanged forever with yahns.
> Is there something we can do about this? Would yahns respawn
> a new worker thread? Can we close the socket when this happen?

It's unfortunately difficult to detect thread death from ruby (no
SIGCHLD handler unlike for processes) besides polling Thread#join

We had this issue in ruby-core a few years back, but apparently
it was forgotten/ignored by matz.  Care to chime in?
https://bugs.ruby-lang.org/issues/6647

> I am aware that yahns is *extremely sensitive to fatal bugs in the
> applications it hosts*, so I am just curious.
> 
> For reference, Puma would immediately close the socket without
> sending anything, and Unicorn would log the error backtrace and
> kill the worker (If I read it correctly).
> 
> In this case, Unicorn helped me figure out what's happened.

yahns can probably rescue Exception (or Object(!) like puma)
and then log + abort the entire process.