Re: Random crash when sending USR2 + QUIT signals to Unicorn process

[Pere Joan Martorell <pere.joan@camaloon.com>]

2017-07-13 21:34 GMT+02:00 Eric Wong <e@80x24.org>:
> +Cc: Philip and Jonathan  since they encountered this three years
> ago, but we never heard back from them:
>
>         https://bogomips.org/unicorn-public/?q=T_NODE+d:..20170713
>
>
> Pere Joan Martorell <pere.joan@camaloon.com> wrote:
>> > /home/deployer/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/unicorn-5.3.0/lib/unicorn/http_request.rb:80:in `parse': method `hash' called on unexpected T_NODE object (0x0055b15b973508 flags=0xaa31b) (NotImplementedError)
>
>> Any idea what is happening?
>
> This is most likely a bug in a C extension not using write
> barriers correctly (perhaps via undocumented C-API functions in
> Ruby).
>
> I don't think I've seen this on ruby-core bug reports in a few years:
>
>         https://public-inbox.org/ruby-core/?q=T_NODE
>
> Fwiw, Appendix D on Generational GC in the Ruby source is
> worth reading to any C extension authors:
>
>         https://80x24.org/mirrors/ruby.git/plain/doc/extension.rdoc
>
> There are probably build warnings when using some dangerous methods/macros,
> maybe you can check build logs for const warnings.
>
>
> Can you share the list of RubyGems you have loaded and maybe try
> upgrading/replacing/eliminating the ones with C extensions
> one-by-one until the error stops?

Thank you very much for your fast reply. I'm not using Bundler to
manage my dependencies, but I checked it and there's not any conflict
between gem versions.
Seems that I solved the issue removing some of the gems. This was my gem list:

    cuba -v 3.8.0
    slim -v 3.0.8
    cutest -v 1.2.3
    rack-test -v 0.6.3
    sequel -v 4.46.0
    pg -v 0.20.0
    shotgun -v 0.9.2
    shield -v 2.1.1
    sequel_pg -v 1.6.19
    unicorn -v 5.3.0
    capistrano-rbenv -v 2.1.1

And I finally removed these gems:

    cutest -v 1.2.3
    rack-test -v 0.6.3
    shotgun -v 0.9.2
    sequel_pg -v 1.6.19

I suspect that the conflicting gem was 'sequel_pg' (sequel_pg
overwrites the inner loop of the Sequel postgres adapter row fetching
code with a C version. The C version is significantly faster than the
pure ruby version that Sequel uses by default), but given I didn't
remove these gems one by one I can't completely ensure that.

If the problem reemerges I'll keep you informed.

Thanks!! :)


>
> Also, perhaps the output of "pmap $PID_OF_WORKER" if you're on
> Linux (or equivalent command if you're on another OS).
>
> Anyways, I didn't notice anything suspicious in your config.
>
> I'll do another self-audit of the unicorn + kgio + raindrops
> extensions, too, but judging from the lack of reports and how
> much they get used; I suspect the bug is elsewhere (more eyes
> welcome, of course).
>
> Thanks for the report and any more info you can provide!