[PATCH 1/2] Add QueueQuitter#closed? to queue_quitter_pipe.rb

[PATCH 1/2] Add QueueQuitter#closed? to queue_quitter_pipe.rb

[Lin Jen-Shin]

Since we rely on this class being an IO-like class,
in Yahns::Queue#worker_thread we're using this method.

This fixes an actual error on lib/yahns/queue_kqueue.rb:56
---
 lib/yahns/queue_quitter_pipe.rb | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/lib/yahns/queue_quitter_pipe.rb b/lib/yahns/queue_quitter_pipe.rb
index eabfb00..789eacd 100644
--- a/lib/yahns/queue_quitter_pipe.rb
+++ b/lib/yahns/queue_quitter_pipe.rb
@@ -22,4 +22,8 @@ def close
     @reader.close
     @to_io.close
   end
+
+  def closed?
+    @to_io.closed?
+  end
 end
-- 
2.19.2

[PATCH 2/2] @srv.shutdown could raise Errno::ENOTCONN

[Lin Jen-Shin]

After this fix, all tests except test_client_expire are
passing on my system (macos 10.12.6). Honestly, I don't
really care if it's working perfectly fine. It's just
nice to be able to run the same server on development
machine. Production is of course Linux.
---
 test/test_bin.rb | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/test/test_bin.rb b/test/test_bin.rb
index 6d4faf6..fc85992 100644
--- a/test/test_bin.rb
+++ b/test/test_bin.rb
@@ -99,7 +99,10 @@ def bin_daemon(worker, inherit)
       # Even with a synchronous FD_CLOEXEC, there's a chance of a race
       # because the server does not bind right away.
       unless inherit
-        @srv.shutdown
+        begin
+          @srv.shutdown
+        rescue Errno::ENOTCONN
+        end
         @srv.close
       end
       exec(*@cmd)
-- 
2.19.2

Re: [PATCH 1/2] Add QueueQuitter#closed? to queue_quitter_pipe.rb

[Eric Wong]

Thanks for both patches, applied to https://yhbt.net/yahns.git

Anything else?  Might tag a release, soon.

Getting rid of kgio is going to be a bit of work...

Re: [PATCH 1/2] Add QueueQuitter#closed? to queue_quitter_pipe.rb

[Lin Jen-Shin (godfat)]

On Sat, Dec 8, 2018 at 7:36 PM Eric Wong <e@80x24.org> wrote:
> Thanks for both patches, applied to https://yhbt.net/yahns.git
>
> Anything else?  Might tag a release, soon.

No, so far it's working great locally for me, thank you! I am very
happy I can use yahns on Mac OSX now. So much easier to develop (and
promote to the other developers)

Previously I have no idea why but sleepy_penguin has some compile
error which I can't figure out how to fix, and it really looks like
it's Ruby's bug. They probably fixed it on one of the recent Ruby
patch releases, and it works fine after the `closed?` patch.

It's unrelated but I am happy that yahns quits gracefully upon
receiving SIGINT, while unicorn quits forcefully. I thought I might
try to add something to the application to eliminate the difference,
but since I could just run yahns everywhere I don't have to now.

> Getting rid of kgio is going to be a bit of work...

This really comes with a long way, and on the bright side, we're
moving forward :) Thank you for all the works!

Cheers,

Re: [PATCH 1/2] Add QueueQuitter#closed? to queue_quitter_pipe.rb

[Eric Wong]

"Lin Jen-Shin (godfat)" <godfat@godfat.org> wrote:
> On Sat, Dec 8, 2018 at 7:36 PM Eric Wong <e@80x24.org> wrote:
> > Thanks for both patches, applied to https://yhbt.net/yahns.git
> >
> > Anything else?  Might tag a release, soon.
> 
> No, so far it's working great locally for me, thank you! I am very
> happy I can use yahns on Mac OSX now. So much easier to develop (and
> promote to the other developers)

Great, good to know.

> Previously I have no idea why but sleepy_penguin has some compile
> error which I can't figure out how to fix, and it really looks like
> it's Ruby's bug. They probably fixed it on one of the recent Ruby
> patch releases, and it works fine after the `closed?` patch.

Is it this?
https://bogomips.org/sleepy-penguin/20170415002501.19627-1-e@80x24.org/

Totally forgot about it, and I've even been using a FreeBSD a
bunch, just not with 3rd-party gems.

> It's unrelated but I am happy that yahns quits gracefully upon
> receiving SIGINT, while unicorn quits forcefully. I thought I might
> try to add something to the application to eliminate the difference,
> but since I could just run yahns everywhere I don't have to now.

Yes, I figured since the APIs are different (and I consider signal
handlers part of an API) it's safe to play around with the defaults
a little as long as it's non-destructive on errors.

One downside with graceful-shutdown-by-default and the generous
timeouts is I've had it cause rebooting a machine to take way
longer than expected, which increased downtime :x

> > Getting rid of kgio is going to be a bit of work...
> 
> This really comes with a long way, and on the bright side, we're
> moving forward :) Thank you for all the works!

You're welcome.

Btw, which versions of Ruby are you using?  I'm pretty sure
Ruby 2.0 support compatibility is no longer necessary, these
days; so we can rely on "exception: false" in more places
and slowly remove kgio dependencies.

Fwiw, I'm prepping a fork of yahns to demo Thread::Light
<https://bugs.ruby-lang.org/issues/13618>, too
working on lib/yahns/proxy_pass.rb was too much of a nightmare;
so I wanted to keep all the Ruby parts synchronous.

Re: [PATCH 1/2] Add QueueQuitter#closed? to queue_quitter_pipe.rb

[Lin Jen-Shin (godfat)]

On Sun, Dec 9, 2018 at 1:20 AM Eric Wong <e@80x24.org> wrote:
> > Previously I have no idea why but sleepy_penguin has some compile
> > error which I can't figure out how to fix, and it really looks like
> > it's Ruby's bug. They probably fixed it on one of the recent Ruby
> > patch releases, and it works fine after the `closed?` patch.
>
> Is it this?
> https://bogomips.org/sleepy-penguin/20170415002501.19627-1-e@80x24.org/
>
> Totally forgot about it, and I've even been using a FreeBSD a
> bunch, just not with 3rd-party gems.

It could be related. It was around RSTRUCT_PTR indeed. Looking at the
Ruby header file (ruby.h), I can't remember the detail but it's rather
weird. However if I just fix the header file, it did pass compilation,
yet crashed at runtime. I stopped there.

The thing is, the same version of sleepy_penguin now compiles and
working properly :) The only thing changed was Ruby version.

However indeed sleepy_penguin was actually working on MacOSX in some
older versions, not sure older Ruby or older sleepy_penguin though.
Since then I just swapped to Unicorn locally and didn't really bother
more after failing to fix it.

> One downside with graceful-shutdown-by-default and the generous
> timeouts is I've had it cause rebooting a machine to take way
> longer than expected, which increased downtime :x

That's sad, but if the default doesn't work great, we can always tweak
it? :) I think it's really nice to be shutting down gracefully by
default, since it's much easier to forcefully quit, but not the other
way around. I see that we can still use SIGQUIT for Unicorn, but
ctrl+c is easier to use, and double SIGINT could still shut it down
very quickly anyway.

> Btw, which versions of Ruby are you using?  I'm pretty sure
> Ruby 2.0 support compatibility is no longer necessary, these
> days; so we can rely on "exception: false" in more places
> and slowly remove kgio dependencies.

A few months ago one of the apps was still on 2.3.x, but now it caught
up with 2.5.x. I think Ubuntu default is a good measurement, which
seems to be 2.3 now? To extend support perhaps 2.2 will be safe
enough.

> Fwiw, I'm prepping a fork of yahns to demo Thread::Light
> <https://bugs.ruby-lang.org/issues/13618>, too
> working on lib/yahns/proxy_pass.rb was too much of a nightmare;
> so I wanted to keep all the Ruby parts synchronous.

Wow that thread is really long and I am still reading it, because I
think it's very interesting. I didn't touch this kind of things for
awhile now, but it's still nice to play with them. What I don't quite
get is why people hate green thread yet keeping GVL. I thought that's
somehow contradicting. I would love to see improvement over Fiber or
just Thread::Light, whatever it'll end up with. I really think this is
one of the missing pieces.

I don't know why this is related to proxy_pass though. Would love to
see when you have done something with that.