possible ThreadSpawn + Ruby 1.8 issues

possible ThreadSpawn + Ruby 1.8 issues

[Eric Wong]

Anybody else using this combination?  If you are, please let us know
if you have any problems with it or even if it's been working great.

I've heard (privately) about issues with accept() somehow blocking the
entire worker process under 1.8.  Obviously this really should not
happen, but I'm unable to reproduce it myself.

Exact details would be most helpful:

* minor version, patchlevel, distribution of Ruby 1.8

* architecture (green threads may behave differently on 32 vs 64-bit)

* outside patches applied (REE, MBARI, zero-copy context-switch, ...)

* ./configure options (--{disable,enable}-pthread, ...)

* application/gems that use threading features internally
  (includes any uses of the 'timeout' library, which Net::HTTP uses).

* Rainbows! version

Unfortunately, there are also IO.select bugs to worry about, too:

  http://www.daniel-azuma.com/blog/view/z2ysbx0e4c3it9/ruby_1_8_7_io_select_threading_bug
  http://redmine.ruby-lang.org/issues/show/1484

Even though we believe it's bad policy to workaround bugs when they're
fixable, 1.8 users (as evidenced by still using 1.8 :P) can be slow to
upgrade and we'll tolerate workarounds to make ThreadSpawn acceptable
under 1.8 since it's mainly designed with 1.8 in mind.

-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

[PATCH] possible MRI 1.8 thread fix to avoid blocking accept()

[Eric Wong]

Under MRI 1.8, listen sockets do not appear to have the
nonblocking I/O flag on by default, nor does it set the
nonblocking I/O flag when calling #accept (but it does
when using #accept_nonblock, of course).

Normally this is not a problem even when using green threads
since MRI will internally select(2) on the file descriptor
before attempting a blocking (and immediately successful)
accept(2).

However, when sharing a listen descriptor across multiple
processes, spurious wakeups are likely to occur, causing
multiple processes may be woken up when a single client
connects.

This causes a problem because accept(2)-ing on multiple
threads/processes for a single connection causes blocking accepts in
multiple processes, leading to stalled green threads.

This is not an issue under 1.9 where a blocking accept() call
unlocks the GVL to let other threads run.
---
  I just pushed this out to git://git.bogomips.org/rainbows.git, too.
  Testers appreciated, thanks.

  Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
  > I've heard (privately) about issues with accept() somehow blocking the
  > entire worker process under 1.8.  Obviously this really should not
  > happen, but I'm unable to reproduce it myself.

 lib/rainbows/base.rb |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/lib/rainbows/base.rb b/lib/rainbows/base.rb
index 211b41c..4a4d076 100644
--- a/lib/rainbows/base.rb
+++ b/lib/rainbows/base.rb
@@ -14,6 +14,12 @@ module Rainbows
       super(worker)
       G.tmp = worker.tmp
 
+      # avoid spurious wakeups and blocking-accept() with 1.8 green threads
+      if RUBY_VERSION.to_f < 1.8
+        require "io/nonblock"
+        LISTENERS.each { |l| l.nonblock = true }
+      end
+
       # we're don't use the self-pipe mechanism in the Rainbows! worker
       # since we don't defer reopening logs
       HttpServer::SELF_PIPE.each { |x| x.close }.clear
-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

[PATCH 2/1] fix Ruby 1.8 detection for (possible) green thread fix

[Eric Wong]

Thanks to Ben Sandofsky for the extra set of eyes
---

 The patch yesterday was bogus and needs this, also pushed out to
 rainbows.git of course.

 lib/rainbows/base.rb |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/rainbows/base.rb b/lib/rainbows/base.rb
index 4a4d076..20c7681 100644
--- a/lib/rainbows/base.rb
+++ b/lib/rainbows/base.rb
@@ -15,7 +15,7 @@ module Rainbows
       G.tmp = worker.tmp
 
       # avoid spurious wakeups and blocking-accept() with 1.8 green threads
-      if RUBY_VERSION.to_f < 1.8
+      if RUBY_VERSION.to_f < 1.9
         require "io/nonblock"
         LISTENERS.each { |l| l.nonblock = true }
       end
-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

[PATCH 3/1] base: fix constant resolution under 1.8 for 1.8 bugfix

[Eric Wong]

---
 Oops :x

 lib/rainbows/base.rb |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/rainbows/base.rb b/lib/rainbows/base.rb
index 20c7681..a29a5bb 100644
--- a/lib/rainbows/base.rb
+++ b/lib/rainbows/base.rb
@@ -17,7 +17,7 @@ module Rainbows
       # avoid spurious wakeups and blocking-accept() with 1.8 green threads
       if RUBY_VERSION.to_f < 1.9
         require "io/nonblock"
-        LISTENERS.each { |l| l.nonblock = true }
+        HttpServer::LISTENERS.each { |l| l.nonblock = true }
       end
 
       # we're don't use the self-pipe mechanism in the Rainbows! worker
-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying