Rainbows! Rack HTTP server user/dev discussion
 help / color / mirror / code / Atom feed
* possible ThreadSpawn + Ruby 1.8 issues
@ 2009-12-21 22:10 Eric Wong
       [not found] ` <20091221221002.GA12277-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
  0 siblings, 1 reply; 4+ messages in thread
From: Eric Wong @ 2009-12-21 22:10 UTC (permalink / raw)
  To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw

Anybody else using this combination?  If you are, please let us know
if you have any problems with it or even if it's been working great.

I've heard (privately) about issues with accept() somehow blocking the
entire worker process under 1.8.  Obviously this really should not
happen, but I'm unable to reproduce it myself.

Exact details would be most helpful:

* minor version, patchlevel, distribution of Ruby 1.8

* architecture (green threads may behave differently on 32 vs 64-bit)

* outside patches applied (REE, MBARI, zero-copy context-switch, ...)

* ./configure options (--{disable,enable}-pthread, ...)

* application/gems that use threading features internally
  (includes any uses of the 'timeout' library, which Net::HTTP uses).

* Rainbows! version

Unfortunately, there are also IO.select bugs to worry about, too:

  http://www.daniel-azuma.com/blog/view/z2ysbx0e4c3it9/ruby_1_8_7_io_select_threading_bug
  http://redmine.ruby-lang.org/issues/show/1484

Even though we believe it's bad policy to workaround bugs when they're
fixable, 1.8 users (as evidenced by still using 1.8 :P) can be slow to
upgrade and we'll tolerate workarounds to make ThreadSpawn acceptable
under 1.8 since it's mainly designed with 1.8 in mind.

-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH] possible MRI 1.8 thread fix to avoid blocking accept()
       [not found] ` <20091221221002.GA12277-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
@ 2009-12-22  0:54   ` Eric Wong
       [not found]     ` <20091222005421.GA23044-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
  0 siblings, 1 reply; 4+ messages in thread
From: Eric Wong @ 2009-12-22  0:54 UTC (permalink / raw)
  To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw

Under MRI 1.8, listen sockets do not appear to have the
nonblocking I/O flag on by default, nor does it set the
nonblocking I/O flag when calling #accept (but it does
when using #accept_nonblock, of course).

Normally this is not a problem even when using green threads
since MRI will internally select(2) on the file descriptor
before attempting a blocking (and immediately successful)
accept(2).

However, when sharing a listen descriptor across multiple
processes, spurious wakeups are likely to occur, causing
multiple processes may be woken up when a single client
connects.

This causes a problem because accept(2)-ing on multiple
threads/processes for a single connection causes blocking accepts in
multiple processes, leading to stalled green threads.

This is not an issue under 1.9 where a blocking accept() call
unlocks the GVL to let other threads run.
---
  I just pushed this out to git://git.bogomips.org/rainbows.git, too.
  Testers appreciated, thanks.

  Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
  > I've heard (privately) about issues with accept() somehow blocking the
  > entire worker process under 1.8.  Obviously this really should not
  > happen, but I'm unable to reproduce it myself.

 lib/rainbows/base.rb |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/lib/rainbows/base.rb b/lib/rainbows/base.rb
index 211b41c..4a4d076 100644
--- a/lib/rainbows/base.rb
+++ b/lib/rainbows/base.rb
@@ -14,6 +14,12 @@ module Rainbows
       super(worker)
       G.tmp = worker.tmp
 
+      # avoid spurious wakeups and blocking-accept() with 1.8 green threads
+      if RUBY_VERSION.to_f < 1.8
+        require "io/nonblock"
+        LISTENERS.each { |l| l.nonblock = true }
+      end
+
       # we're don't use the self-pipe mechanism in the Rainbows! worker
       # since we don't defer reopening logs
       HttpServer::SELF_PIPE.each { |x| x.close }.clear
-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 2/1] fix Ruby 1.8 detection for (possible) green thread fix
       [not found]     ` <20091222005421.GA23044-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
@ 2009-12-22 20:50       ` Eric Wong
       [not found]         ` <20091222205008.GA11119-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
  0 siblings, 1 reply; 4+ messages in thread
From: Eric Wong @ 2009-12-22 20:50 UTC (permalink / raw)
  To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw

Thanks to Ben Sandofsky for the extra set of eyes
---

 The patch yesterday was bogus and needs this, also pushed out to
 rainbows.git of course.

 lib/rainbows/base.rb |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/rainbows/base.rb b/lib/rainbows/base.rb
index 4a4d076..20c7681 100644
--- a/lib/rainbows/base.rb
+++ b/lib/rainbows/base.rb
@@ -15,7 +15,7 @@ module Rainbows
       G.tmp = worker.tmp
 
       # avoid spurious wakeups and blocking-accept() with 1.8 green threads
-      if RUBY_VERSION.to_f < 1.8
+      if RUBY_VERSION.to_f < 1.9
         require "io/nonblock"
         LISTENERS.each { |l| l.nonblock = true }
       end
-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 3/1] base: fix constant resolution under 1.8 for 1.8 bugfix
       [not found]         ` <20091222205008.GA11119-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
@ 2009-12-22 21:34           ` Eric Wong
  0 siblings, 0 replies; 4+ messages in thread
From: Eric Wong @ 2009-12-22 21:34 UTC (permalink / raw)
  To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw

---
 Oops :x

 lib/rainbows/base.rb |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/rainbows/base.rb b/lib/rainbows/base.rb
index 20c7681..a29a5bb 100644
--- a/lib/rainbows/base.rb
+++ b/lib/rainbows/base.rb
@@ -17,7 +17,7 @@ module Rainbows
       # avoid spurious wakeups and blocking-accept() with 1.8 green threads
       if RUBY_VERSION.to_f < 1.9
         require "io/nonblock"
-        LISTENERS.each { |l| l.nonblock = true }
+        HttpServer::LISTENERS.each { |l| l.nonblock = true }
       end
 
       # we're don't use the self-pipe mechanism in the Rainbows! worker
-- 
Eric Wong
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2009-12-22 21:35 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-12-21 22:10 possible ThreadSpawn + Ruby 1.8 issues Eric Wong
     [not found] ` <20091221221002.GA12277-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-12-22  0:54   ` [PATCH] possible MRI 1.8 thread fix to avoid blocking accept() Eric Wong
     [not found]     ` <20091222005421.GA23044-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-12-22 20:50       ` [PATCH 2/1] fix Ruby 1.8 detection for (possible) green thread fix Eric Wong
     [not found]         ` <20091222205008.GA11119-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-12-22 21:34           ` [PATCH 3/1] base: fix constant resolution under 1.8 for 1.8 bugfix Eric Wong

Code repositories for project(s) associated with this inbox:

	../../../rainbows.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).