Rainbows! Rack HTTP server user/dev discussion
 help / color / mirror / code / Atom feed
From: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
To: Rainbows! list <rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org>
Subject: Re: negative timeout in Rainbows::Fiber::Base
Date: Wed, 29 Aug 2012 21:17:07 +0000	[thread overview]
Message-ID: <20120829211707.GA22726@dcvr.yhbt.net> (raw)
In-Reply-To: <CAA2_N1thakAOVp7ibCNic+TjEVvXE0OGLgzXH3fJ1c2UTs68oQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

"Lin Jen-Shin (godfat)" <godfat-hOE/xeEBYYIdnm+yROfE0A@public.gmane.org> wrote:
> Sorry that I didn't subscribe the mailing list (now subscribed),
> so I didn't realize there's already a response. The quoted
> text is copied from archive.

Oops :x  I encourage unsubscribed senders to remind repliers
to Cc: them for this reason.

Unfortunately ruby-talk has conditioned the Ruby community to favor
subscriber-only lists and the Mailman config for rainbows-talk avoids
Cc-by-default (even though we'll still accept unsubscribed senders).

> Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> > "Lin Jen-Shin (godfat)" <godfat at godfat.org> wrote:
> > Yes.  (Though it should've be easy to just pipe that part of my
> > message to "git apply" or even "patch -p1")
> I did that in the beginning, but it looks like your patch was not
> created from 12281e4cee86496588814cd1851e8764caea024c,
> so I cannot apply it? Anyway, not important :)

Anyways, I've pushed my patch to master as
commit e794a40049959a23ba311c572e518bb7c2861812

Will probably release v4.4.1 with that tonight unless there's something

> > All the Thread-based concurrency models should already just work
> > with Celluloid-based apps.
> > http://news.gmane.org/find-root.php?message_id=%3cBANLkTimnNQYpk7TRkiCvhDikzGHVy%3dkOdw%40mail.gmail.com%3e
> And it's not worth the effort to wrap around celluloid-io for
> buffering requests? Since it's using nio4r[0] underneath
> and it's using libev for I/O, I thought that would be a replacement
> for eventmachine?
> [0]: https://github.com/tarcieri/nio4r

I haven't followed celluloid* closely since it was originally announced.
Maybe it's worth it to offer celluloid-io as an option and I will accept
patches for it.

> > It's much easier to unwatch IO objects with Cool.io than with EM, so I
> > haven't done much with EM + Fibers.  There's also rack-fiber_pool...
> True, working with Cool.io is much more pleasant. As for rack-fiber_pool,
> I am not sure if I am correct or not, but in my humble option, that's a
> totally wrong approach. I think fibers should be handled in server level,
> not application level. That way, we don't need the hack for throwing :async,
> and the middlewares do not need to be aware of fibers and :async,
> and we won't need async-rack, and the server doesn't need to be
> aware of :async, so on so forth.
> I am not sure if this approach would have any bad influence, but at least
> it works fine in our applications. That is, just wrap fibers around the calling
> the app. (i.e. Fiber.new{ app_call(env) }.resume)

Rainbows! tries not to favor/disfavor any approaches to concurrency.
And rack-fiber_pool was easy-to-support, so I added support for it.

> > Rainbows::EventMachine::TryDefer already uses the EM-internal thread
> > pool, so I think EventMachineThreadPool would be redundant.
> I didn't know this before, I guess it should work for me, though
> it might not be as direct as:
> https://github.com/godfat/ruby-server-exp/blob/9d44f8387739f5395bf97aff1689f17615cc4c7e/config/rainbows-em-thread-pool.rb#L21-25
>   class REMTPoolClient < Rainbows::EventMachine::Client
>     def app_call input
>       EM.defer{ super }
>     end
>   end

I seem to recall this didn't work well with some corner cases
(large static files/socket-proxying in responses).

> > > https://github.com/cardinalblue/rest-more/blob/master/example/rainbows.rb#L15-19
> > > class REMFClient < Rainbows::EventMachine::Client
> > >   def app_call input
> > >     Fiber.new{ super }.resume
> > >   end
> > > end
> > Does it actually show benefits compared to regular EM?
> > I suppose the Rack application still needs to be made Fiber-aware
> > to reap benefits of Fibers
> The Rack application doesn't need to be fiber-aware, but if
> the libraries are fiber-aware, then it would be beneficial.
> The rest of program doesn't have to know the existence of
> fibers, because we don't throw :async.

OK.  But are clients served concurrently by Rainbows! in this case?

I'm not sure if I'm following this correctly (haven't thought about
Fibers in a long time), but control never goes back to Rainbows! itself
in your above case, does it?

> > Yes, threads are far more compatible with existing gems/libraries and I
> > think that's the better way to go.  Of course, just processes (with
> > regular unicorn) should be most compatible, too.
> At first I thought fibers are better for I/O, but after implementing
> a thread based approach for the same mechanism, I started to
> doubt that. I don't have a conclusion yet though.
> (I am a bit nervous to switch to thread based approach on
> production to find out, since it's not yet tested intensively)
> On the other hand, all our apps are running on Heroku, and
> we need to reduce memory usage because they have a hard
> limit. That's why we're running Zbatery at the moment.

Interesting.  Are you on 32 or 64-bit and are you constrained by VMSize
or RSS?

If it's VMSize and you're not dealing with deep data structures _and_
your code doesn't use recursion much; lowering pthreads stack size to
64K in thread_pthread.c should help with memory usage.  Not sure if you
can supply your own Ruby builds for that hosting service, though.

If you have time, you could also try making something like GNU pth or
npth work with Ruby 1.9.  I suspect Fibers will become unusable
with *pth, though...

> > > Sorry that I might be asking too many questions in a single thread.
> > No problem!  I'm just happy that folks are showing interest
> I really don't understand why Unicorn family didn't get much
> attention in Ruby community. And I always feel wrong when
> people are comparing Unicorn with Thin or even Passenger.
> (sigh)

I'm actually surprised unicorn got the attention it has.

As project leader, my preference to maintain a low public profile and my
refusal to endorse/associate with for-profit interests certainly hurts

If somebody else (like you) wants to market these projects, then more
power to them :)  I'm certainly never going to speak at any conference.

Rainbows! just overwhelms potential users with too many choices,
so it's unlikely to ever be widely-adopted.

> Many thanks for your work and help!

No problem :)
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
Do not quote signatures (like this one) or top post when replying

  parent reply	other threads:[~2012-08-29 21:17 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-08-23 20:36 negative timeout in Rainbows::Fiber::Base Lin Jen-Shin (godfat)
     [not found] ` <CAA2_N1unOXb7Z4Jr8oKoSLu266O9Ko4o=oWzAcMA1w3=9X74KA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-08-25  2:45   ` Eric Wong
     [not found]     ` <20120825024556.GA25977-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-08-26  0:12       ` Lin Jen-Shin (godfat)
     [not found]         ` <CAA2_N1uhfcHDbTvY+ke0Cid6=i7KEhFn8jvEirx+ptYVDacdvA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-08-26  1:15           ` Eric Wong
2012-08-29 16:00           ` Lin Jen-Shin (godfat)
     [not found]             ` <CAA2_N1thakAOVp7ibCNic+TjEVvXE0OGLgzXH3fJ1c2UTs68oQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-08-29 21:17               ` Eric Wong [this message]
     [not found]                 ` <20120829211707.GA22726-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-08-30 21:33                   ` Lin Jen-Shin (godfat)
     [not found]                     ` <CAA2_N1tc=Xx8WHaM8H=EWshyzGEyX04PnkdBGj9Jdb7cSzmbRQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-08-31  1:37                       ` Eric Wong
     [not found]                         ` <20120831013731.GA16613-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-09-05 20:06                           ` Lin Jen-Shin (godfat)
     [not found]                             ` <CAA2_N1vfWXGw_CaaMWMijUSdMN2Pz882SYDtNEW2_6YWffgTKQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-09-05 23:27                               ` Eric Wong
     [not found]                                 ` <20120905232739.GA25153-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-09-22  9:52                                   ` Lin Jen-Shin (godfat)
     [not found]                                     ` <CAA2_N1v460utbL31Qu-JbGuUxav1hY4X5+cEf=Mp2rOC5efzMw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-09-22 19:42                                       ` Eric Wong
     [not found]                                         ` <20120922194222.GA6839-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-09-28 15:14                                           ` Lin Jen-Shin (godfat)
     [not found]                                             ` <CAA2_N1usHJVZgn5n7RaTyDCbK7eu6G4ocZAsvqsVeL6cPERskw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-09-28 19:11                                               ` Eric Wong
     [not found]                                                 ` <20120928191132.GA14292-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-09-28 19:24                                                   ` Eric Wong
     [not found]                                                     ` <20120928192449.GB14292-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-10-31  0:14                                                       ` Lin Jen-Shin (godfat)
2012-12-18 11:09                                                       ` Lin Jen-Shin (godfat)
     [not found]                                                         ` <CAA2_N1tcA-HK20C8Ok1Lv9KWwMD4fctCOPHTLeD9ayRJqWby1Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-12-18 19:19                                                           ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information: https://yhbt.net/rainbows/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20120829211707.GA22726@dcvr.yhbt.net \
    --to=normalperson-rmlxzr9ms24@public.gmane.org \
    --cc=rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).