From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: * X-Spam-ASN: AS14383 205.234.109.0/24 X-Spam-Status: No, score=1.0 required=3.0 tests=AWL,HK_RANDOM_FROM, MSGID_FROM_MTA_HEADER shortcircuit=no autolearn=no version=3.3.2 Path: news.gmane.org!not-for-mail From: Eric Wong Newsgroups: gmane.comp.lang.ruby.rainbows.general Subject: Re: rainbows for 3rd party api Date: Wed, 4 Nov 2009 13:40:18 -0800 Message-ID: <20091104214018.GA25942@dcvr.yhbt.net> References: <2007122a0911041049u2b4376dbpd3b1f727e315ea88@mail.gmail.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1257370830 2519 80.91.229.12 (4 Nov 2009 21:40:30 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 4 Nov 2009 21:40:30 +0000 (UTC) To: Rainbows! list Original-X-From: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Wed Nov 04 22:40:23 2009 Return-path: Envelope-to: gclrrg-rainbows-talk@m.gmane.org X-Original-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Delivered-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Content-Disposition: inline In-Reply-To: <2007122a0911041049u2b4376dbpd3b1f727e315ea88-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-BeenThere: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.rainbows.general:17 Archived-At: Received: from rubyforge.org ([205.234.109.19]) by lo.gmane.org with esmtp (Exim 4.50) id 1N5naZ-0003Ow-4P for gclrrg-rainbows-talk@m.gmane.org; Wed, 04 Nov 2009 22:40:23 +0100 Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 29F3B185828D; Wed, 4 Nov 2009 16:40:21 -0500 (EST) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by rubyforge.org (Postfix) with ESMTP id E816E185828D for ; Wed, 4 Nov 2009 16:40:19 -0500 (EST) Received: from localhost (unknown [127.0.2.5]) by dcvr.yhbt.net (Postfix) with ESMTP id 09FDF1F449; Wed, 4 Nov 2009 21:40:19 +0000 (UTC) Giovanni Lion wrote: > Hi all, > > I came across rainbows while I was looking for a smart solution for > handling 3rd party api calls for my rails app. I would like to know a > little more about how to achieve efficency in the following context: > > 1 user requests a page > 2 page content requires xml to be retrieved from 3rd party server > through http call > 3 page is rendered, without the 3rd party data but with an onload ajax > request back to the app to retrieve 3rd party data > 4 app generates an http call to 3rd party api > 5 app waits for 3rd party response > 6 app responds to ajax call rendering html out of the xml response > from 3rd party api > > Right now my current setup is apache + passenger, no constraints on > switching to anything else. This setup is not optimal of course > because if i receive many concurrent requests that need 3rd party > response passenger app pool is full and sleepy. From what i read in > the documentation rainbows should come handy in this situation. I had > a look at unicorn and i think i got more or less how it works. Can > anyone suggest me how to set up the app deployment in order to reduce > waste on step 5? My guessing is should create a rack app to handle > these calls using DevFdResponse and run it with rainbows. Only problem > is can i have the rails environment in there? Hi Giovanni, 3rd party API responses are exactly one of the uses Rainbows! was built for. You really only want DevFdResponse if you're doing a straight proxy between the 3rd party and your client without modifying the data. Since you seem to be getting XML and rendering HTML, you probably can't use DevFdResponse efficiently. Don't despair, though, Rainbows! still gives you plenty of options :) You can build a Rack config.ru to use with Rails, too. In fact, you'll have to for now since we're unsure if we want to support a "rainbows_rails" wrapper like I do with "unicorn_rails". Using config.ru gives you much more flexibility to route around/outside of Rails. Your config.ru can be something like this: ---------------- 8< ------------------ # this example is totally untested and may have syntax errors require 'config/boot' # might not be necessary with newer Rails require 'config/environment' # you only need one of these: dispatcher = if $old_rails require 'unicorn/app/old_rails' Unicorn::App::OldRails.new else ActionController::Dispatcher.new end # send all 3rd party API requests to "/3rd_party" through this block: map("/3rd_party") do use Rack::ContentLength run lambda { |env| # error-checking is left as an exercise to the reader :) body = if env['rainbows.model'] == :Revactor url = "http://example.com/#{ENV['PATH_INFO']}" Revactor::HttpClient.request("GET", url).body else Net::HTTP.get("api.example.com", env["PATH_INFO"]) end # render_to_html(body) # define your own function here [ 200, { "Content-Type" => "text/html" }, [ body ] ] } end # send normal Rails requests here: map("/") do use Rack::Lock # only needed in case your Rails app is not thread-safe # you can also use Rainbows::AppPool to limit Rails concurrency here # independently of "/3rd_party" requests if your Rails app is # thread-safe but not happy with too many threads. run dispatcher end ---------------------------------- 8< -------------------------------- The above example will work best with the ThreadPool or ThreadSpawn or Revactor concurrency models. I hope to have time to work on hybrid concurrency models with Rev and EventMachine to mix threads into them so the application dispatch can be concurrent for those, not just client <-> server I/O too. -- Eric Wong