Re: rainbows for 3rd party api

[Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>]

Giovanni Lion <giovanni.lion-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Hi all,
> 
> I came across rainbows while I was looking for a smart solution for
> handling 3rd party api calls for my rails app. I would like to know a
> little more about how to achieve efficency in the following context:
> 
> 1 user requests a page
> 2 page content requires xml to be retrieved from 3rd party server
> through http call
> 3 page is rendered, without the 3rd party data but with an onload ajax
> request back to the app to retrieve 3rd party data
> 4 app generates an http call to 3rd party api
> 5 app waits for 3rd party response
> 6 app responds to ajax call rendering html out of the xml response
> from 3rd party api
> 
> Right now my current setup is apache + passenger, no constraints on
> switching to anything else. This setup is not optimal of course
> because if i receive many concurrent requests that need 3rd party
> response passenger app pool is full and sleepy. From what i read in
> the documentation rainbows should come handy in this situation. I had
> a look at unicorn and i think i got more or less how it works. Can
> anyone suggest me how to set up the app deployment in order to reduce
> waste on step 5? My guessing is should create a rack app to handle
> these calls using DevFdResponse and run it with rainbows. Only problem
> is can i have the rails environment in there?

Hi Giovanni,

3rd party API responses are exactly one of the uses Rainbows! was built
for.

You really only want DevFdResponse if you're doing a straight proxy
between the 3rd party and your client without modifying the data.  Since
you seem to be getting XML and rendering HTML, you probably can't use
DevFdResponse efficiently.  Don't despair, though, Rainbows! still
gives you plenty of options :)

You can build a Rack config.ru to use with Rails, too. In fact, you'll
have to for now since we're unsure if we want to support a
"rainbows_rails" wrapper like I do with "unicorn_rails".  Using
config.ru gives you much more flexibility to route around/outside
of Rails.

Your config.ru can be something like this:
---------------- 8< ------------------
# this example is totally untested and may have syntax errors
require 'config/boot' # might not be necessary with newer Rails
require 'config/environment'

# you only need one of these:
dispatcher = if $old_rails
  require 'unicorn/app/old_rails'
  Unicorn::App::OldRails.new
else
  ActionController::Dispatcher.new
end

# send all 3rd party API requests to "/3rd_party" through this block:
map("/3rd_party") do
  use Rack::ContentLength
  run lambda { |env|
    # error-checking is left as an exercise to the reader :)

    body = if env['rainbows.model'] == :Revactor
      url = "http://example.com/#{ENV['PATH_INFO']}"
      Revactor::HttpClient.request("GET", url).body
    else
      Net::HTTP.get("api.example.com", env["PATH_INFO"])
    end
    # render_to_html(body) # define your own function here
    [ 200, { "Content-Type" => "text/html" }, [ body ] ]
  }
end

# send normal Rails requests here:
map("/") do
  use Rack::Lock # only needed in case your Rails app is not thread-safe
  # you can also use Rainbows::AppPool to limit Rails concurrency here
  # independently of "/3rd_party" requests if your Rails app is
  # thread-safe but not happy with too many threads.
  run dispatcher
end
---------------------------------- 8< --------------------------------

The above example will work best with the ThreadPool or ThreadSpawn or
Revactor concurrency models.

I hope to have time to work on hybrid concurrency models with Rev and
EventMachine to mix threads into them so the application dispatch can
be concurrent for those, not just client <-> server I/O too.

-- 
Eric Wong