[RFC Mongrel2] simpler response API + updated HTTP parser

[Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>]

Hi all,

I've pushed out some changes based on fauna/master[1] to
git://git.bogomips.org/ur-mongrel that includes a good chunk of the
platform-independent stuff found in Unicorn.

The new HTTP parser is named "mongrel_http" to avoid loadtime conflicts
with the old one ("http11") but maintains the same class name
(Mongrel::HttpParser).  This one even supports HTTP/0.9, so "http11"
wasn't an appropriate name for it :)


Problems:

  I'm having some trouble with Rake+Echoe 3.2 with an "uninitialized
  constant Platform" error but everything seems to work by hand without
  Rake+Echoe.

  I'm also getting some test failures under 1.9.1-p243 with the
  semaphore/threading tests.  I haven't looked too hard at this current
  threading model, but my gut feeling is that it's too complicated and a
  "dumber" model in mongrel 1.x *or* a fixed number of worker threads
  doing accept() is sufficient...

  One thing that may be cool is to support multiple
  threading/concurrency models since 1.8/1.9/jruby/rubinius all
  implement threads differently and we can also get Actors with
  1.9/Rubinius.

shortlog and diffstat below:

Eric Wong (6):
      http_response: replace old API with simpler one
      http_response: drop old API compatibility
      remove HeaderOut class
      Add new HTTP/{0.9,1.0,1.1} parser
      Start using the new HTTP parser + TeeInput
      Remove unused Const::HTTP_STATUS_CODES hash

 Manifest                                     |   10 +-
 ext/mongrel_http/c_util.h                    |  107 ++++
 ext/mongrel_http/common_field_optimization.h |  111 ++++
 ext/mongrel_http/ext_help.h                  |   48 ++
 ext/mongrel_http/extconf.rb                  |    8 +
 ext/mongrel_http/global_variables.h          |   91 ++++
 ext/mongrel_http/mongrel_http.rl             |  708 ++++++++++++++++++++++++++
 ext/mongrel_http/mongrel_http_common.rl      |   74 +++
 lib/mongrel.rb                               |   64 +---
 lib/mongrel/const.rb                         |   46 +--
 lib/mongrel/header_out.rb                    |   34 --
 lib/mongrel/http_request.rb                  |  147 ++----
 lib/mongrel/http_response.rb                 |  202 ++------
 lib/mongrel/tee_input.rb                     |  144 ++++++
 test/unit/test_http_parser.rb                |  425 ++++++++++++++--
 test/unit/test_http_parser_ng.rb             |  307 +++++++++++
 test/unit/test_response.rb                   |   12 +-
 test/unit/test_server.rb                     |    3 +
 18 files changed, 2101 insertions(+), 440 deletions(-)

Full changelog:

commit 4e6ab7b7d608bd074107c6a1804401d8165062d4
Author: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
Date:   Sat Sep 12 16:38:22 2009 -0700

    Remove unused Const::HTTP_STATUS_CODES hash
    
    It's no longer used when we generate responses, instead we just
    use the one found in Rack (which was originally "stolen" from
    us) so it's one less thing for us to maintain.

commit 46ca4a1c35b92109cedd59808908e7ad1d289abb
Author: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
Date:   Sat Sep 12 10:40:30 2009 -0700

    Start using the new HTTP parser + TeeInput
    
    The new HTTP parser minimizes the amount of Ruby support code
    needed and the HttpRequest class has been changed to a single
    class method: HttpRequest.read
    
    As a result, this hooks up the TeeInput class into the request
    processing cycle.  TeeInput lets us read the request body off
    the socket while the Rack application is being called (instead
    of being buffered before-hand) while providing rewindable
    semantics that the Rack spec requires.

commit c5a63522bc7e323c706609f7d99ed9f09fe9975d
Author: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
Date:   Fri Sep 11 13:55:20 2009 -0700

    Add new HTTP/{0.9,1.0,1.1} parser
    
    This is descended from the Mongrel parser but modified to
    support:
    
      * chunked transfer-encoding
      * trailers after chunked request bodies
      * HTTP/0.9
      * absolute URI requests
      * multi-line headers with continuation lines
      * repeated headers (joined by commas)
      * #keepalive? boolean method
      * better integration with Rack
    
    This is not yet hooked into any existing parts of Mongrel,
    that is the next step.

commit 8c1c7bdd3c1767708f8507d5aef8ded03b6f1796
Author: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
Date:   Fri Sep 11 13:16:25 2009 -0700

    remove HeaderOut class
    
    HttpResponse has been rewritten to just iterate through the
    headers Rack gives us in a GC-friendly way so we have no need
    for this any longer.

commit 392ea08624e39faec8d5e10ba04b21dfd9ca19a1
Author: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
Date:   Fri Sep 11 12:58:42 2009 -0700

    http_response: drop old API compatibility
    
    Avoid needless overhead in allocating a HttpResponse object and
    instead just use a class method.  This is alright with Rack
    applications since Rack specifies the response is already a
    tuple for writing.  Of course the headers and body of the
    response can both be generated iteratively with #each.

commit 469a507133bd20034df485f03b6eb7b0e82080d6
Author: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
Date:   Fri Sep 11 12:52:20 2009 -0700

    http_response: replace old API with simpler one
    
    The old API is completely dropped, a compatibility layer for the
    old one will be added as Rack middleware instead.  This allows
    newly-written applications to go through fewer layers of
    abstraction.

git: git://git.bogomips.org/ur-mongrel
cgit: http://git.bogomips.org/cgit/ur-mongrel.git

[1] 9f9a9d488ed32a2891dc3dd7d50a17a16357042d

-- 
Eric Wong