From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eric Wong Newsgroups: gmane.comp.lang.ruby.mongrel.devel Subject: Re: [PATCH] http11: ~6% performance increase in header parsing Date: Mon, 24 Mar 2008 20:43:07 -0700 Message-ID: <20080325034307.GA13379@soma> References: <20080229015309.GA9080@untitled> <20080303044659.5a550c19.zedshaw@zedshaw.com> <20080302123712.GA13979@hand.yhbt.net> <20080306075421.GA1583@hand.yhbt.net> <3ae7f4480803060153v18e955a5j9038f93e558f81d1@mail.gmail.com> <20080308081210.GA30702@hand.yhbt.net> <20080308220220.GA27749@mayonaise> Reply-To: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1206416606 20976 80.91.229.12 (25 Mar 2008 03:43:26 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 25 Mar 2008 03:43:26 +0000 (UTC) To: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Original-X-From: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Tue Mar 25 04:43:55 2008 Return-path: Envelope-to: gclrmd-mongrel-development@m.gmane.org Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.13 (2006-08-11) X-BeenThere: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.mongrel.devel:7 Archived-At: Received: from rubyforge.org ([205.234.109.19]) by lo.gmane.org with esmtp (Exim 4.50) id 1Je04n-0007HN-Cz for gclrmd-mongrel-development@m.gmane.org; Tue, 25 Mar 2008 04:43:53 +0100 Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 9BB8D1858617; Mon, 24 Mar 2008 23:43:14 -0400 (EDT) Received: from hand.yhbt.net (hand.yhbt.net [66.150.188.102]) by rubyforge.org (Postfix) with ESMTP id D52D71858617 for ; Mon, 24 Mar 2008 23:43:10 -0400 (EDT) Received: from localhost.localdomain (localhost [127.0.0.1]) by hand.yhbt.net (Postfix) with ESMTP id B63057F4153; Mon, 24 Mar 2008 20:43:08 -0700 (PDT) List-Post: Evan Weaver wrote: > Hi all, > > I backported the trunk parser changes and applied the Ragel fix to > branches/stable_1-2 . Two tests are currently failing... could someone > (Eric, Ry?) take a look at these? I am not good enough with Ragel to > be able to say whether or not the eof change is related to the test > failures. Hi Evan, I never committed the Ragel 6 upgrade to trunk because I couldn't figure out why it was broken. I was hoping someone else would step in and fix it for us :) Reverting my attempt at upgrading to Ragel 6 fixes one test... As for the other test that fails, you uncommented the nasty_pound_header test in the commit for multi-line. This test doesn't work in stable-1_1 nor trunk, and I've never made any effort into getting it working (nor was it my fault it's broken in the first place :) Digging into the revision history (with git :), the nasty_pound_header test was committed in its commented form in r361 and has never changed until now. The Mongrel HTTP parser code does not appear to handle multi-line HTTP headers at all, so I'm not surprised that test fails. I would suggest disabling the nasty_pound_header test again until we can successfully parse multi-line headers. > Thanks, > > Evan > > On Sat, Mar 8, 2008 at 6:02 PM, Eric Wong wrote: > > Evan Weaver wrote: > > > > > Upgrading ragel is fine with me. > > > > Here's my work-in-progress. The changes below are pretty much copied > > from ry's changes in ebb. > > > > I haven't been able to figure out why > > test_horrible_queries(HttpParserTest) is failing, so any help here would > > be appreciated. > > > > Thanks. > > > > diff --git a/Rakefile b/Rakefile > > index f47d7a2..07a89c2 100644 > > --- a/Rakefile > > +++ b/Rakefile > > @@ -55,13 +55,15 @@ task :ragel do > > Dir.chdir "ext/http11" do > > target = "http11_parser.c" > > File.unlink target if File.exist? target > > - sh "ragel http11_parser.rl | rlgen-cd -G2 -o #{target}" > > + # sh "ragel http11_parser.rl | rlgen-cd -G2 -o #{target}" # ragel 5.x > > + sh "ragel -G2 http11_parser.rl" # ragel 6.0 > > raise "Failed to build C source" unless File.exist? target > > end > > Dir.chdir "ext/http11" do > > target = "../../ext/http11_java/org/jruby/mongrel/Http11Parser.java" > > File.unlink target if File.exist? target > > - sh "ragel -J http11_parser.java.rl | rlgen-java -o #{target}" > > + # sh "ragel -J http11_parser.java.rl | rlgen-java -o #{target}" # ragel 5.x > > + sh "ragel -J -o #{target} http11_parser.java.rl" # ragel 6.0 > > raise "Failed to build Java source" unless File.exist? target > > end > > end > > diff --git a/ext/http11/http11_parser.rl b/ext/http11/http11_parser.rl > > index a418605..0c4e2d4 100644 > > --- a/ext/http11/http11_parser.rl > > +++ b/ext/http11/http11_parser.rl > > @@ -114,7 +114,7 @@ size_t http_parser_execute(http_parser *parser, const char *buffer, size_t len, > > p = buffer+off; > > pe = buffer+len; > > > > - assert(*pe == '\0' && "pointer does not end on NUL"); > > + /* assert(*pe == '\0' && "pointer does not end on NUL"); */ > > assert(pe - p == len - off && "pointers aren't same distance"); > > > > > > @@ -130,23 +130,11 @@ size_t http_parser_execute(http_parser *parser, const char *buffer, size_t len, > > assert(parser->field_len <= len && "field has length longer than whole buffer"); > > assert(parser->field_start < len && "field starts after buffer end"); > > > > - if(parser->body_start) { > > - /* final \r\n combo encountered so stop right here */ > > - %%write eof; > > - parser->nread++; > > - } > > - > > return(parser->nread); > > } > > > > int http_parser_finish(http_parser *parser) > > { > > - int cs = parser->cs; > > - > > - %%write eof; > > - > > - parser->cs = cs; > > - > > if (http_parser_has_error(parser) ) { > > return -1; > > } else if (http_parser_is_finished(parser) ) { > > @@ -161,5 +149,5 @@ int http_parser_has_error(http_parser *parser) { > > } > > > > int http_parser_is_finished(http_parser *parser) { > > - return parser->cs == http_parser_first_final; > > + return parser->cs >= http_parser_first_final; > > } > > -- > > > > > > Eric Wong > -- > Evan Weaver > Cloudburst, LLC