From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Evan Weaver Newsgroups: gmane.comp.lang.ruby.mongrel.devel Subject: Re: cornifying Mongrel? Date: Thu, 9 Apr 2009 17:34:15 -0400 Message-ID: References: <20090409020925.GA26434@dcvr.yhbt.net> Reply-To: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1239313605 5818 80.91.229.12 (9 Apr 2009 21:46:45 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 9 Apr 2009 21:46:45 +0000 (UTC) To: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Original-X-From: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Thu Apr 09 23:48:03 2009 Return-path: Envelope-to: gclrmd-mongrel-development@m.gmane.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:from:date:x-google-sender-auth:message-id:subject:to :content-type:content-transfer-encoding; bh=DHJcZU0GvPhccKOUW/RfjmyxUqqBCqr+2SAMk7p76Aw=; b=WxX2LHaooJ1+Jx5rty1im+mDOpkxbdJEi8zvQ9ivTDBhSInHzL7OsSXIiR+VFJdSd9 YA5gf9Ao9sv7q5XVE3d3DukYIBP1XO9BF/ztwEgiWEsYJBOMThRdChDiGHGkuVEFxByY L1MATfNi5Il3oFO171OWhBncy5FOZXWnmCC2I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:content-type :content-transfer-encoding; b=BNYIA+Ri3vsrlufRNN+v13g6R50FFCuqM1/12ouV6NTr67OCbJFK5Wv1oO5p/l5bSI kVsxIVgiBh09lQRkgpCNAxBxgGaBxD8vMQbAIN/+8OipqzbeBdGAxDfvr2wmbPV7RMvI 245Td90SKyTW6E0Q5tv2LNBwNzvrnr2b6hqXk= In-Reply-To: <20090409020925.GA26434-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org> X-Google-Sender-Auth: fe6f579b4cfb3ed1 X-BeenThere: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.mongrel.devel:105 Archived-At: Received: from rubyforge.org ([205.234.109.19]) by lo.gmane.org with esmtp (Exim 4.50) id 1Ls26L-0006j3-3G for gclrmd-mongrel-development@m.gmane.org; Thu, 09 Apr 2009 23:48:01 +0200 Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id ED1C5185818F; Thu, 9 Apr 2009 17:46:32 -0400 (EDT) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.29]) by rubyforge.org (Postfix) with ESMTP id 6060A1779931 for ; Thu, 9 Apr 2009 17:34:31 -0400 (EDT) Received: by yw-out-2324.google.com with SMTP id 2so553565ywt.59 for ; Thu, 09 Apr 2009 14:34:30 -0700 (PDT) Received: by 10.150.97.19 with SMTP id u19mr5333411ybb.25.1239312870453; Thu, 09 Apr 2009 14:34:30 -0700 (PDT) List-Post: Ah, awesome. Next weekend is my first free weekend in a month and I will attempt to work on it then. Evan On Wed, Apr 8, 2009 at 10:09 PM, Eric Wong wrote: > Hi all, > > Unicorn development has has been pretty boring lately, so maybe it's > time to merge some of the improvements back into Mongrel2. =A0I think most > of the Unicorn changes can be merged into Mongrel with very few > modifications. =A0I'll give a highish-level rundown of what I changed from > Mongrel (since 9f9a9d488ed32a2891dc3dd7d50a17a16357042d) > > I'll start backwards with HttpResponse since that's the least > intrusive. > > > =3D=3D HttpResponse > > I removed the old Mongrel response interface entirely and just made it > work directly with the Rack response tuplet. =A0The most important change > to me was to stop slurping the entire body into one StringIO. > > I've also added support for Rack-style multi-value headers to > allow duplicate headers joined by "\n" in the value. =A0I consider > suppressing duplicates for certain headers to be the job of Rack > or the framework, so Unicorn itself does no duplicate suppression. > > Additionally, Content-Length: calculation is gone as Rack or the > framework will do that (or send "Transfer-Encoding: chunked"). > > Generally, these changes avoid doing work Rack has already done for us. > > > =3D=3D HttpParser > > I've simplified interface for HttpRequest a bit. =A0It relies solely on > exceptions for errors so callers won't have to explicitly check for > them. =A0There's no longer a need to do nread accounting in the caller > since it's always been done internally. =A0There are also some small > bugfixes for some Rack-isms in there. > > > =3D=3D HttpRequest > > Some of these are tied to the HttpParser changes. =A0The HttpParser > simplification allows me to avoid some function calls for some small > performance boosts. > > I've also renamed "HTTP_BODY" in the HttpParser to :http_body to avoid > conflicting with a theoretical "Body:" HTTP header. > > > =3D=3D Listener > > Unicorn doesn't override TCPListener.new. It supports multiple listeners > within the same process and also allows fine-grained control of backlog, > sndbuf, rcvbuf on a per-listener basis. =A0Lowering listen backlog size > actually has better potential for failover, but it's unlikely I'll get > to test that soon on a real production cluster. =A0Of course, it can > bind to UNIX domain sockets as well as TCP ones. > > I'll probably document the listener inheritance between upgrades feature > separately since I don't think I've ever seen it documented (or used > outside of nginx; let me know if it has). =A0I'm not sure if Mongrel can > support it portably outside of POSIX, probably not... > > > =3D=3D Misc > > The global log-reopening facility in Unicorn will be less clean in the > presence of threads. =A0Unicorn waits until the current request is > completely finished before reopening the logs; this allows requests that > generate multi-line logs to keep all logs within the same file for > easier processing. =A0Nevertheless, Unicorn::Util.reopen_logs would > be preferable over logrotate doing copytruncate. > > > =3D=3D General > > Throughout Unicorn, I use sysread/syswrite instead of the more common > read/write functions. =A0I personally find luserspace I/O-buffering > distasteful, especially on sockets. =A0These calls should be safely > switchable to use read/write for Mongrel since it has threads > and slow clients to deal with (neither are a concern for Unicorn). > > The process management is generally boring and generic (besides reexec). > It can probably be used in a future mongrel_cluster and allowed shared > listeners, too. =A0With preload_app, it works out-of-the box for nearly > everything (Sinatra 0.3.x with code reloading enabled being an > exception). > > Tests can run in parallel using GNU make (my favorite language for > parallel programming). =A0This was one of the first things I did to the > original Mongrel code (though I tweaked it along the way). > > More details for all of these changes are of course in the commit > messages and comments, even. =A0Unicorn is probably the most heavily > commented/documented code I've ever written, too. > > Anyways, let me know if you guys want more details on certain > changes or help merging certain changes into Mongrel. > > > =3D=3D Disclaimer > > Keep in mind that Unicorn still has no real Ruby applications running on > it (fork+exec on cgit doesn't count). =A0The Sinatra projects I initially > tested this on both got canceled, even, so Unicorn could even be bad > luck :) > > Since Unicorn is/was a new project, I took many more liberties and risks > with it than I would with an established project like Mongrel, so keep > that in mind, too :) > > I also don't think I'll be allowed much more $DAYJOB time to work on it. > Most of our projects already run well enough with the nasty > qrp+num_processors=3D1 hack I did last year, and these projects have been > in code/feature freeze for some time, so I don't think I'll ever be > allowed to deploy Unicorn with them... Oh well. > > -- > Eric Wong > _______________________________________________ > Mongrel-development mailing list > Mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org > http://rubyforge.org/mailman/listinfo/mongrel-development > -- = Evan Weaver