From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eric Wong Newsgroups: gmane.comp.lang.ruby.mongrel.devel Subject: cornifying Mongrel? Date: Wed, 8 Apr 2009 19:09:25 -0700 Message-ID: <20090409020925.GA26434@dcvr.yhbt.net> Reply-To: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1239243738 30779 80.91.229.12 (9 Apr 2009 02:22:18 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 9 Apr 2009 02:22:18 +0000 (UTC) To: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Original-X-From: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Thu Apr 09 04:23:36 2009 Return-path: Envelope-to: gclrmd-mongrel-development@m.gmane.org Content-Disposition: inline User-Agent: Mutt/1.5.18 (2008-05-17) X-BeenThere: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.mongrel.devel:104 Archived-At: Received: from rubyforge.org ([205.234.109.19]) by lo.gmane.org with esmtp (Exim 4.50) id 1LrjvU-0002MN-HF for gclrmd-mongrel-development@m.gmane.org; Thu, 09 Apr 2009 04:23:36 +0200 Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 4F8841779926; Wed, 8 Apr 2009 22:22:09 -0400 (EDT) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by rubyforge.org (Postfix) with ESMTP id 89FB8177992B for ; Wed, 8 Apr 2009 22:09:26 -0400 (EDT) Received: from localhost (unknown [127.0.2.5]) by dcvr.yhbt.net (Postfix) with ESMTP id C940C1F799; Thu, 9 Apr 2009 02:09:25 +0000 (UTC) List-Post: Hi all, Unicorn development has has been pretty boring lately, so maybe it's time to merge some of the improvements back into Mongrel2. I think most of the Unicorn changes can be merged into Mongrel with very few modifications. I'll give a highish-level rundown of what I changed from Mongrel (since 9f9a9d488ed32a2891dc3dd7d50a17a16357042d) I'll start backwards with HttpResponse since that's the least intrusive. == HttpResponse I removed the old Mongrel response interface entirely and just made it work directly with the Rack response tuplet. The most important change to me was to stop slurping the entire body into one StringIO. I've also added support for Rack-style multi-value headers to allow duplicate headers joined by "\n" in the value. I consider suppressing duplicates for certain headers to be the job of Rack or the framework, so Unicorn itself does no duplicate suppression. Additionally, Content-Length: calculation is gone as Rack or the framework will do that (or send "Transfer-Encoding: chunked"). Generally, these changes avoid doing work Rack has already done for us. == HttpParser I've simplified interface for HttpRequest a bit. It relies solely on exceptions for errors so callers won't have to explicitly check for them. There's no longer a need to do nread accounting in the caller since it's always been done internally. There are also some small bugfixes for some Rack-isms in there. == HttpRequest Some of these are tied to the HttpParser changes. The HttpParser simplification allows me to avoid some function calls for some small performance boosts. I've also renamed "HTTP_BODY" in the HttpParser to :http_body to avoid conflicting with a theoretical "Body:" HTTP header. == Listener Unicorn doesn't override TCPListener.new. It supports multiple listeners within the same process and also allows fine-grained control of backlog, sndbuf, rcvbuf on a per-listener basis. Lowering listen backlog size actually has better potential for failover, but it's unlikely I'll get to test that soon on a real production cluster. Of course, it can bind to UNIX domain sockets as well as TCP ones. I'll probably document the listener inheritance between upgrades feature separately since I don't think I've ever seen it documented (or used outside of nginx; let me know if it has). I'm not sure if Mongrel can support it portably outside of POSIX, probably not... == Misc The global log-reopening facility in Unicorn will be less clean in the presence of threads. Unicorn waits until the current request is completely finished before reopening the logs; this allows requests that generate multi-line logs to keep all logs within the same file for easier processing. Nevertheless, Unicorn::Util.reopen_logs would be preferable over logrotate doing copytruncate. == General Throughout Unicorn, I use sysread/syswrite instead of the more common read/write functions. I personally find luserspace I/O-buffering distasteful, especially on sockets. These calls should be safely switchable to use read/write for Mongrel since it has threads and slow clients to deal with (neither are a concern for Unicorn). The process management is generally boring and generic (besides reexec). It can probably be used in a future mongrel_cluster and allowed shared listeners, too. With preload_app, it works out-of-the box for nearly everything (Sinatra 0.3.x with code reloading enabled being an exception). Tests can run in parallel using GNU make (my favorite language for parallel programming). This was one of the first things I did to the original Mongrel code (though I tweaked it along the way). More details for all of these changes are of course in the commit messages and comments, even. Unicorn is probably the most heavily commented/documented code I've ever written, too. Anyways, let me know if you guys want more details on certain changes or help merging certain changes into Mongrel. == Disclaimer Keep in mind that Unicorn still has no real Ruby applications running on it (fork+exec on cgit doesn't count). The Sinatra projects I initially tested this on both got canceled, even, so Unicorn could even be bad luck :) Since Unicorn is/was a new project, I took many more liberties and risks with it than I would with an established project like Mongrel, so keep that in mind, too :) I also don't think I'll be allowed much more $DAYJOB time to work on it. Most of our projects already run well enough with the nasty qrp+num_processors=1 hack I did last year, and these projects have been in code/feature freeze for some time, so I don't think I'll ever be allowed to deploy Unicorn with them... Oh well. -- Eric Wong