Re: cornifying Mongrel?

[Evan Weaver <evan-72XWLPH10WVXUHR/Jj/Uug@public.gmane.org>]

Ah, awesome.

Next weekend is my first free weekend in a month and I will attempt to
work on it then.

Evan

On Wed, Apr 8, 2009 at 10:09 PM, Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:
> Hi all,
>
> Unicorn development has has been pretty boring lately, so maybe it's
> time to merge some of the improvements back into Mongrel2.  I think most
> of the Unicorn changes can be merged into Mongrel with very few
> modifications.  I'll give a highish-level rundown of what I changed from
> Mongrel (since 9f9a9d488ed32a2891dc3dd7d50a17a16357042d)
>
> I'll start backwards with HttpResponse since that's the least
> intrusive.
>
>
> == HttpResponse
>
> I removed the old Mongrel response interface entirely and just made it
> work directly with the Rack response tuplet.  The most important change
> to me was to stop slurping the entire body into one StringIO.
>
> I've also added support for Rack-style multi-value headers to
> allow duplicate headers joined by "\n" in the value.  I consider
> suppressing duplicates for certain headers to be the job of Rack
> or the framework, so Unicorn itself does no duplicate suppression.
>
> Additionally, Content-Length: calculation is gone as Rack or the
> framework will do that (or send "Transfer-Encoding: chunked").
>
> Generally, these changes avoid doing work Rack has already done for us.
>
>
> == HttpParser
>
> I've simplified interface for HttpRequest a bit.  It relies solely on
> exceptions for errors so callers won't have to explicitly check for
> them.  There's no longer a need to do nread accounting in the caller
> since it's always been done internally.  There are also some small
> bugfixes for some Rack-isms in there.
>
>
> == HttpRequest
>
> Some of these are tied to the HttpParser changes.  The HttpParser
> simplification allows me to avoid some function calls for some small
> performance boosts.
>
> I've also renamed "HTTP_BODY" in the HttpParser to :http_body to avoid
> conflicting with a theoretical "Body:" HTTP header.
>
>
> == Listener
>
> Unicorn doesn't override TCPListener.new. It supports multiple listeners
> within the same process and also allows fine-grained control of backlog,
> sndbuf, rcvbuf on a per-listener basis.  Lowering listen backlog size
> actually has better potential for failover, but it's unlikely I'll get
> to test that soon on a real production cluster.  Of course, it can
> bind to UNIX domain sockets as well as TCP ones.
>
> I'll probably document the listener inheritance between upgrades feature
> separately since I don't think I've ever seen it documented (or used
> outside of nginx; let me know if it has).  I'm not sure if Mongrel can
> support it portably outside of POSIX, probably not...
>
>
> == Misc
>
> The global log-reopening facility in Unicorn will be less clean in the
> presence of threads.  Unicorn waits until the current request is
> completely finished before reopening the logs; this allows requests that
> generate multi-line logs to keep all logs within the same file for
> easier processing.  Nevertheless, Unicorn::Util.reopen_logs would
> be preferable over logrotate doing copytruncate.
>
>
> == General
>
> Throughout Unicorn, I use sysread/syswrite instead of the more common
> read/write functions.  I personally find luserspace I/O-buffering
> distasteful, especially on sockets.  These calls should be safely
> switchable to use read/write for Mongrel since it has threads
> and slow clients to deal with (neither are a concern for Unicorn).
>
> The process management is generally boring and generic (besides reexec).
> It can probably be used in a future mongrel_cluster and allowed shared
> listeners, too.  With preload_app, it works out-of-the box for nearly
> everything (Sinatra 0.3.x with code reloading enabled being an
> exception).
>
> Tests can run in parallel using GNU make (my favorite language for
> parallel programming).  This was one of the first things I did to the
> original Mongrel code (though I tweaked it along the way).
>
> More details for all of these changes are of course in the commit
> messages and comments, even.  Unicorn is probably the most heavily
> commented/documented code I've ever written, too.
>
> Anyways, let me know if you guys want more details on certain
> changes or help merging certain changes into Mongrel.
>
>
> == Disclaimer
>
> Keep in mind that Unicorn still has no real Ruby applications running on
> it (fork+exec on cgit doesn't count).  The Sinatra projects I initially
> tested this on both got canceled, even, so Unicorn could even be bad
> luck :)
>
> Since Unicorn is/was a new project, I took many more liberties and risks
> with it than I would with an established project like Mongrel, so keep
> that in mind, too :)
>
> I also don't think I'll be allowed much more $DAYJOB time to work on it.
> Most of our projects already run well enough with the nasty
> qrp+num_processors=1 hack I did last year, and these projects have been
> in code/feature freeze for some time, so I don't think I'll ever be
> allowed to deploy Unicorn with them... Oh well.
>
> --
> Eric Wong
> _______________________________________________
> Mongrel-development mailing list
> Mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
> http://rubyforge.org/mailman/listinfo/mongrel-development
>



-- 
Evan Weaver