From: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
Subject: cornifying Mongrel?
Date: Wed, 8 Apr 2009 19:09:25 -0700 [thread overview]
Message-ID: <20090409020925.GA26434@dcvr.yhbt.net> (raw)
Unicorn development has has been pretty boring lately, so maybe it's
time to merge some of the improvements back into Mongrel2. I think most
of the Unicorn changes can be merged into Mongrel with very few
modifications. I'll give a highish-level rundown of what I changed from
Mongrel (since 9f9a9d488ed32a2891dc3dd7d50a17a16357042d)
I'll start backwards with HttpResponse since that's the least
I removed the old Mongrel response interface entirely and just made it
work directly with the Rack response tuplet. The most important change
to me was to stop slurping the entire body into one StringIO.
I've also added support for Rack-style multi-value headers to
allow duplicate headers joined by "\n" in the value. I consider
suppressing duplicates for certain headers to be the job of Rack
or the framework, so Unicorn itself does no duplicate suppression.
Additionally, Content-Length: calculation is gone as Rack or the
framework will do that (or send "Transfer-Encoding: chunked").
Generally, these changes avoid doing work Rack has already done for us.
I've simplified interface for HttpRequest a bit. It relies solely on
exceptions for errors so callers won't have to explicitly check for
them. There's no longer a need to do nread accounting in the caller
since it's always been done internally. There are also some small
bugfixes for some Rack-isms in there.
Some of these are tied to the HttpParser changes. The HttpParser
simplification allows me to avoid some function calls for some small
I've also renamed "HTTP_BODY" in the HttpParser to :http_body to avoid
conflicting with a theoretical "Body:" HTTP header.
Unicorn doesn't override TCPListener.new. It supports multiple listeners
within the same process and also allows fine-grained control of backlog,
sndbuf, rcvbuf on a per-listener basis. Lowering listen backlog size
actually has better potential for failover, but it's unlikely I'll get
to test that soon on a real production cluster. Of course, it can
bind to UNIX domain sockets as well as TCP ones.
I'll probably document the listener inheritance between upgrades feature
separately since I don't think I've ever seen it documented (or used
outside of nginx; let me know if it has). I'm not sure if Mongrel can
support it portably outside of POSIX, probably not...
The global log-reopening facility in Unicorn will be less clean in the
presence of threads. Unicorn waits until the current request is
completely finished before reopening the logs; this allows requests that
generate multi-line logs to keep all logs within the same file for
easier processing. Nevertheless, Unicorn::Util.reopen_logs would
be preferable over logrotate doing copytruncate.
Throughout Unicorn, I use sysread/syswrite instead of the more common
read/write functions. I personally find luserspace I/O-buffering
distasteful, especially on sockets. These calls should be safely
switchable to use read/write for Mongrel since it has threads
and slow clients to deal with (neither are a concern for Unicorn).
The process management is generally boring and generic (besides reexec).
It can probably be used in a future mongrel_cluster and allowed shared
listeners, too. With preload_app, it works out-of-the box for nearly
everything (Sinatra 0.3.x with code reloading enabled being an
Tests can run in parallel using GNU make (my favorite language for
parallel programming). This was one of the first things I did to the
original Mongrel code (though I tweaked it along the way).
More details for all of these changes are of course in the commit
messages and comments, even. Unicorn is probably the most heavily
commented/documented code I've ever written, too.
Anyways, let me know if you guys want more details on certain
changes or help merging certain changes into Mongrel.
Keep in mind that Unicorn still has no real Ruby applications running on
it (fork+exec on cgit doesn't count). The Sinatra projects I initially
tested this on both got canceled, even, so Unicorn could even be bad
Since Unicorn is/was a new project, I took many more liberties and risks
with it than I would with an established project like Mongrel, so keep
that in mind, too :)
I also don't think I'll be allowed much more $DAYJOB time to work on it.
Most of our projects already run well enough with the nasty
qrp+num_processors=1 hack I did last year, and these projects have been
in code/feature freeze for some time, so I don't think I'll ever be
allowed to deploy Unicorn with them... Oh well.
next reply other threads:[~2009-04-09 2:23 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-04-09 2:09 Eric Wong [this message]
[not found] ` <20090409020925.GA26434-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-04-09 21:34 ` cornifying Mongrel? Evan Weaver
replies disabled, historical list
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).