mirror of mongrel-development@rubyforge.org (inactive)
 help / color / mirror / Atom feed
From: Ezra Zygmuntowicz <ezmobius-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
To: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
Subject: Re: [ANN] Unicorn: UNIX+localhost/LAN-only Mongrel fork
Date: Wed, 11 Feb 2009 16:40:08 -0800	[thread overview]
Message-ID: <DA13594B-59BD-4EBD-8326-C98D5B85819C@gmail.com> (raw)
In-Reply-To: <20090211230457.GB22926-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>

This is really cool. I'm going to play with this now and see how it  


On Feb 11, 2009, at 3:04 PM, Eric Wong wrote:

> Hello all,
> Last week, I finally decided to put into motion some ideas I've been
> kicking around for a year in my head since last year...
> Basically I don't want to have to deal with threads or support  
> platforms
> that rely on or encourage threads.  Especially given MRI 1.9 where
> kernel threads are more difficult to debug than green ones.
> Given the limited scope of this project, I'm thinking it's better to  
> be
> a part of the Mongrel project since there is shared code and use
> the Mongrel Rubyforge account for distributing tarballs/gems.
> I don't have the resources or drive to manage things like support,
> documentation, bug-trackers myself.  Being a UNIX terminal fart, I  
> still
> find the web-based things painful to use.  The source is under the  
> same
> license as Mongrel and in git (see below) so feel free to  
> contribute :)
> Of course I've yet to actually thoroughly test, deploy real  
> applications
> or even run benchmarks on Unicorn.  That'll probably happen later  
> today
> or tomorrow...
> Here's the README (also at http://unicorn.bogomips.org/)
> git repository locations are in there, as well.
> ------------------------------- 8< ----------------------------
> = Unicorn: UNIX + LAN/localhost-only fork of Mongrel
> Only run this behind a full-HTTP-request-buffering reverse proxy if
> you're serving slow clients.  That said, nginx is the only reverse
> proxy we know of that meets this requirement.
> == Features
> * process management: Unicorn will reap and restart workers that
>   die because of broken apps and there is no need to manage
>   multiple processes yourself.
> * doesn't matter if your application is thread-safe or not, workers
>   all run within their own isolated address space and only serve one
>   client at a time...
> * able to listen on multiple interfaces, including UNIX sockets,
>   each worker process can also bind to a private port via the
>   after_fork hook for easy debugging.
> * should support all Rack applications (though only Sinatra has been  
> tested)
> * nginx-style binary re-execution without losing connections.
>   You can upgrade unicorn, your entire application, libraries
>   and even your Ruby interpreter as long as unicorn is
>   installed in the same path
> * before_fork and after_fork hooks in case your application
>   has special needs when dealing with forked processes.  These
>   can be used to setup signal handlers for logrotation, too.
> * Ruby 1.9-compatible (at least the test cases all pass :>)
> == Design
> * Simplicity: Unicorn is a traditional UNIX prefork web server.
>   No threads are used at all, this makes applications easier to debug
>   and fix.  When your application goes awry, a BOFH can just
>   "kill -9" the runaway worker process without worrying about tearing
>   all clients down, just one.  Only UNIX-like systems supporting
>   fork() and file descriptor inheritance is supported.
> * The Ragel->C HTTP parser is taken from Mongrel.  This is the
>   only non-Ruby part and there are no plans to add any more
>   non-Ruby components.
> * All HTTP protocol parsing and I/O is done just like Mongrel:
>   1. read/parse HTTP request in full
>   2. call Rack application
>   3. write HTTP response back to the client
> * Like Mongrel, neither keepalive nor pipelining are supported.
>   These aren't needed since Unicorn is only designed to serve
>   fast, low-latency clients directly.  Do one thing, do it well;
>   let nginx handle slow clients.
> * Configuration is purely in Ruby and eval().  Ruby is less
>   ambiguous than YAML and lets lambdas for before_fork/after_fork
>   hooks be defined inline.  An optional, separate hot_config_file
>   may be used to modify supported configuration changes
>   (and also gives you plenty of rope if you RTFS :>)
> * One master process spawns and reaps worker processes.  The
>   Rack application itself is called only within the worker process  
> (but
>   can be loaded within the master).  A copy-on-write friendly garbage
>   collector like Ruby Enterprise Edition can be used to minimize  
> memory
>   usage.
> * The number of worker processes should be scaled to the number of
>   CPUs, memory or even spindles you have.  If you have an existing
>   Mongrel cluster, using the same amount of processes should work.
>   Let a full-HTTP-request-buffering reverse proxy like nginx manage
>   concurrency to thousands of slow clients for you.  Unicorn scaling
>   should only be concerned about limits of your backend system(s).
> * Load balancing between worker processes is done by the OS kernel.
>   All workers share a common set of listener sockets and does
>   non-blocking accept() on them.  The kernel will decide which worker
>   process to give a socket to and workers will sleep if there is
>   nothing to accept().
> * Since non-blocking accept() is used, there can be a thundering
>   herd when an occasional client connects when application
>   *is not busy*.  The thundering herd problem should not affect
>   applications that are running all the time since worker processes
>   will only select()/accept() outside of the application dispatch.
> * Blocking I/O is used for clients.  This allows a simpler code path
>   to be followed within the Ruby interpreter and fewer syscalls.
>   Applications that use threads should continue to work if Unicorn
>   is serving LAN or localhost clients.
> * Timeout implementation is done via fchmod(2) in each worker
>   on a shared file descriptor to update st_ctime on the inode.
>   Master process wakeups for checking on timeouts is throttled
>   one a second to minimize the performance impact and simplify
>   the code path within the worker.  Neither futimes(2) nor
>   pwrite(2)/pread(2) are supported by base MRI, nor are they as
>   portable on UNIX systems as fchmod(2).
> * SIGKILL is used to terminate the timed-out workers as reliably
>   as possible on a UNIX system.
> * The poor performance of select() on large FD sets is avoided
>   as few file descriptors are used in each worker.
>   There should be no gain from moving to highly scalable but
>   unportable event notification solutions for watching few
>   file descriptors.
> * Since workers only work on one client at a time, the temporary
>   file for storing large POST/PUT requests is reused for the
>   lifetime of the process.  This should limit temp directory
>   growth on UNIX filesystems where temp file names are never
>   purged from the containing directory.
> * If the master process dies unexpectedly for any reason,
>   workers will notice within :timeout/2 seconds and follow
>   the master to its death.
> == License
> Unicorn is copyright 2009 Eric Wong and contributors.
> It is based on Mongrel:
> Mongrel is copyright 2007 Zed A. Shaw and contributors. It is licensed
> under the Ruby license and the GPL2. See the include LICENSE file for
> details.
> == Install
> The library consists of a C extension so you'll need a C compiler or  
> at
> least a friend who can build it for you.
> Finally, the source includes a setup.rb for those who hate RubyGems.
> You can get the source via git via the following locations:
>  git://git.bogomips.org/unicorn.git
>  http://git.bogomips.org/unicorn.git
> == Usage
> See the Sinatra example.
> It should be capable of running all Rack applications.  Since this
> is a preforking webserver
> == Contact
> Newsgroup and mailing list coming, or it'll be a part of the Mongrel  
> project...
> Email Eric Wong at normalperson-rMlxZR9MS24@public.gmane.org for now.
> ------------------------------- 8< ----------------------------
> _______________________________________________
> Mongrel-development mailing list
> Mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
> http://rubyforge.org/mailman/listinfo/mongrel-development

Ezra Zygmuntowicz

  parent reply	other threads:[~2009-02-12  0:46 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-02-11 23:04 [ANN] Unicorn: UNIX+localhost/LAN-only Mongrel fork Eric Wong
     [not found] ` <20090211230457.GB22926-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-02-12  0:32   ` Ryan Dahl
     [not found]     ` <21ee31950902111632y6df95e9h1f9dd642bcc55baf-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2009-02-12  0:59       ` Eric Wong
     [not found]         ` <20090212005932.GB26706-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-02-12  1:08           ` Ryan Dahl
     [not found]             ` <21ee31950902111708m370f6a28s477b2d2fb4af960b-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2009-02-12  3:28               ` Eric Wong
     [not found]                 ` <20090212032844.GA24045-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-02-12 20:05                   ` Evan Weaver
     [not found]                     ` <b6f68fc60902121205t67bc8bd7n2740162a76d1b852-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2009-02-14  9:46                       ` Eric Wong
2009-02-12  0:40   ` Ezra Zygmuntowicz [this message]
     [not found]     ` <DA13594B-59BD-4EBD-8326-C98D5B85819C-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2009-02-16 23:39       ` Eric Wong
     [not found]         ` <20090216233904.GB3198-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-02-17  1:59           ` Ezra Zygmuntowicz
     [not found]             ` <2A92C72C-498A-4A6E-9035-059CCF4C7371-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2009-02-18  0:40               ` Eric Wong
     [not found]                 ` <20090218004036.GA29439-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-02-21 15:10                   ` Eric Wong
2009-02-24  1:03                     ` Eric Wong
     [not found]                       ` <20090224010344.GG26706-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-02-24  2:28                         ` Ezra Zygmuntowicz
     [not found]                           ` <409638DC-A76B-40E1-AE5C-326F2573DACC-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2009-02-24  3:12                             ` Eric Wong
     [not found]                               ` <20090224031223.GH26706-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-02-26  5:55                                 ` Eric Wong
     [not found]                                   ` <20090226055544.GA20153-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2009-02-26  7:06                                     ` Ezra Zygmuntowicz
replies disabled, historical list

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).