yahns Ruby server user/dev discussion
 help / color / mirror / code / Atom feed
* yahns vs unicorn, part 1: overview
       [not found] <20151101-yahns-vs-unicorn-p@rt1>
@ 2015-11-01  9:20 ` Eric Wong
  2016-04-12 21:12   ` yahns vs unicorn, part 2: accepting connections Eric Wong
  0 siblings, 1 reply; 2+ messages in thread
From: Eric Wong @ 2015-11-01  9:20 UTC (permalink / raw)
  To: yahns-public

yahns and unicorns are complete opposites in terms of design.

Unfortunately, comparisons between yahns and unicorn may be inevitable
since they are implemented and maintained by the same BDFL, and even
share the common HTTP parser and some code.

unicorn was only introduced in 2009, with a design which would have been
considered obsolete in 1999.

But even in 2015, implementations (even non-Ruby ones) based on
one-shot epoll/kqueue notifications like yahns are barely used
and can probably be counted on one hand.

Overview of differences:

* blocking vs non-blocking accept (yahns does the former(!))
* blocking vs non-blocking read/write
* "timeout" configurator directive can only be supported in unicorn
* slow vs fast client handling
* thread-safety, async-signal safety

More to come...

^ permalink raw reply	[flat|nested] 2+ messages in thread

* yahns vs unicorn, part 2: accepting connections
  2015-11-01  9:20 ` yahns vs unicorn, part 1: overview Eric Wong
@ 2016-04-12 21:12   ` Eric Wong
  0 siblings, 0 replies; 2+ messages in thread
From: Eric Wong @ 2016-04-12 21:12 UTC (permalink / raw)
  To: yahns-public

Both servers have the ability to bind and listen on multiple
Unix or TCP sockets and share them across multiple worker
processes, but that's where the similarities end.

While yahns is marketed as a "non-blocking application server",
yahns actually uses the blocking accept(2) syscall (or
accept4(2) under Linux) to extract connections from the pending
connection queue.

In an unexpected twist, the old-fashioned unicorn has always
relied on non-blocking accept calls despite relying on blocking

unicorn must use non-blocking accept for several reasons:

1. it is the only way to support multiple listen sockets in
   a single-threaded server design

2. spurious wakeups[1] will unfortunately happen because
   multiple workers will select(2) on the same listen socket(s),
   but only one worker can accept the given connection.

3. because of spurious wakeups, a blocking accept which waits
   can invoke the "timeout" feature which causes the master
   process to send SIGKILL to the worker.

Because unicorn has a "timeout" feature, using blocking accept
is actually impossible with a single thread.

With the design of unicorn for short-lived connections, it is
beneficial to reuse the same worker processes as much as
possible to keep CPU caches hot.  Thus we don't care about
fairly distributing connections across workers, as each worker
can only have one connection.

With yahns, connections are long-lived and workers may have
thousands (if not millions :>) of idle connections.
Thus we need to try to balance them as much as possible
across workers.

Unlike nearly every other massively concurrent server, yahns
relies on multiple threads to operate on a SINGLE kqueue/epoll
wait set.  This unique design allows us to use dedicated
threads for accept, allowing at least two beneficial behaviors:

1) we can rely on never getting spurious wakeups since Linux
   provides "wake-one" behavior for blocking accept calls.
   For reference, see inet_csk_wait_for_connect in
   net/ipv4/inet_connection_sock.c of the Linux source:


   ...which calls the prepare_to_wait_exclusive function

2) the "wake-one" mechanism also leads to fair FIFO ordering
   between acceptor threads by using kernel wait queues.

   The FIFO ordering only happens for "exclusive" operations
   on wait queues as seen by reading the
   prepare_to_wait_exclusive function in kernel/sched/wait.c


   Note that other wait operations are LIFO (for CPU cache efficiency),
   only the exclusive wait queue operations are FIFO.

With one thread accepting connections per-worker-process, this
FIFO behavior results in a fair distribution of connections
between worker processes.  This is important as yahns
(optionally) uses multiple worker processes.

Thanks for reading.

[1] a spurious wakeup is when a worker wakes up but does
    not have data to read or a connection to accept

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2016-04-12 21:12 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <20151101-yahns-vs-unicorn-p@rt1>
2015-11-01  9:20 ` yahns vs unicorn, part 1: overview Eric Wong
2016-04-12 21:12   ` yahns vs unicorn, part 2: accepting connections Eric Wong

Code repositories for project(s) associated with this public inbox


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).