From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-2.7 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, URIBL_BLOCKED shortcircuit=no autolearn=unavailable version=3.3.2 X-Original-To: yahns-public@yhbt.net Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 5367420300 for ; Tue, 12 Apr 2016 21:12:55 +0000 (UTC) Date: Tue, 12 Apr 2016 21:12:56 +0000 From: Eric Wong To: yahns-public@yhbt.net Subject: yahns vs unicorn, part 2: accepting connections Message-ID: <20160412-yahns-vs-unicorn-p@rt2> References: <20151101-yahns-vs-unicorn-p@rt1> <20151101092053.GA5328@dcvr.yhbt.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151101092053.GA5328@dcvr.yhbt.net> List-Id: Both servers have the ability to bind and listen on multiple Unix or TCP sockets and share them across multiple worker processes, but that's where the similarities end. While yahns is marketed as a "non-blocking application server", yahns actually uses the blocking accept(2) syscall (or accept4(2) under Linux) to extract connections from the pending connection queue. In an unexpected twist, the old-fashioned unicorn has always relied on non-blocking accept calls despite relying on blocking I/O. unicorn must use non-blocking accept for several reasons: 1. it is the only way to support multiple listen sockets in a single-threaded server design 2. spurious wakeups[1] will unfortunately happen because multiple workers will select(2) on the same listen socket(s), but only one worker can accept the given connection. 3. because of spurious wakeups, a blocking accept which waits can invoke the "timeout" feature which causes the master process to send SIGKILL to the worker. Because unicorn has a "timeout" feature, using blocking accept is actually impossible with a single thread. With the design of unicorn for short-lived connections, it is beneficial to reuse the same worker processes as much as possible to keep CPU caches hot. Thus we don't care about fairly distributing connections across workers, as each worker can only have one connection. With yahns, connections are long-lived and workers may have thousands (if not millions :>) of idle connections. Thus we need to try to balance them as much as possible across workers. Unlike nearly every other massively concurrent server, yahns relies on multiple threads to operate on a SINGLE kqueue/epoll wait set. This unique design allows us to use dedicated threads for accept, allowing at least two beneficial behaviors: 1) we can rely on never getting spurious wakeups since Linux provides "wake-one" behavior for blocking accept calls. For reference, see inet_csk_wait_for_connect in net/ipv4/inet_connection_sock.c of the Linux source: https://80x24.org/mirrors/linux.git/tree/net/ipv4/inet_connection_sock.c?id=v4.5#n255 ...which calls the prepare_to_wait_exclusive function 2) the "wake-one" mechanism also leads to fair FIFO ordering between acceptor threads by using kernel wait queues. The FIFO ordering only happens for "exclusive" operations on wait queues as seen by reading the prepare_to_wait_exclusive function in kernel/sched/wait.c https://80x24.org/mirrors/linux.git/tree/kernel/sched/wait.c?id=v4.5#n193 Note that other wait operations are LIFO (for CPU cache efficiency), only the exclusive wait queue operations are FIFO. With one thread accepting connections per-worker-process, this FIFO behavior results in a fair distribution of connections between worker processes. This is important as yahns (optionally) uses multiple worker processes. Thanks for reading. [1] a spurious wakeup is when a worker wakes up but does not have data to read or a connection to accept