From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-2.8 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00 shortcircuit=no autolearn=unavailable version=3.3.2 X-Original-To: yahns-public@yhbt.net Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 4DD9A20453 for ; Sun, 21 Feb 2016 07:16:09 +0000 (UTC) From: Eric Wong To: yahns-public@yhbt.net Subject: [PATCH] doc: mention kqueue/kevent alongside epoll Date: Sun, 21 Feb 2016 07:16:09 +0000 Message-Id: <20160221071609.19834-1-e@80x24.org> List-Id: epoll and kqueue are similar and we use them in a similar way; so mention kqueue alongside epoll for users who may already be familiar with kqueue on *BSD but not epoll under Linux. epoll is a queue, too! --- Documentation/design_notes.txt | 29 +++++++++++++++-------------- Documentation/yahns_config.pod | 7 ++++--- 2 files changed, 19 insertions(+), 17 deletions(-) diff --git a/Documentation/design_notes.txt b/Documentation/design_notes.txt index bf63617..308faa6 100644 --- a/Documentation/design_notes.txt +++ b/Documentation/design_notes.txt @@ -24,12 +24,12 @@ nothing but accepting sockets and injecting into to the event queue worker thread pool ------------------ -This is where all the interesting application dispatch happens in yahns. -epoll(2) (or kqueue(2)) descriptor is the heart of event queue. This -design allows clients to migrate between different threads as they -become active, preventing head-of-line blocking in traditional designs -where a client is pinned to a thread (at the cost of weaker cache -locality). +This is where all the interesting application dispatch happens in +yahns. A descriptor returned by epoll_create1(2) (or kqueue(2)) is +the heart of event queue. This design allows clients to migrate +between different threads as they become active, preventing +head-of-line blocking in traditional designs where a client is +pinned to a thread (at the cost of weaker cache locality). The critical component for implementing this thread pool is "one-shot" notifications in the epoll and kqueue APIs, allowing them to be used as @@ -37,8 +37,8 @@ readiness queues for feeding the thread pool. Used correctly, this allows us to guarantee exclusive access to a client socket without additional locks managed in userspace. -Idle threads will sit performing epoll_wait (or kqueue) indefinitely -until a socket is reported as "ready" by the kernel. +Idle threads will sit performing epoll_wait(2) (or kevent(2)) +indefinitely until a socket is reported as "ready" by the kernel. queue flow ---------- @@ -46,7 +46,7 @@ queue flow Once a client is accept(2)-ed, it is immediately pushed into the worker thread pool (via EPOLL_CTL_ADD or EV_ADD). This mimics the effect of TCP_DEFER_ACCEPT (in Linux) and the "dataready" accept filter (in -FreeBSD) from the perspective of the epoll_wait(2)/kqueue(2) caller. +FreeBSD) from the perspective of the epoll_wait(2)/kevent(2) caller. No explicit locking controlled from userspace is necessary. TCP_DEFER_ACCEPT/"dataready"/"httpready" themselves are not used as it @@ -70,12 +70,13 @@ have completed processing. "Yielding" a client is accomplished by re-arming the already "ready" socket by using EPOLL_CTL_MOD (with EPOLLONESHOT) with a one-shot -notification requeues the descriptor at the end of the internal epoll -ready queue; achieving a similar effect to yielding a thread (via -sched_yield or Thread.pass) in a purely multi-threaded design. +notification requeues the descriptor at the end of the internal +epoll (or kevent) ready queue; achieving a similar effect to +yielding a thread (via sched_yield or Thread.pass) in a purely +multi-threaded design. -Once the client is yielded, epoll_wait is called again to pull -the next client off the ready queue. +Once the client is yielded, epoll_wait or kevent is called again to +pull the next client off the ready queue. Output buffering notes ---------------------- diff --git a/Documentation/yahns_config.pod b/Documentation/yahns_config.pod index cc1ea71..3b1f2e4 100644 --- a/Documentation/yahns_config.pod +++ b/Documentation/yahns_config.pod @@ -110,7 +110,7 @@ be given without a block to associate an app block with a named queue. Usually, only one queue is necessary. Each queue corresponds to -an epoll descriptor and worker thread pool. +an epoll or kqueue descriptor and worker thread pool. Default: NAME defaults to :default @@ -161,9 +161,10 @@ Default: / if daemonized, current working directory if not =item max_events INTEGER This controls the number of events a worker thread will fetch at -once via L. There is no good reason to change this +once via L or L. +There is no good reason to change this unless you use very few (e.g. 1) worker_threads. Leaving this at -1 will give the fairest load balancing behavior with epoll. +1 will give the fairest load balancing behavior with epoll or kqueue. Default: 1 -- EW