From: Eric Wong <email@example.com> To: firstname.lastname@example.org Subject: graceful shutdown can take forever Date: Thu, 20 Apr 2017 07:35:15 +0000 [thread overview] Message-ID: <20170420073515.GA6109@dcvr> (raw) Mostly thinking out loud, here... Yes, shutdown_timeout exists as a config directive, but it also kinda sucks since it doesn't currently distinguish between idle and active clients. For timing out clients the server is writing to, that can be really tricky, since the server doesn't always know how much a client is buffering. So if your kernel socket buffers are 4MB and your client is on dialup downloading at 4K/s, the default shutdown_timeout of 15s is completely insufficient. On https://YHBT.net/ where I host some gigantic git repositories and files over 1GB, I have the shutdown_timeout set to 86400 seconds (1 day), which is enough to make "systemctl restart" timeout. This only seems to be triggered by idle clients from some web crawlers, so yes, idle clients should be treated different from active clients... I don't think people with dialup connections tend to download gigantic files off my server... Now that I think about it again, cmogstored (the single-purpose server on which yahns is based) actually becomes single-threaded during the graceful shutdown phase, which might be a reasonable course for yahns to follow, too... The multithreaded queue design can get very tricky as far as checking timeouts go, so simplifying the shutdown phase into a single-threaded portion removed the need for extra locking overhead (any amount of per-client locking overhead would spill into the regular non-shutdown event loop). It also avoided extra branches during normal operation as an additional win. But, I knew all this back in 2013, so... why didn't yahns use it? Hm...., yahns does allow running arbitrary user-supplied code, which can have unpredictable runtimes even if bug-free. In contrast, cmogstored is a single-purpose; and even the rotational disks cmogstored was designed for has a more predictable upper-bound seek times than the dispatch time of user-supplied Rack apps. So, yes, shutting down a multi-threaded event loop gracefully can be tricky...
reply other threads:[~2017-04-20 7:35 UTC|newest] Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: https://yhbt.net/yahns/README * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20170420073515.GA6109@dcvr \ --email@example.com \ --firstname.lastname@example.org \ --subject='Re: graceful shutdown can take forever' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Code repositories for project(s) associated with this inbox: ../../yahns.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).