* [ANN] cmogstored 1.5.0rc1 - a mogstored alternative @ 2015-11-11 22:05 Eric Wong [not found] ` <CABJfL5iEz47zinp0rbOanrSN=1ubRFAa18wHaBnKSiTc+-Zpew@mail.gmail.com> 0 siblings, 1 reply; 3+ messages in thread From: Eric Wong @ 2015-11-11 22:05 UTC (permalink / raw) To: mogile; +Cc: cmogstored-public cmogstored is an alternative implementation of the "mogstored" storage component of MogileFS. cmogstored is implemented in C and does not use Perl at runtime. cmogstored is the only component you need to install on a MogileFS storage node. Changes since 1.4.3: A bunch of minor changes; most notable is systemd-style socket activation support. This was easy-to-add since we've always had socket activation support for nginx-style SIGUSR2 upgrades. This places no link or runtime dependency on libsystemd, so the LISTEN_FDS and LISTEN_PID environment variables may be used in other init systems as well. While I have my own reservations about systemd itself, I also strongly believe in using socket activation to prevent downtime. Behavior changes: Bad Range: headers return 416 responses in more cases for invalid ranges (e.g. miscalculated ranges such as "1--1", while completely wrong ones (lacking a "bytes=" prefix)) are ignored entirely as in nginx. Bugfixes: There are also some cleanups to avoid dying on OOM in more places on weird systems which trigger OOM. More work on this is ongoing. Also updates to the latest gnulib.git commit f197c2c9e5e0d12c373f26d5b3211809457bc972 ("intprops: new public macro EXPR_SIGNED") along with a change which fixes a memory leak when people build from cmogstored.git using gnulib commit c6148bca89e9465fd6ba3a10d273ec4cb58c2dbe or later ("mountlist: add me_mntroot field on Linux machines"). This memory leak did not affect any released tarballs of cmogstored. shortlog of changes since 1.4.3: doc: use "builder" RubyGem to generate Atom feed dev.c: fail gracefully on out-of-memory errors do not die on OOM when for mgmt paths HACKING: update URLs to reduce redirects http: return 416 errors in more cases for bad Ranges update .gitignores for latest autotools + gnulib Rakefile: remove text-only part from the Atom feed support systemd-style socket activation via environment set TCP listener options on inherited sockets doc: add example systemd config files use free_mount_entry from gnulib instead of rolling our own fix tmpdir dependency for slow Ruby tests doc: publish examples directory to website Expect 1.5.0 final in a week or so (unless I forget :x) http://bogomips.org/cmogstored/files/pre/cmogstored-1.5.0rc1.tar.gz SHA-1 1b301ce5f180f9001275e26fa1c065593cc3569c SHA-256 f99a9d4186c0b267952ec4806e6ccd333ab4441fe36e41417e4a09ccbd0c3e1e * homepage: http://bogomips.org/cmogstored/README * git clone git://bogomips.org/cmogstored.git * cgit: http://bogomips.org/cmogstored.git * gitweb: http://repo.or.cz/w/cmogstored.git * list: cmogstored-public@bogomips.org (subscription optional) * archives: http://bogomips.org/cmogstored-public/ * nntp://news.public-inbox.org/inbox.comp.file-systems.mogilefs.cmogstored -- EW ^ permalink raw reply [flat|nested] 3+ messages in thread
[parent not found: <CABJfL5iEz47zinp0rbOanrSN=1ubRFAa18wHaBnKSiTc+-Zpew@mail.gmail.com>]
* Re: [ANN] cmogstored 1.5.0rc1 - a mogstored alternative [not found] ` <CABJfL5iEz47zinp0rbOanrSN=1ubRFAa18wHaBnKSiTc+-Zpew@mail.gmail.com> @ 2015-11-11 22:41 ` Eric Wong [not found] ` <CABJfL5iZOM3T_pVsqevFx=Q1cFXgaxJjKQhb2=TfGHbwaekevQ@mail.gmail.com> 0 siblings, 1 reply; 3+ messages in thread From: Eric Wong @ 2015-11-11 22:41 UTC (permalink / raw) To: mogile; +Cc: cmogstored-public David Birdsong <david.birdsong -AT- gmail -DOT- com> wrote: > I love the LISTEN_FDS option. Would it be possible to specify the FD > integer as a command line parameter? Possible, yes; but I'm not a fan of having multiple ways to accomplish the same thing. I'd also want to avoid deviating too much from the Perl mogstored command-line. Unless somebody beats me to it, I also plan on implementing LISTEN_FDS/LISTEN_PID support in mogilefsd and Perlbal/mogstored I already did it in Perl a few weeks ago: http://public-inbox.org/meta/20150920044320.2338-2-e%4080x24.org/t/ I consider the inherit() function too trivial to implement any other way and can be public domain / CC0. And also wrote about it in ruby-talk: http://mid.gmane.org/20151030-inherit-fds-w-o-sd_listen_fds-in-pure@ruby > circus from mozilla is a more modern version of supervisor and it offers > socket management but provides the socket through a templated command line. It can't set environment variables? I suppose you can use env(1) from the command-line to launch cmogstored, as it always supported the CMOGSTORED_FD environment. Unlike LISTEN_FDS, CMOGSTORED_FD does not validate $$ == LISTEN_PID like LISTEN_FDS does. It is comma-delimited, so it can use non-contiguous FDs: env CMOGSTORED_FD=3,4 cmogstored ... ^ permalink raw reply [flat|nested] 3+ messages in thread
[parent not found: <CABJfL5iZOM3T_pVsqevFx=Q1cFXgaxJjKQhb2=TfGHbwaekevQ@mail.gmail.com>]
* [PATCH (cmogstored)] doc: document CMOGSTORED_FDS in the manpage [not found] ` <CABJfL5iZOM3T_pVsqevFx=Q1cFXgaxJjKQhb2=TfGHbwaekevQ@mail.gmail.com> @ 2015-11-11 22:55 ` Eric Wong 0 siblings, 0 replies; 3+ messages in thread From: Eric Wong @ 2015-11-11 22:55 UTC (permalink / raw) To: mogile; +Cc: cmogstored-public David Birdsong <david.birdsong -AT- gmail -DOT- com> wrote: > On Wed, Nov 11, 2015 at 2:41 PM, Eric Wong <e@80x24.org> wrote: > > It can't set environment variables? I suppose you can use env(1) from > > the command-line to launch cmogstored, as it always supported the > > CMOGSTORED_FD environment. Unlike LISTEN_FDS, CMOGSTORED_FD > > does not validate $$ == LISTEN_PID like LISTEN_FDS does. It > > is comma-delimited, so it can use non-contiguous FDs: > > > > env CMOGSTORED_FD=3,4 cmogstored ... > > > > > it can set env, but i was thinking that the templating from the socket > support doesn't extend into env. either way, a simple shell shm could do > it, so yeah that's probably fine. The env(1) command-line would be templated, actually:: env CMOGSTORED_FD=%d,%d cmogstored ... Just pushed the following out to cmogstored.git, too. Thanks. Subject: [PATCH] doc: document CMOGSTORED_FDS in the manpage This has always been supported internally, and we can't stop supporting it since we'll be supporting upgrades from old versions indefinitely. So document it, as it has some minor advantages over the LISTEN_{FDS,PID} environment handling of systemd. --- cmogstored.x | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/cmogstored.x b/cmogstored.x index a6b96e6..20eeb36 100644 --- a/cmogstored.x +++ b/cmogstored.x @@ -22,6 +22,12 @@ different sockets, the command line or config file must still specify which addresses to listen on. This feature is available in cmogstored 1.5.0 and later. +CMOGSTORED_FD - a comma-delimited list of integer file descriptors. +This is used internally for SIGUSR2 upgrades, but may be specified +at startup (e.g. via env(1)) as it is not tied to LISTEN_PID +like LISTEN_FDS is. Unlike LISTEN_FDS, CMOGSTORED_FD does not rely +on inherited file descriptors being contiguous. + See MALLOC TUNING for environment variables which may affect memory usage. -- EW ^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2015-11-11 22:55 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-11-11 22:05 [ANN] cmogstored 1.5.0rc1 - a mogstored alternative Eric Wong [not found] ` <CABJfL5iEz47zinp0rbOanrSN=1ubRFAa18wHaBnKSiTc+-Zpew@mail.gmail.com> 2015-11-11 22:41 ` Eric Wong [not found] ` <CABJfL5iZOM3T_pVsqevFx=Q1cFXgaxJjKQhb2=TfGHbwaekevQ@mail.gmail.com> 2015-11-11 22:55 ` [PATCH (cmogstored)] doc: document CMOGSTORED_FDS in the manpage Eric Wong
Code repositories for project(s) associated with this public inbox https://yhbt.net/cmogstored.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).