From: Eric Wong <bofh@yhbt.net>
To: Toshimaru Enomoto <me@toshimaru.net>
Cc: unicorn-public@yhbt.net
Subject: Re: Memory usage enhancement in unicorn 6.1?
Date: Wed, 16 Feb 2022 22:39:18 +0000 [thread overview]
Message-ID: <20220216223918.M808451@dcvr> (raw)
In-Reply-To: <CABFfzqv+K3ufQ=6wZBF_1J05_rKDF7CoCG5qE7eztur9txVf3Q@mail.gmail.com>
Toshimaru Enomoto <me@toshimaru.net> wrote:
> Hi,
>
> After bumping unicorn from 6.0 to 6.1,
> I observed it reduced memory usage much.
>
> In production, the memory usage has been almost halved! :0
> Startup memory usage is almost the same, but after a few minutes,
> there is a big difference in the memory usage between v6.0 and v6.1.
>
> I looked into the cause of this and I found using epoll reduces memory usage.
> https://gist.github.com/toshimaru/c33cfdb2e1b2540ff0481092e8c36745
Quoting the lynx dump of above gist:
> Environment
>
> * Ruby 3.1 (ruby:3.1 docker image)
> * unicorn 6.1
> * Rails 6.1
>
> Workload
>
> * Run unicorn with 4 worker processes
> * Request 100 times to unicorn app
>
> unicorn 6.1 without epoll
>
> def prep_readers(readers)
> - wtr = Unicorn::Waiter.prep_readers(readers)
> - @timeout *= 500 # to milliseconds for epoll, but halved
> - wtr
> - rescue
> require_relative 'select_waiter'
> @timeout /= 2.0 # halved for IO.select
> Unicorn::SelectWaiter.new
> end
>
> # free -h # before 100 requests
> total used free shared buff/cache available
> Mem: 1.9Gi 805Mi 169Mi 199Mi 1.0Gi 797Mi
> Swap: 1.0Gi 155Mi 868Mi
>
> # free -h # after 100 requests
> total used free shared buff/cache available
> Mem: 1.9Gi 943Mi 85Mi 192Mi 956Mi 670Mi
> Swap: 1.0Gi 163Mi 860Mi
>
> unicorn 6.1 with epoll
>
> def prep_readers(readers)
> wtr = Unicorn::Waiter.prep_readers(readers)
> @timeout *= 500 # to milliseconds for epoll, but halved
> wtr
> - rescue
> - require_relative 'select_waiter'
> - @timeout /= 2.0 # halved for IO.select
> - Unicorn::SelectWaiter.new
> end
>
> # free -h # before 100 requests
> total used free shared buff/cache available
> Mem: 1.9Gi 802Mi 160Mi 192Mi 1.0Gi 811Mi
> Swap: 1.0Gi 163Mi 860Mi
> # free -h # after 100 requests
> total used free shared buff/cache available
> Mem: 1.9Gi 805Mi 156Mi 192Mi 1.0Gi 808Mi
> Swap: 1.0Gi 163Mi 860Mi
>
> Does anyone observe the same enhancement?
I haven't seen it, nor is it expected...
Note: free(1) may be misleading if other processes are active.
What do the VSZ and RSS columns of "ps aux" say?
> Or does anyone have an idea why it happened?
Nobody else responded, so I guess not.
How complex is your application w.r.t memory allocations? The
epoll code path may use fewer allocations and stack than
IO.select, but I figured it'd be lost in the noise since most
applications do a lot more allocations.
Btw, please don't use gist or similar services for small text
that can easily be inlined w/o HTML/JS overhead on websites.
Some users download and read mail offline; requiring extra network
round-trips decreases the likelyhood your messages will be read.
My mindset as a GNU/Linux hacker remains optimizing everything
for i486 users on dialup :)
prev parent reply other threads:[~2022-02-16 22:39 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-23 16:10 Memory usage enhancement in unicorn 6.1? Toshimaru Enomoto
2022-02-16 22:39 ` Eric Wong [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://yhbt.net/unicorn/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220216223918.M808451@dcvr \
--to=bofh@yhbt.net \
--cc=me@toshimaru.net \
--cc=unicorn-public@yhbt.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).