unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
* Memory usage enhancement in unicorn 6.1?
@ 2022-01-23 16:10 Toshimaru Enomoto
  2022-02-16 22:39 ` Eric Wong
  0 siblings, 1 reply; 2+ messages in thread
From: Toshimaru Enomoto @ 2022-01-23 16:10 UTC (permalink / raw)
  To: unicorn-public

Hi,

After bumping unicorn from 6.0 to 6.1,
I observed it reduced memory usage much.

In production, the memory usage has been almost halved! :0
Startup memory usage is almost the same, but after a few minutes,
there is a big difference in the memory usage between v6.0 and v6.1.

I looked into the cause of this and I found using epoll reduces memory usage.
https://gist.github.com/toshimaru/c33cfdb2e1b2540ff0481092e8c36745

Does anyone observe the same enhancement?
Or does anyone have an idea why it happened?

Thanks,
Toshi

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: Memory usage enhancement in unicorn 6.1?
  2022-01-23 16:10 Memory usage enhancement in unicorn 6.1? Toshimaru Enomoto
@ 2022-02-16 22:39 ` Eric Wong
  0 siblings, 0 replies; 2+ messages in thread
From: Eric Wong @ 2022-02-16 22:39 UTC (permalink / raw)
  To: Toshimaru Enomoto; +Cc: unicorn-public

Toshimaru Enomoto <me@toshimaru.net> wrote:
> Hi,
> 
> After bumping unicorn from 6.0 to 6.1,
> I observed it reduced memory usage much.
> 
> In production, the memory usage has been almost halved! :0
> Startup memory usage is almost the same, but after a few minutes,
> there is a big difference in the memory usage between v6.0 and v6.1.
> 
> I looked into the cause of this and I found using epoll reduces memory usage.
> https://gist.github.com/toshimaru/c33cfdb2e1b2540ff0481092e8c36745

Quoting the lynx dump of above gist:
> Environment
> 
>      * Ruby 3.1 (ruby:3.1 docker image)
>      * unicorn 6.1
>      * Rails 6.1
> 
> Workload
> 
>      * Run unicorn with 4 worker processes
>      * Request 100 times to unicorn app
> 
> unicorn 6.1 without epoll
> 
>   def prep_readers(readers)
> -   wtr = Unicorn::Waiter.prep_readers(readers)
> -   @timeout *= 500 # to milliseconds for epoll, but halved
> -   wtr
> - rescue
>     require_relative 'select_waiter'
>     @timeout /= 2.0 # halved for IO.select
>     Unicorn::SelectWaiter.new
>   end
> 
> # free -h # before 100 requests
>                total        used        free      shared  buff/cache   available
> Mem:           1.9Gi       805Mi       169Mi       199Mi       1.0Gi       797Mi
> Swap:          1.0Gi       155Mi       868Mi
> 
> # free -h # after 100 requests
>                total        used        free      shared  buff/cache   available
> Mem:           1.9Gi       943Mi        85Mi       192Mi       956Mi       670Mi
> Swap:          1.0Gi       163Mi       860Mi
> 
> unicorn 6.1 with epoll
> 
>   def prep_readers(readers)
>     wtr = Unicorn::Waiter.prep_readers(readers)
>     @timeout *= 500 # to milliseconds for epoll, but halved
>     wtr
> - rescue
> -   require_relative 'select_waiter'
> -   @timeout /= 2.0 # halved for IO.select
> -   Unicorn::SelectWaiter.new
>   end
> 
> # free -h # before 100 requests
>                total        used        free      shared  buff/cache   available
> Mem:           1.9Gi       802Mi       160Mi       192Mi       1.0Gi       811Mi
> Swap:          1.0Gi       163Mi       860Mi
> # free -h # after 100 requests
>                total        used        free      shared  buff/cache   available
> Mem:           1.9Gi       805Mi       156Mi       192Mi       1.0Gi       808Mi
> Swap:          1.0Gi       163Mi       860Mi
> 
> Does anyone observe the same enhancement?

I haven't seen it, nor is it expected...
Note: free(1) may be misleading if other processes are active.

What do the VSZ and RSS columns of "ps aux" say?

> Or does anyone have an idea why it happened?

Nobody else responded, so I guess not.

How complex is your application w.r.t memory allocations?  The
epoll code path may use fewer allocations and stack than
IO.select, but I figured it'd be lost in the noise since most
applications do a lot more allocations.

Btw, please don't use gist or similar services for small text
that can easily be inlined w/o HTML/JS overhead on websites.
Some users download and read mail offline; requiring extra network
round-trips decreases the likelyhood your messages will be read.
My mindset as a GNU/Linux hacker remains optimizing everything
for i486 users on dialup :)

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2022-02-16 22:39 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-23 16:10 Memory usage enhancement in unicorn 6.1? Toshimaru Enomoto
2022-02-16 22:39 ` Eric Wong

Code repositories for project(s) associated with this public inbox

	https://yhbt.net/unicorn.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).