Re: after_fork and redis

[Eric Wong <normalperson@yhbt.net>]

bradford <fingermark@gmail.com> wrote:
> I'm using unicorn w/ a rails app.  I have the following in my
> environment.rb $redis = MyApplication::Application.config.redis and in
> production.rb I have config.redis = Redis.new(host: "localhost").
> 
> I've read I'm supposed to $redis = Redis.new(host: "localhost") in
> after_fork when preload_app is true.
> 
> When I don't do this, each worker/pid seems to have their own redis
> instance.  So, why is this needed?  Here's the logs of me printing out
> $redis.client.inspect when both $redis = Redis.new in the after_fork
> and just $redis = Redis.new in the environment.rb.

(Disclaimer: I still haven't used Redis, but similar knowledge applies
to every TCP-based service)

If a Redis client opens a TCP (or any stream) socket connection before
forking, all the children that are forked will share that _same_ local
TCP socket.  Sharing stream sockets across processes/threads is nearly
always a bad idea: data streams become interleaved and impossible to
separate in either direction.

> No after fork
> https://gist.github.com/bc2c2a3bda01c35730e2
> 
> After fork
> https://gist.github.com/0c96550660d3926ffe16
> 
> The only thing different I notice is Connection::TCPSocket:fd is
> always 13 w/ the after fork.

FD shouldn't really matter.  FDs get recycled ASAP once they're closed
(and GC can automatically close them).  What you want is a different
local port for the TCPSocket on every worker process, and the after_fork
hook will give you that.

Btw, you can demonstrate FD recycling with:

  require 'socket'
  loop do
    c = TCPSocket.new("example.com", 80)
    p [ :fileno, c.fileno ]
    p [ :addr, c.addr ] # (local address)
    c.close
    sleep 1 # be nice to the remote server
  end

You'll see the same FD recycled over and over, but c.addr will
have a different local port.

You can see if after_fork is working correctly by checking the output of
tools like `ss' or `netstat':
(something like: ss -tan | grep -F :$PORT_OF_REDIS)

You can also check connections on the Redis server/process itself
using "lsof -p $PID_OF_REDIS_DAEMON"
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying