Re: Unicorn 3.3.1 "Too many open files" error in kgio_tryaccept

[Eric Wong <normalperson@yhbt.net>]

ghazel@gmail.com wrote:
> On Thu, Jan 13, 2011 at 6:47 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > ghazel@gmail.com wrote:
> >> On Thu, Jan 13, 2011 at 3:06 PM, Eric Wong <normalperson@yhbt.net> wrote:
> >> > ghazel@gmail.com wrote:
> >> > How does lsof output look for your workers?
> >>
> >> Hm. The workers seem to be at 80-90 file descriptors each. I did catch
> >> one at 787 (!) with mostly handles to a geoip database from the geoip
> >> gem, but they got collected pretty quickly. Perhaps that's the cause!
> >
> > OK, that's a fairly likely cause of EMFILE.
> >
> > A tip for geoip users:
> >
> >  Install the io-extra gem to get IO.pread.  This allows you to reuse
> >  the same file descriptor with geoip automatically between any number
> >  of threads/processes without reopening it.
> 
> Fascinating. Google tells be you've been down this road before. My
> code previously was doing:
> def get_city(ip)
>   GeoIP.new("/path/to/geo.dat").city(ip)
> end
> 
> Which seems to create one fd per call (and leave it for the GC to
> cleanup). What's the new proposed interface if io-extra is installed?
> Keep a global GeoIP object somewhere? My fix was to stuff it in to
> Thread.current, but obviously that has one fd per thread.

Yes, with io-extra, just define a constant somewhere and use it in any
thread/process at any time.  No need to deal with locks or after_fork
hooks in Unicorn at all, either, pread() is just that awesome :)

    GEO_DB = GeoIP.new("/path/to/geo.dat")

    def get_city(ip)
      GEO_DB.city(ip)
    end

-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying