From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS33070 50.56.128.0/17 X-Spam-Status: No, score=0.0 required=3.0 tests=AWL,MSGID_FROM_MTA_HEADER, TVD_RCVD_IP shortcircuit=no autolearn=unavailable version=3.3.2 Path: news.gmane.org!not-for-mail From: Claudio Poli Newsgroups: gmane.comp.lang.ruby.rainbows.general Subject: Re: How to manage growing memory with Rainbows! Date: Thu, 14 Feb 2013 09:49:16 +0100 Message-ID: References: <20130212050021.GA18443@dcvr.yhbt.net> <20130214071512.GA10890@dcvr.yhbt.net> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1360831768 2128 80.91.229.3 (14 Feb 2013 08:49:28 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 14 Feb 2013 08:49:28 +0000 (UTC) To: Rainbows! list Original-X-From: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Thu Feb 14 09:49:48 2013 Return-path: Envelope-to: gclrrg-rainbows-talk@m.gmane.org X-Original-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Delivered-To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:sender:content-type:mime-version:subject:from :in-reply-to:date:content-transfer-encoding:message-id:references:to :x-mailer:x-gm-message-state; bh=4PbYkrp/7T2oIbZSWqVR5v4CCHcZvrPzLG4RHiPFnQ4=; b=YfJi+Zt+waW6vrnHn8x1rOFsVz3kyRHAbkTDql1jd0KrEzzW1sum5sKZ1sNx3w+Qc/ apRv9342G6PF/eDyDoI0UlYABBsYNHWEvL+BXMLowzWvkmHNL2KeNFIJ1FtOB6DlRSAe mbq+Mk9nTej9GRCheJ60myA38PRQWfnpMUIZwlBdkA/y940xGwDZtl1zp78IrDylVToz GirYPbMLE9BbesGvHQw3tlpHWnA4Hj6dBDrgaZ+NID/0THW244aQDoxOf7FCG1lO2/ei s6F4287aA9B/hjhCYwEHaWmqxspWGgBqYwMDi1NnNF8pJhTTh6GvmTKogFlanOqyNjOJ 35BA== X-Received: by 10.14.200.137 with SMTP id z9mr85304299een.20.1360831759721; Thu, 14 Feb 2013 00:49:19 -0800 (PST) In-Reply-To: <20130214071512.GA10890-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org> X-Mailer: Apple Mail (2.1499) X-Gm-Message-State: ALoCoQmOoQrYWwC0xcpaLUicvT5QIiead2y7NehCpubJhvZ9eYM85ATrLiN96Q+jRMvt7D0TM+zP X-BeenThere: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: rainbows-talk-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.rainbows.general:450 Archived-At: Received: from 50-56-192-79.static.cloud-ips.com ([50.56.192.79] helo=rubyforge.org) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1U5uVm-0006J2-2g for gclrrg-rainbows-talk@m.gmane.org; Thu, 14 Feb 2013 09:49:46 +0100 Received: from localhost.localdomain (localhost [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 890972E09A; Thu, 14 Feb 2013 08:49:26 +0000 (UTC) Received: from mail-ee0-f50.google.com (mail-ee0-f50.google.com [74.125.83.50]) by rubyforge.org (Postfix) with ESMTP id 9C40D2E069 for ; Thu, 14 Feb 2013 08:49:21 +0000 (UTC) Received: by mail-ee0-f50.google.com with SMTP id e51so1084052eek.9 for ; Thu, 14 Feb 2013 00:49:20 -0800 (PST) Received: from [192.168.5.103] ([151.64.208.92]) by mx.google.com with ESMTPS id h5sm77308310eem.1.2013.02.14.00.49.17 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 14 Feb 2013 00:49:18 -0800 (PST) Il giorno 14/feb/2013, alle ore 08:15, Eric Wong ha scritto: > > Last I checked, tcmalloc never releases memory to the OS, so that > could be a problem. (Giving memory back to the kernel and then > getting it back soon afterwards is slow because the kernel must > clear that memory, first, so most malloc implementations (including > glibc) will try to keep memory for the process) > > You might also want to try MALLOC_MMAP_THRESHOLD_=131072 to disable the > sliding sbrk/mmap window (you can try larger/smaller values). The > latest Linux man-pages releases have better mallopt(3) documentation > (but the arenas stuff was only documented on Dreppers blog, AFAIK (and > glibc.git/malloc/*.[ch] comments) Thanks for the advices Eric, will try. I have few more questions before definitely make the switch if you don't mind. ActiveRecord/Redis. This is the configuration I'm trying out in staging: http://pastie.org/private/zepmdeduvlrvtdkc32unnq Is there something special that should I know about AR connection pool and Rainbows! + Threadpool? My pool in database.yml is 30. After a small load test currently I'm sitting around 60/80 connections, which is kinda expected given that also Sidekiq is in action. I don't like the global $redis connection. Can you, or anyone else, offer any advice how to improve that, if you use it in your projects? The goal was to keep a connection alive in a single process due to the sheer number of commands that can be put in redis as consequence of actions, but now that I'm using threads it probably needs to be changed. The cost however for opening a new connection seems high, I was reading this: https://groups.google.com/forum/?fromgroups=#!topic/redis-db/xcz5MXykXdk Thanks for the help, Claudio _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying