From: Chetan Loke <generationgnu@yahoo.com>
To: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Cc: rick.sherm@yahoo.com, andi@firstfloor.org,
linux-numa@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: RE: Memory policy question for NUMA arch....
Date: Tue, 20 Apr 2010 13:34:10 -0700 (PDT) [thread overview]
Message-ID: <301842.27948.qm@web111913.mail.gq1.yahoo.com> (raw)
Lee,
--- On Mon, 4/19/10, Lee Schermerhorn <Lee.Schermerhorn@hp.com> wrote:
> You should never change the system default once the system
> is up and
> running.
>
Not sure what I was thinking.Agreed, we should leave the default-policy alone.
> Still not clear on what your requirements are
Thanks for pointing me to the other post. I might have a similar problem.I've a nehalem box.
1) Drivers supports MSI-X.
1.1) Drivers at load time allocate a chunk of DMA'able memory.
2) Sometime later, after the OS boots, I need to load my apps.
3) Now, the apps and the drivers communicate via a mmap'd region.
I need a deterministic way of allocating memory depending on my needs(interleave or localalloc).So I can't be at the mercy of either 'current' or 'global-policy' or anyone else.Also, why reference 'current' to begin with? And everytime I reference 'current' from within my driver, current points to 'work_for_cpu' kthread. So that clearly doesn't help.
> A device-centric interface--e.g.,
> 'get_free_pages_dev()'--could get the
> device/bus node affinity via dev_to_node() and then do the
> allocation/conversion. I think this is
> close to what you're suggesting
> above. See dma_generic_alloc_coherent() [in
> arch/x86/kernel/pci-dma.c]
> for an example of a wrapper that does the device affinity
> lookup and
> allocation in one function.
>
> Of course, you could just do this in your driver, as well.
>
Very helpful thanks.I will mimic 'dma_generic_alloc_coherent' in my driver when I need local-node memory.
> > Also, is there a way to configure irqbalance and ask
> it to leave these
> > guys alone? Like a config file that says - leave
> these
> > irqs/pci-devices alone.For now I've shut down
> irqbalance.
>
> You can set the environment variable
> IRQBALANCE_BANNED_INTERRUPTS--when
> starting irqbalance--to list of interrupts that irqbalance
> should ignore
> if you're using a version that supports that. Check
> the init script
> that starts irqbalance on your distro of choice.
>
aaaah...mine is old and I could see IRQBALANCE_BANNED_CPUS and not _INTERRUPTS.I will upgrade it.
> Regards,
> Lee
Thanks
Chetan Loke
next reply other threads:[~2010-04-20 20:34 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-04-20 20:34 Chetan Loke [this message]
[not found] <243351.5510.qm@web111910.mail.gq1.yahoo.com>
2010-04-16 23:17 ` Memory policy question for NUMA arch Chetan Loke
2010-04-17 6:35 ` Andi Kleen
2010-04-17 14:59 ` Chetan Loke
2010-04-17 19:30 ` Andi Kleen
2010-04-19 15:16 ` Lee Schermerhorn
-- strict thread matches above, loose matches on Subject: below --
2010-04-06 20:46 Rick Sherm
2010-04-07 9:00 ` Andi Kleen
2010-04-07 15:48 ` Rick Sherm
2010-04-07 17:27 ` Lee Schermerhorn
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=301842.27948.qm@web111913.mail.gq1.yahoo.com \
--to=generationgnu@yahoo.com \
--cc=Lee.Schermerhorn@hp.com \
--cc=andi@firstfloor.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-numa@vger.kernel.org \
--cc=rick.sherm@yahoo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).