From: Jesse Brandeburg <jesse.brandeburg@intel.com>
To: Nitesh Narayan Lal <nitesh@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>,
Marcelo Tosatti <mtosatti@redhat.com>,
Robin Murphy <robin.murphy@arm.com>,
linux-kernel@vger.kernel.org, linux-api@vger.kernel.org,
frederic@kernel.org, juri.lelli@redhat.com, abelits@marvell.com,
bhelgaas@google.com, linux-pci@vger.kernel.org,
rostedt@goodmis.org, mingo@kernel.org, peterz@infradead.org,
davem@davemloft.net, akpm@linux-foundation.org,
sfr@canb.auug.org.au, stephen@networkplumber.org,
rppt@linux.vnet.ibm.com, jinyuqi@huawei.com,
zhangshaokun@hisilicon.com
Subject: Re: [Patch v4 1/3] lib: Restrict cpumask_local_spread to houskeeping CPUs
Date: Tue, 6 Apr 2021 10:22:07 -0700 [thread overview]
Message-ID: <20210406102207.0000485c@intel.com> (raw)
In-Reply-To: <07c04bc7-27f0-9c07-9f9e-2d1a450714ef@redhat.com>
Continuing a thread from a bit ago...
Nitesh Narayan Lal wrote:
> > After a little more digging, I found out why cpumask_local_spread change
> > affects the general/initial smp_affinity for certain device IRQs.
> >
> > After the introduction of the commit:
> >
> > e2e64a932 genirq: Set initial affinity in irq_set_affinity_hint()
> >
>
> Continuing the conversation about the above commit and adding Jesse.
> I was trying to understand the problem that the commit message explains
> "The default behavior of the kernel is somewhat undesirable as all
> requested interrupts end up on CPU0 after registration.", I have also been
> trying to reproduce this behavior without the patch but I failed in doing
> so, maybe because I am missing something here.
>
> @Jesse Can you please explain? FWIU IRQ affinity should be decided based on
> the default affinity mask.
The original issue as seen, was that if you rmmod/insmod a driver
*without* irqbalance running, the default irq mask is -1, which means
any CPU. The older kernels (this issue was patched in 2014) used to use
that affinity mask, but the value programmed into all the interrupt
registers "actual affinity" would end up delivering all interrupts to
CPU0, and if the machine was under traffic load incoming when the
driver loaded, CPU0 would start to poll among all the different netdev
queues, all on CPU0.
The above then leads to the condition that the device is stuck polling
even if the affinity gets updated from user space, and the polling will
continue until traffic stops.
> The problem with the commit is that when we overwrite the affinity mask
> based on the hinting mask we completely ignore the default SMP affinity
> mask. If we do want to overwrite the affinity based on the hint mask we
> should atleast consider the default SMP affinity.
Maybe the right thing is to fix which CPUs are passed in as the valid
mask, or make sure the kernel cross checks that what the driver asks
for is a "valid CPU"?
next prev parent reply other threads:[~2021-04-06 17:22 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-25 22:34 [PATCH v4 0/3] Preventing job distribution to isolated CPUs Nitesh Narayan Lal
2020-06-25 22:34 ` [Patch v4 1/3] lib: Restrict cpumask_local_spread to houskeeping CPUs Nitesh Narayan Lal
2020-06-29 16:11 ` Nitesh Narayan Lal
2020-07-01 0:32 ` Andrew Morton
2020-07-01 0:47 ` Nitesh Narayan Lal
2020-07-09 8:45 ` [tip: sched/core] " tip-bot2 for Alex Belits
2021-01-27 11:57 ` [Patch v4 1/3] " Robin Murphy
2021-01-27 12:19 ` Marcelo Tosatti
2021-01-27 12:36 ` Robin Murphy
2021-01-27 13:09 ` Marcelo Tosatti
2021-01-27 13:49 ` Robin Murphy
2021-01-27 14:16 ` Nitesh Narayan Lal
2021-01-28 15:56 ` Thomas Gleixner
2021-01-28 16:33 ` Marcelo Tosatti
[not found] ` <02ac9d85-7ddd-96da-1252-4663feea7c9f@marvell.com>
2021-02-01 17:50 ` [EXT] " Marcelo Tosatti
2021-01-28 16:02 ` Thomas Gleixner
2021-01-28 16:59 ` Marcelo Tosatti
2021-01-28 17:35 ` Nitesh Narayan Lal
2021-01-28 20:01 ` Thomas Gleixner
[not found] ` <d2a4dc97-a9ed-e0e7-3b9c-c56ae46f6608@redhat.com>
[not found] ` <20210129142356.GB40876@fuller.cnet>
2021-01-29 17:34 ` [EXT] " Alex Belits
[not found] ` <18584612-868c-0f88-5de2-dc93c8638816@redhat.com>
2021-02-05 19:56 ` Thomas Gleixner
2021-02-04 18:15 ` Marcelo Tosatti
2021-02-04 18:47 ` Nitesh Narayan Lal
2021-02-04 19:06 ` Marcelo Tosatti
2021-02-04 19:17 ` Nitesh Narayan Lal
2021-02-05 22:23 ` Thomas Gleixner
2021-02-05 22:26 ` Thomas Gleixner
2021-02-05 23:02 ` [tip: sched/urgent] Revert "lib: Restrict cpumask_local_spread to houskeeping CPUs" tip-bot2 for Thomas Gleixner
2021-02-07 0:43 ` [Patch v4 1/3] lib: Restrict cpumask_local_spread to houskeeping CPUs Nitesh Narayan Lal
2021-02-11 15:55 ` Nitesh Narayan Lal
2021-03-04 18:15 ` Nitesh Narayan Lal
[not found] ` <faa8d84e-db67-7fbe-891e-f4987f106b20@marvell.com>
2021-03-04 23:23 ` [EXT] " Nitesh Narayan Lal
2021-04-06 17:22 ` Jesse Brandeburg [this message]
2021-04-07 15:18 ` Nitesh Narayan Lal
2021-04-08 18:49 ` Nitesh Narayan Lal
2021-04-14 16:11 ` Jesse Brandeburg
2021-04-15 22:11 ` Nitesh Narayan Lal
2021-04-29 21:44 ` Nitesh Lal
2021-04-30 1:48 ` Jesse Brandeburg
2021-04-30 13:10 ` Nitesh Lal
2021-04-30 7:10 ` Thomas Gleixner
2021-04-30 16:14 ` Nitesh Lal
2021-04-30 18:21 ` Thomas Gleixner
2021-04-30 21:07 ` Nitesh Lal
2021-05-01 2:21 ` Jesse Brandeburg
2021-05-03 13:15 ` Nitesh Lal
2020-06-25 22:34 ` [Patch v4 2/3] PCI: Restrict probe functions to housekeeping CPUs Nitesh Narayan Lal
2020-07-09 8:45 ` [tip: sched/core] " tip-bot2 for Alex Belits
2020-06-25 22:34 ` [Patch v4 3/3] net: Restrict receive packets queuing " Nitesh Narayan Lal
2020-06-26 11:14 ` Peter Zijlstra
2020-06-26 17:20 ` David Miller
2020-07-09 8:45 ` [tip: sched/core] " tip-bot2 for Alex Belits
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210406102207.0000485c@intel.com \
--to=jesse.brandeburg@intel.com \
--cc=abelits@marvell.com \
--cc=akpm@linux-foundation.org \
--cc=bhelgaas@google.com \
--cc=davem@davemloft.net \
--cc=frederic@kernel.org \
--cc=jinyuqi@huawei.com \
--cc=juri.lelli@redhat.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=mtosatti@redhat.com \
--cc=nitesh@redhat.com \
--cc=peterz@infradead.org \
--cc=robin.murphy@arm.com \
--cc=rostedt@goodmis.org \
--cc=rppt@linux.vnet.ibm.com \
--cc=sfr@canb.auug.org.au \
--cc=stephen@networkplumber.org \
--cc=tglx@linutronix.de \
--cc=zhangshaokun@hisilicon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).