From: Nitesh Narayan Lal <nitesh@redhat.com>
To: Marcelo Tosatti <mtosatti@redhat.com>,
Thomas Gleixner <tglx@linutronix.de>
Cc: Robin Murphy <robin.murphy@arm.com>,
linux-kernel@vger.kernel.org, linux-api@vger.kernel.org,
frederic@kernel.org, juri.lelli@redhat.com, abelits@marvell.com,
bhelgaas@google.com, linux-pci@vger.kernel.org,
rostedt@goodmis.org, mingo@kernel.org, peterz@infradead.org,
davem@davemloft.net, akpm@linux-foundation.org,
sfr@canb.auug.org.au, stephen@networkplumber.org,
rppt@linux.vnet.ibm.com, jinyuqi@huawei.com,
zhangshaokun@hisilicon.com
Subject: Re: [Patch v4 1/3] lib: Restrict cpumask_local_spread to houskeeping CPUs
Date: Thu, 28 Jan 2021 12:35:35 -0500 [thread overview]
Message-ID: <3f138a52-7e6a-2c40-23f7-33515893a178@redhat.com> (raw)
In-Reply-To: <20210128165903.GB38339@fuller.cnet>
On 1/28/21 11:59 AM, Marcelo Tosatti wrote:
> On Thu, Jan 28, 2021 at 05:02:41PM +0100, Thomas Gleixner wrote:
>> On Wed, Jan 27 2021 at 09:19, Marcelo Tosatti wrote:
>>> On Wed, Jan 27, 2021 at 11:57:16AM +0000, Robin Murphy wrote:
>>>>> + hk_flags = HK_FLAG_DOMAIN | HK_FLAG_MANAGED_IRQ;
>>>>> + mask = housekeeping_cpumask(hk_flags);
>>>> AFAICS, this generally resolves to something based on cpu_possible_mask
>>>> rather than cpu_online_mask as before, so could now potentially return an
>>>> offline CPU. Was that an intentional change?
>>> Robin,
>>>
>>> AFAICS online CPUs should be filtered.
>> The whole pile wants to be reverted. It's simply broken in several ways.
> I was asking for your comments on interaction with CPU hotplug :-)
> Anyway...
>
> So housekeeping_cpumask has multiple meanings. In this case:
>
> HK_FLAG_DOMAIN | HK_FLAG_MANAGED_IRQ
>
> domain
> Isolate from the general SMP balancing and scheduling
> algorithms. Note that performing domain isolation this way
> is irreversible: it's not possible to bring back a CPU to
> the domains once isolated through isolcpus. It's strongly
> advised to use cpusets instead to disable scheduler load
> balancing through the "cpuset.sched_load_balance" file.
> It offers a much more flexible interface where CPUs can
> move in and out of an isolated set anytime.
>
> You can move a process onto or off an "isolated" CPU via
> the CPU affinity syscalls or cpuset.
> <cpu number> begins at 0 and the maximum value is
> "number of CPUs in system - 1".
>
> managed_irq
>
> Isolate from being targeted by managed interrupts
> which have an interrupt mask containing isolated
> CPUs. The affinity of managed interrupts is
> handled by the kernel and cannot be changed via
> the /proc/irq/* interfaces.
>
> This isolation is best effort and only effective
> if the automatically assigned interrupt mask of a
> device queue contains isolated and housekeeping
> CPUs. If housekeeping CPUs are online then such
> interrupts are directed to the housekeeping CPU
> so that IO submitted on the housekeeping CPU
> cannot disturb the isolated CPU.
>
> If a queue's affinity mask contains only isolated
> CPUs then this parameter has no effect on the
> interrupt routing decision, though interrupts are
> only delivered when tasks running on those
> isolated CPUs submit IO. IO submitted on
> housekeeping CPUs has no influence on those
> queues.
>
> So as long as the meaning of the flags are respected, seems
> alright.
>
> Nitesh, is there anything preventing this from being fixed
> in userspace ? (as Thomas suggested previously).
I think it should be doable atleast for most of the devices.
However, I do wonder if there is a better way of fixing this generically
from the kernel?
Currently, as Thomas mentioned housekeeping_cpumask() is used at different
locations just to fix the issue corresponding to that component or driver.
--
Thanks
Nitesh
next prev parent reply other threads:[~2021-01-28 17:39 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-25 22:34 [PATCH v4 0/3] Preventing job distribution to isolated CPUs Nitesh Narayan Lal
2020-06-25 22:34 ` [Patch v4 1/3] lib: Restrict cpumask_local_spread to houskeeping CPUs Nitesh Narayan Lal
2020-06-29 16:11 ` Nitesh Narayan Lal
2020-07-01 0:32 ` Andrew Morton
2020-07-01 0:47 ` Nitesh Narayan Lal
2020-07-09 8:45 ` [tip: sched/core] " tip-bot2 for Alex Belits
2021-01-27 11:57 ` [Patch v4 1/3] " Robin Murphy
2021-01-27 12:19 ` Marcelo Tosatti
2021-01-27 12:36 ` Robin Murphy
2021-01-27 13:09 ` Marcelo Tosatti
2021-01-27 13:49 ` Robin Murphy
2021-01-27 14:16 ` Nitesh Narayan Lal
2021-01-28 15:56 ` Thomas Gleixner
2021-01-28 16:33 ` Marcelo Tosatti
[not found] ` <02ac9d85-7ddd-96da-1252-4663feea7c9f@marvell.com>
2021-02-01 17:50 ` [EXT] " Marcelo Tosatti
2021-01-28 16:02 ` Thomas Gleixner
2021-01-28 16:59 ` Marcelo Tosatti
2021-01-28 17:35 ` Nitesh Narayan Lal [this message]
2021-01-28 20:01 ` Thomas Gleixner
[not found] ` <d2a4dc97-a9ed-e0e7-3b9c-c56ae46f6608@redhat.com>
[not found] ` <20210129142356.GB40876@fuller.cnet>
2021-01-29 17:34 ` [EXT] " Alex Belits
[not found] ` <18584612-868c-0f88-5de2-dc93c8638816@redhat.com>
2021-02-05 19:56 ` Thomas Gleixner
2021-02-04 18:15 ` Marcelo Tosatti
2021-02-04 18:47 ` Nitesh Narayan Lal
2021-02-04 19:06 ` Marcelo Tosatti
2021-02-04 19:17 ` Nitesh Narayan Lal
2021-02-05 22:23 ` Thomas Gleixner
2021-02-05 22:26 ` Thomas Gleixner
2021-02-05 23:02 ` [tip: sched/urgent] Revert "lib: Restrict cpumask_local_spread to houskeeping CPUs" tip-bot2 for Thomas Gleixner
2021-02-07 0:43 ` [Patch v4 1/3] lib: Restrict cpumask_local_spread to houskeeping CPUs Nitesh Narayan Lal
2021-02-11 15:55 ` Nitesh Narayan Lal
2021-03-04 18:15 ` Nitesh Narayan Lal
[not found] ` <faa8d84e-db67-7fbe-891e-f4987f106b20@marvell.com>
2021-03-04 23:23 ` [EXT] " Nitesh Narayan Lal
2021-04-06 17:22 ` Jesse Brandeburg
2021-04-07 15:18 ` Nitesh Narayan Lal
2021-04-08 18:49 ` Nitesh Narayan Lal
2021-04-14 16:11 ` Jesse Brandeburg
2021-04-15 22:11 ` Nitesh Narayan Lal
2021-04-29 21:44 ` Nitesh Lal
2021-04-30 1:48 ` Jesse Brandeburg
2021-04-30 13:10 ` Nitesh Lal
2021-04-30 7:10 ` Thomas Gleixner
2021-04-30 16:14 ` Nitesh Lal
2021-04-30 18:21 ` Thomas Gleixner
2021-04-30 21:07 ` Nitesh Lal
2021-05-01 2:21 ` Jesse Brandeburg
2021-05-03 13:15 ` Nitesh Lal
2020-06-25 22:34 ` [Patch v4 2/3] PCI: Restrict probe functions to housekeeping CPUs Nitesh Narayan Lal
2020-07-09 8:45 ` [tip: sched/core] " tip-bot2 for Alex Belits
2020-06-25 22:34 ` [Patch v4 3/3] net: Restrict receive packets queuing " Nitesh Narayan Lal
2020-06-26 11:14 ` Peter Zijlstra
2020-06-26 17:20 ` David Miller
2020-07-09 8:45 ` [tip: sched/core] " tip-bot2 for Alex Belits
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3f138a52-7e6a-2c40-23f7-33515893a178@redhat.com \
--to=nitesh@redhat.com \
--cc=abelits@marvell.com \
--cc=akpm@linux-foundation.org \
--cc=bhelgaas@google.com \
--cc=davem@davemloft.net \
--cc=frederic@kernel.org \
--cc=jinyuqi@huawei.com \
--cc=juri.lelli@redhat.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=mtosatti@redhat.com \
--cc=peterz@infradead.org \
--cc=robin.murphy@arm.com \
--cc=rostedt@goodmis.org \
--cc=rppt@linux.vnet.ibm.com \
--cc=sfr@canb.auug.org.au \
--cc=stephen@networkplumber.org \
--cc=tglx@linutronix.de \
--cc=zhangshaokun@hisilicon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).