From: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
To: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>,
x86 Maintainers <x86@kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
Linux PM <linux-pm@vger.kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Peter Zijlstra <peterz@infradead.org>,
Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>,
"Rafael J. Wysocki" <rafael@kernel.org>,
Ricardo Neri <ricardo.neri@intel.com>,
Tim Chen <tim.c.chen@intel.com>
Subject: Re: [RFC][PATCH v1 3/3] cpufreq: intel_pstate: Set asymmetric CPU capacity on hybrid systems
Date: Thu, 2 May 2024 20:32:42 -0700 [thread overview]
Message-ID: <20240503033242.GA14835@ranerica-svr.sc.intel.com> (raw)
In-Reply-To: <050c561c-487e-4e89-a7b2-9752cebc9f46@arm.com>
On Thu, May 02, 2024 at 12:42:54PM +0200, Dietmar Eggemann wrote:
> On 25/04/2024 21:06, Rafael J. Wysocki wrote:
> > From: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> >
> > Make intel_pstate use the HWP_HIGHEST_PERF values from
> > MSR_HWP_CAPABILITIES to set asymmetric CPU capacity information
> > via the previously introduced arch_set_cpu_capacity() on hybrid
> > systems without SMT.
>
> Are there such systems around? My i7-13700K has P-cores (CPU0..CPU15)
> with SMT.
We have been experimenting with nosmt in the kernel command line.
>
> > Setting asymmetric CPU capacity is generally necessary to allow the
> > scheduler to compute task sizes in a consistent way across all CPUs
> > in a system where they differ by capacity. That, in turn, should help
> > to improve task placement and load balancing decisions. It is also
> > necessary for the schedutil cpufreq governor to operate as expected
> > on hybrid systems where tasks migrate between CPUs of different
> > capacities.
> >
> > The underlying observation is that intel_pstate already uses
> > MSR_HWP_CAPABILITIES to get CPU performance information which is
> > exposed by it via sysfs and CPU performance scaling is based on it.
> > Thus using this information for setting asymmetric CPU capacity is
> > consistent with what the driver has been doing already. Moreover,
> > HWP_HIGHEST_PERF reflects the maximum capacity of a given CPU including
> > both the instructions-per-cycle (IPC) factor and the maximum turbo
> > frequency and the units in which that value is expressed are the same
> > for all CPUs in the system, so the maximum capacity ratio between two
> > CPUs can be obtained by computing the ratio of their HWP_HIGHEST_PERF
> > values. Of course, in principle that capacity ratio need not be
> > directly applicable at lower frequencies, so using it for providing the
> > asymmetric CPU capacity information to the scheduler is a rough
> > approximation, but it is as good as it gets. Also, measurements
> > indicate that this approximation is not too bad in practice.
>
> So cpu_capacity has a direct mapping to itmt prio. cpu_capacity is itmt
> prio with max itmt prio scaled to 1024.
ITMT enables asym_packing in the load balancer. Since it only cares about
which CPU has higher priority, scaling to 1024 is not necessary.
>
> Running it on i7-13700K (while allowing SMT) gives:
>
> root@gulliver:~# dmesg | grep sched_set_itmt_core_prio
> [ 3.957826] sched_set_itmt_core_prio() cpu=0 prio=68
> [ 3.990401] sched_set_itmt_core_prio() cpu=1 prio=68
> [ 4.015551] sched_set_itmt_core_prio() cpu=2 prio=68
> [ 4.040720] sched_set_itmt_core_prio() cpu=3 prio=68
> [ 4.065871] sched_set_itmt_core_prio() cpu=4 prio=68
> [ 4.091018] sched_set_itmt_core_prio() cpu=5 prio=68
> [ 4.116175] sched_set_itmt_core_prio() cpu=6 prio=68
> [ 4.141374] sched_set_itmt_core_prio() cpu=7 prio=68
> [ 4.166543] sched_set_itmt_core_prio() cpu=8 prio=69
> [ 4.196289] sched_set_itmt_core_prio() cpu=9 prio=69
> [ 4.214964] sched_set_itmt_core_prio() cpu=10 prio=69
> [ 4.239281] sched_set_itmt_core_prio() cpu=11 prio=69
> [ 4.263438] sched_set_itmt_core_prio() cpu=12 prio=68
> [ 4.283790] sched_set_itmt_core_prio() cpu=13 prio=68
> [ 4.308905] sched_set_itmt_core_prio() cpu=14 prio=68
> [ 4.331751] sched_set_itmt_core_prio() cpu=15 prio=68
> [ 4.356002] sched_set_itmt_core_prio() cpu=16 prio=42
> [ 4.381639] sched_set_itmt_core_prio() cpu=17 prio=42
> [ 4.395175] sched_set_itmt_core_prio() cpu=18 prio=42
> [ 4.425625] sched_set_itmt_core_prio() cpu=19 prio=42
> [ 4.449670] sched_set_itmt_core_prio() cpu=20 prio=42
> [ 4.479681] sched_set_itmt_core_prio() cpu=21 prio=42
> [ 4.506319] sched_set_itmt_core_prio() cpu=22 prio=42
> [ 4.523774] sched_set_itmt_core_prio() cpu=23 prio=42
>
> root@gulliver:~# dmesg | grep hybrid_set_cpu_capacity
> [ 4.450883] hybrid_set_cpu_capacity() cpu=0 cap=1009
> [ 4.455846] hybrid_set_cpu_capacity() cpu=1 cap=1009
> [ 4.460806] hybrid_set_cpu_capacity() cpu=2 cap=1009
> [ 4.465766] hybrid_set_cpu_capacity() cpu=3 cap=1009
> [ 4.470730] hybrid_set_cpu_capacity() cpu=4 cap=1009
> [ 4.475699] hybrid_set_cpu_capacity() cpu=5 cap=1009
> [ 4.480664] hybrid_set_cpu_capacity() cpu=6 cap=1009
> [ 4.485626] hybrid_set_cpu_capacity() cpu=7 cap=1009
> [ 4.490588] hybrid_set_cpu_capacity() cpu=9 cap=1024
> [ 4.495550] hybrid_set_cpu_capacity() cpu=10 cap=1024
> [ 4.500598] hybrid_set_cpu_capacity() cpu=11 cap=1024
> [ 4.505649] hybrid_set_cpu_capacity() cpu=12 cap=1009
> [ 4.510701] hybrid_set_cpu_capacity() cpu=13 cap=1009
> [ 4.515749] hybrid_set_cpu_capacity() cpu=14 cap=1009
> [ 4.520802] hybrid_set_cpu_capacity() cpu=15 cap=1009
> [ 4.525846] hybrid_set_cpu_capacity() cpu=16 cap=623
> [ 4.530810] hybrid_set_cpu_capacity() cpu=17 cap=623
> [ 4.535772] hybrid_set_cpu_capacity() cpu=18 cap=623
> [ 4.540732] hybrid_set_cpu_capacity() cpu=19 cap=623
> [ 4.545690] hybrid_set_cpu_capacity() cpu=20 cap=623
> [ 4.550651] hybrid_set_cpu_capacity() cpu=21 cap=623
> [ 4.555612] hybrid_set_cpu_capacity() cpu=22 cap=623
> [ 4.560571] hybrid_set_cpu_capacity() cpu=23 cap=623
>
> > If the given system is hybrid and non-SMT, the new code disables ITMT
> > support in the scheduler (because it may get in the way of asymmetric CPU
> > capacity code in the scheduler that automatically gets enabled by setting
> > asymmetric CPU capacity) after initializing all online CPUs and finds
> > the one with the maximum HWP_HIGHEST_PERF value. Next, it computes the
> > capacity number for each (online) CPU by dividing the product of its
> > HWP_HIGHEST_PERF and SCHED_CAPACITY_SCALE by the maximum HWP_HIGHEST_PERF.
>
> SO either CAS at wakeup and in load_balance or SIS at wakeup and ITMT in
> load balance.
May I know what CAS and SIS stand for?
Thanks and BR,
Ricardo
next prev parent reply other threads:[~2024-05-03 3:25 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-25 19:03 [RFC][PATCH v1 0/3] x86 / intel_pstate: Set asymmetric CPU capacity on hybrid systems Rafael J. Wysocki
2024-04-25 19:04 ` [RFC][PATCH v1 1/3] x86/sched: Introduce arch_rebuild_sched_domains() Rafael J. Wysocki
2024-04-25 19:04 ` [RFC][PATCH v1 2/3] " Rafael J. Wysocki
2024-04-25 19:14 ` Rafael J. Wysocki
2024-04-25 19:15 ` [RFC][PATCH v1 2/3] x86/sched: Add basic support for CPU capacity scaling Rafael J. Wysocki
2024-04-25 19:06 ` [RFC][PATCH v1 3/3] cpufreq: intel_pstate: Set asymmetric CPU capacity on hybrid systems Rafael J. Wysocki
2024-05-02 10:42 ` Dietmar Eggemann
2024-05-03 3:32 ` Ricardo Neri [this message]
2024-05-03 18:29 ` Dietmar Eggemann
2024-05-06 14:39 ` Rafael J. Wysocki
2024-05-21 12:51 ` Dietmar Eggemann
2024-05-24 12:43 ` Dietmar Eggemann
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240503033242.GA14835@ranerica-svr.sc.intel.com \
--to=ricardo.neri-calderon@linux.intel.com \
--cc=dietmar.eggemann@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=rafael@kernel.org \
--cc=ricardo.neri@intel.com \
--cc=rjw@rjwysocki.net \
--cc=srinivas.pandruvada@linux.intel.com \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@intel.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).