From: "Michael Kelley (LINUX)" <mikelley@microsoft.com>
To: Thomas Gleixner <tglx@linutronix.de>,
LKML <linux-kernel@vger.kernel.org>
Cc: "x86@kernel.org" <x86@kernel.org>,
David Woodhouse <dwmw2@infradead.org>,
Andrew Cooper <andrew.cooper3@citrix.com>,
Brian Gerst <brgerst@gmail.com>,
Arjan van de Veen <arjan@linux.intel.com>,
Paolo Bonzini <pbonzini@redhat.com>,
Paul McKenney <paulmck@kernel.org>,
Tom Lendacky <thomas.lendacky@amd.com>,
Sean Christopherson <seanjc@google.com>,
Oleksandr Natalenko <oleksandr@natalenko.name>,
Paul Menzel <pmenzel@molgen.mpg.de>,
"Guilherme G. Piccoli" <gpiccoli@igalia.com>,
Piotr Gorski <lucjan.lucjanov@gmail.com>,
Usama Arif <usama.arif@bytedance.com>,
Juergen Gross <jgross@suse.com>,
Boris Ostrovsky <boris.ostrovsky@oracle.com>,
"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
Russell King <linux@armlinux.org.uk>,
Arnd Bergmann <arnd@arndb.de>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>, Guo Ren <guoren@kernel.org>,
"linux-csky@vger.kernel.org" <linux-csky@vger.kernel.org>,
Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
"linux-mips@vger.kernel.org" <linux-mips@vger.kernel.org>,
"James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>,
Helge Deller <deller@gmx.de>,
"linux-parisc@vger.kernel.org" <linux-parisc@vger.kernel.org>,
Paul Walmsley <paul.walmsley@sifive.com>,
Palmer Dabbelt <palmer@dabbelt.com>,
"linux-riscv@lists.infradead.org"
<linux-riscv@lists.infradead.org>,
Mark Rutland <Mark.Rutland@arm.com>,
Sabin Rapan <sabrapan@amazon.com>
Subject: RE: [patch V2 38/38] x86/smpboot/64: Implement arch_cpuhp_init_parallel_bringup() and enable it
Date: Sun, 7 May 2023 04:14:08 +0000 [thread overview]
Message-ID: <BYAPR21MB1688A75D614165C5E4E0302AD7709@BYAPR21MB1688.namprd21.prod.outlook.com> (raw)
In-Reply-To: <87sfc92zw2.ffs@tglx>
From: Thomas Gleixner <tglx@linutronix.de> Sent: Saturday, May 6, 2023 9:23 AM
>
> On Sat, May 06 2023 at 00:53, Michael Kelley wrote:
> > From: Thomas Gleixner <tglx@linutronix.de> Sent: Thursday, May 4, 2023 12:03 PM
> > [snip]
> >
> >> @@ -934,10 +961,10 @@ static void announce_cpu(int cpu, int ap
> >> if (!node_width)
> >> node_width = num_digits(num_possible_nodes()) + 1; /* + '#' */
> >>
> >> - if (cpu == 1)
> >> - printk(KERN_INFO "x86: Booting SMP configuration:\n");
> >> -
> >> if (system_state < SYSTEM_RUNNING) {
> >> + if (num_online_cpus() == 1)
> >
> > Unfortunately, this new check doesn't work. Here's the output I get:
> >
> > [ 0.721384] smp: Bringing up secondary CPUs ...
> > [ 0.725359] smpboot: x86: Booting SMP configuration:
> > [ 0.729249] .... node #0, CPUs: #2
> > [ 0.729654] smpboot: x86: Booting SMP configuration:
> > [ 0.737247] #4
> >
> > Evidently num_online_cpus() isn't updated until after all the primary
> > siblings get started.
>
> Duh. Where is that brown paperbag?
>
> > When booting with cpuhp.parallel=0, the output is good.
>
> Exactly that was on the command line when I quickly booted that change :(
>
> The below should fix it for real.
>
> Thanks,
>
> tglx
> ---
> --- a/arch/x86/kernel/smpboot.c
> +++ b/arch/x86/kernel/smpboot.c
> @@ -951,9 +951,9 @@ static int wakeup_secondary_cpu_via_init
> /* reduce the number of lines printed when booting a large cpu count system */
> static void announce_cpu(int cpu, int apicid)
> {
> + static int width, node_width, first = 1;
> static int current_node = NUMA_NO_NODE;
> int node = early_cpu_to_node(cpu);
> - static int width, node_width;
>
> if (!width)
> width = num_digits(num_possible_cpus()) + 1; /* + '#' sign */
> @@ -962,7 +962,7 @@ static void announce_cpu(int cpu, int ap
> node_width = num_digits(num_possible_nodes()) + 1; /* + '#' */
>
> if (system_state < SYSTEM_RUNNING) {
> - if (num_online_cpus() == 1)
> + if (first)
> pr_info("x86: Booting SMP configuration:\n");
>
> if (node != current_node) {
> @@ -975,11 +975,11 @@ static void announce_cpu(int cpu, int ap
> }
>
> /* Add padding for the BSP */
> - if (num_online_cpus() == 1)
> + if (first)
> pr_cont("%*s", width + 1, " ");
> + first = 0;
>
> pr_cont("%*s#%d", width - num_digits(cpu), " ", cpu);
> -
> } else
> pr_info("Booting Node %d Processor %d APIC 0x%x\n",
> node, cpu, apicid);
This works. dmesg output is clean for these guest VM combinations
on Hyper-V that I tested:
* Normal VM: 16 vCPUs in 1 NUMA node and 32 vCPUs in 2 NUMA nodes
* Same configs for a SEV-SNP Confidential VM with paravisor
Tested with and without cpuhp.parallel=0
For the entire series:
Tested-by: Michael Kelley <mikelley@microsoft.com>
prev parent reply other threads:[~2023-05-07 4:14 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-04 19:01 [patch V2 00/38] cpu/hotplug, x86: Reworked parallel CPU bringup Thomas Gleixner
2023-05-04 19:02 ` [patch V2 01/38] x86/smpboot: Cleanup topology_phys_to_logical_pkg()/die() Thomas Gleixner
2023-05-04 19:02 ` [patch V2 02/38] cpu/hotplug: Mark arch_disable_smp_support() and bringup_nonboot_cpus() __init Thomas Gleixner
2023-05-04 19:02 ` [patch V2 03/38] x86/smpboot: Avoid pointless delay calibration if TSC is synchronized Thomas Gleixner
2023-05-04 19:02 ` [patch V2 04/38] x86/smpboot: Rename start_cpu0() to soft_restart_cpu() Thomas Gleixner
2023-05-04 19:02 ` [patch V2 05/38] x86/topology: Remove CPU0 hotplug option Thomas Gleixner
2023-05-04 19:02 ` [patch V2 06/38] x86/smpboot: Remove the CPU0 hotplug kludge Thomas Gleixner
2023-05-04 19:02 ` [patch V2 07/38] x86/smpboot: Restrict soft_restart_cpu() to SEV Thomas Gleixner
2023-05-04 19:02 ` [patch V2 08/38] x86/smpboot: Split up native_cpu_up() into separate phases and document them Thomas Gleixner
2023-05-04 19:02 ` [patch V2 09/38] x86/smpboot: Get rid of cpu_init_secondary() Thomas Gleixner
2023-05-04 19:02 ` [patch V2 10/38] x86/cpu/cacheinfo: Remove cpu_callout_mask dependency Thomas Gleixner
2023-05-04 19:02 ` [patch V2 11/38] x86/smpboot: Move synchronization masks to SMP boot code Thomas Gleixner
2023-05-04 19:02 ` [patch V2 12/38] x86/smpboot: Make TSC synchronization function call based Thomas Gleixner
2023-05-04 19:02 ` [patch V2 13/38] x86/smpboot: Remove cpu_callin_mask Thomas Gleixner
2023-05-04 19:02 ` [patch V2 14/38] cpu/hotplug: Rework sparse_irq locking in bringup_cpu() Thomas Gleixner
2023-05-04 19:02 ` [patch V2 15/38] x86/smpboot: Remove wait for cpu_online() Thomas Gleixner
2023-05-04 19:02 ` [patch V2 16/38] x86/xen/smp_pv: Remove wait for CPU online Thomas Gleixner
2023-05-04 19:02 ` [patch V2 17/38] x86/xen/hvm: Get rid of DEAD_FROZEN handling Thomas Gleixner
2023-05-04 19:02 ` [patch V2 18/38] cpu/hotplug: Add CPU state tracking and synchronization Thomas Gleixner
2023-05-04 19:02 ` [patch V2 19/38] x86/smpboot: Switch to hotplug core state synchronization Thomas Gleixner
2023-05-04 19:02 ` [patch V2 20/38] cpu/hotplug: Remove cpu_report_state() and related unused cruft Thomas Gleixner
2023-05-04 19:02 ` [patch V2 21/38] ARM: smp: Switch to hotplug core state synchronization Thomas Gleixner
2023-05-04 19:02 ` [patch V2 22/38] arm64: " Thomas Gleixner
2023-05-04 19:02 ` [patch V2 23/38] csky/smp: " Thomas Gleixner
2023-05-04 19:02 ` [patch V2 24/38] MIPS: SMP_CPS: " Thomas Gleixner
2023-05-04 19:02 ` [patch V2 25/38] parisc: " Thomas Gleixner
2023-05-04 19:02 ` [patch V2 26/38] riscv: " Thomas Gleixner
2023-05-04 19:02 ` [patch V2 27/38] cpu/hotplug: Remove unused state functions Thomas Gleixner
2023-05-04 19:02 ` [patch V2 28/38] cpu/hotplug: Reset task stack state in _cpu_up() Thomas Gleixner
2023-05-04 19:02 ` [patch V2 29/38] cpu/hotplug: Provide a split up CPUHP_BRINGUP mechanism Thomas Gleixner
2023-05-04 19:02 ` [patch V2 30/38] x86/smpboot: Enable split CPU startup Thomas Gleixner
2023-05-04 19:02 ` [patch V2 31/38] x86/apic: Provide cpu_primary_thread mask Thomas Gleixner
2023-05-04 19:02 ` [patch V2 32/38] cpu/hotplug: Allow "parallel" bringup up to CPUHP_BP_KICK_AP_STATE Thomas Gleixner
2023-05-04 19:02 ` [patch V2 33/38] x86/topology: Store extended topology leaf information Thomas Gleixner
2023-05-04 19:02 ` [patch V2 34/38] x86/cpu/amd; Invoke detect_extended_topology_early() on boot CPU Thomas Gleixner
2023-05-04 23:04 ` Andrew Cooper
2023-05-05 12:45 ` Thomas Gleixner
2023-05-04 19:02 ` [patch V2 35/38] x86/apic: Save the APIC virtual base address Thomas Gleixner
2023-05-04 19:02 ` [patch V2 36/38] x86/smpboot: Implement a bit spinlock to protect the realmode stack Thomas Gleixner
2023-05-04 19:02 ` [patch V2 37/38] x86/smpboot: Support parallel startup of secondary CPUs Thomas Gleixner
2023-05-04 19:03 ` [patch V2 38/38] x86/smpboot/64: Implement arch_cpuhp_init_parallel_bringup() and enable it Thomas Gleixner
2023-05-06 0:53 ` Michael Kelley (LINUX)
2023-05-06 16:22 ` Thomas Gleixner
2023-05-07 4:14 ` Michael Kelley (LINUX) [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BYAPR21MB1688A75D614165C5E4E0302AD7709@BYAPR21MB1688.namprd21.prod.outlook.com \
--to=mikelley@microsoft.com \
--cc=James.Bottomley@HansenPartnership.com \
--cc=Mark.Rutland@arm.com \
--cc=andrew.cooper3@citrix.com \
--cc=arjan@linux.intel.com \
--cc=arnd@arndb.de \
--cc=boris.ostrovsky@oracle.com \
--cc=brgerst@gmail.com \
--cc=catalin.marinas@arm.com \
--cc=deller@gmx.de \
--cc=dwmw2@infradead.org \
--cc=gpiccoli@igalia.com \
--cc=guoren@kernel.org \
--cc=jgross@suse.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-csky@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mips@vger.kernel.org \
--cc=linux-parisc@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=linux@armlinux.org.uk \
--cc=lucjan.lucjanov@gmail.com \
--cc=oleksandr@natalenko.name \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=paulmck@kernel.org \
--cc=pbonzini@redhat.com \
--cc=pmenzel@molgen.mpg.de \
--cc=sabrapan@amazon.com \
--cc=seanjc@google.com \
--cc=tglx@linutronix.de \
--cc=thomas.lendacky@amd.com \
--cc=tsbogend@alpha.franken.de \
--cc=usama.arif@bytedance.com \
--cc=will@kernel.org \
--cc=x86@kernel.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).