From: Dongli Zhang <dongli.zhang@oracle.com>
To: linux-kernel@vger.kernel.org
Cc: tglx@linutronix.de, qais.yousef@arm.com, peterz@infradead.org,
mpe@ellerman.id.au, aneesh.kumar@linux.ibm.com, ethp@qq.com,
npiggin@gmail.com, joe.jin@oracle.com
Subject: Re: [PATCH RFC 1/1] kernel/cpu: to track which CPUHP callback is failed
Date: Mon, 5 Apr 2021 14:22:01 -0700 [thread overview]
Message-ID: <40a7be6b-0603-a2ab-afaa-339e88a03210@oracle.com> (raw)
In-Reply-To: <20210310070837.17674-1-dongli.zhang@oracle.com>
May I have if there is any feedback on this? To pr_err_once() here helps narrow
down what is the root cause of cpu online failure.
The issue fixed by d7eb79c6290c ("KVM: kvmclock: Fix vCPUs > 64 can't be
online/hotpluged") is able to demonstrate how this pr_err_once() helps.
Due to VM kernel issue, at most 64 VCPUs can online if host clocksource is
switched to hpet.
# echo hpet > /sys/devices/system/clocksource/clocksource0/current_clocksource
By default, we have no idea why only 64 out of 100 VCPUs are online in VM. We
need to customize kernel to debug why some CPUs are not able to online.
We will have below and know the root cause is with kvmclock, if we add the
pr_err_once().
[ 0.693112] CPUHP callback failure (-12) for cpu 64 at kvmclock:setup_percpu (66)
Thank you very much!
Dongli Zhang
On 3/9/21 11:08 PM, Dongli Zhang wrote:
> During bootup or cpu hotplug, the cpuhp_up_callbacks() calls many CPUHP
> callbacks (e.g., perf, mm, workqueue, RCU, kvmclock and more) for each
> cpu to online. It may roll back to its previous state if any of
> callbacks is failed. As a result, the user will not be able to know
> which callback is failed and usually the only symptom is cpu online
> failure.
>
> The error log is printed for once to have confirm which callback is
> failed.
>
> Cc: Joe Jin <joe.jin@oracle.com>
> Signed-off-by: Dongli Zhang <dongli.zhang@oracle.com>
> ---
> I used 'RFC' because WARN_ON_ONCE() is always used for the result from
> cpuhp_invoke_callback(). I would prefer to get feedback from
> maintainers/reviewers. Here I prefer to print the cpuhp name and state
> value to help confirm the specific callback that is failed.
>
> kernel/cpu.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 1b6302ecbabe..c7a719079272 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -621,6 +621,10 @@ static int cpuhp_up_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
> st->state++;
> ret = cpuhp_invoke_callback(cpu, st->state, true, NULL, NULL);
> if (ret) {
> + pr_err_once("CPUHP callback failure (%d) for cpu %u at %s (%d)\n",
> + ret, cpu, cpuhp_get_step(st->state)->name,
> + st->state);
> +
> if (can_rollback_cpu(st)) {
> st->target = prev_state;
> undo_cpu_up(cpu, st);
>
next prev parent reply other threads:[~2021-04-05 21:25 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-10 7:08 [PATCH RFC 1/1] kernel/cpu: to track which CPUHP callback is failed Dongli Zhang
2021-04-05 21:22 ` Dongli Zhang [this message]
2021-04-06 16:10 ` Qais Yousef
2021-04-07 22:13 ` Dongli Zhang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=40a7be6b-0603-a2ab-afaa-339e88a03210@oracle.com \
--to=dongli.zhang@oracle.com \
--cc=aneesh.kumar@linux.ibm.com \
--cc=ethp@qq.com \
--cc=joe.jin@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mpe@ellerman.id.au \
--cc=npiggin@gmail.com \
--cc=peterz@infradead.org \
--cc=qais.yousef@arm.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).