From: William Tambe <tambewilliam@gmail.com>
To: linux-smp@vger.kernel.org
Subject: experiencing rcu stalls on Linux 5.13 SMP PREEMPT
Date: Sat, 11 Sep 2021 14:20:45 -0500 [thread overview]
Message-ID: <CAF8i9mN=eiTWzCij1u=yNA9cg3ThD_VhY8Jc4HpxPdowm68jiw@mail.gmail.com> (raw)
Hi,
I am adding Linux SMP support to a custom architecture with sources at:
https://github.com/fontamsoc/linux/tree/pu32.20210831
I am experiencing slow boot and rcu stalls on Linux 5.13 SMP PREEMPT
with a backtrace as shown below.
From below backtrace, would it be possible to pinpoint what could have
possibly been missed ?
[ 64.354850] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 64.355284] (detected by 0, t=2102 jiffies, g=-931, q=20)
[ 64.355959] rcu: All QSes seen, last rcu_preempt kthread activity
2102 (-23612--25714), jiffies_till_next_fqs=1, root ->qsmask 0x0
[ 64.356645] rcu: rcu_preempt kthread timer wakeup didn't happen for
2101 jiffies! g-931 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x200
[ 64.357455] rcu: Possible timer handling issue on cpu=1 timer-softirq=121
[ 64.358122] rcu: rcu_preempt kthread starved for 2102 jiffies!
g-931 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x200 ->cpu=1
[ 64.358916] rcu: Unless rcu_preempt kthread gets sufficient CPU
time, OOM is now expected behavior.
[ 64.359961] rcu: RCU grace-period kthread stack dump:
[ 64.360329] task:rcu_preempt state:R stack: 0 pid: 11
ppid: 2 flags:0x00000000
[ 64.361072] stacktrace:
[ 64.361466] 0x00c19d3c 0x000674d4 sched_show_task+0x1ca/0x1e6
[ 64.362481] 0x00c19d5c 0x000a799c rcu_check_gp_kthread_starvation+0x14c/0x1fa
[ 64.363284] 0x00c19da0 0x000ae47a print_other_cpu_stall+0x39c/0x446
[ 64.363756] 0x00c19dd4 0x000ae70e check_cpu_stall+0x1ea/0x2c2
[ 64.364207] 0x00c19df8 0x000ae81e rcu_pending+0x38/0x1b8
[ 64.364643] 0x00c19e0c 0x000ae9fa rcu_sched_clock_irq+0x5c/0xe6
[ 64.365065] 0x00c19e18 0x000b7e66 update_process_times+0x7e/0xba
[ 64.365538] 0x00c19e20 0x000cf6da tick_sched_handle+0xc8/0xe0
[ 64.366113] 0x00c19e28 0x000cfd70 tick_sched_timer+0x6c/0xe6
[ 64.366606] 0x00c19e54 0x000b8bc4 __run_hrtimer+0x6c/0x12c
[ 64.367028] 0x00c19e70 0x000b8d20 __hrtimer_run_queues+0x9c/0xe4
[ 64.367545] 0x00c19e94 0x000ba460 hrtimer_interrupt+0x1f2/0x556
[ 64.368063] 0x00c19ed4 0x000288fe pu32_timer_intr+0x2a/0x46
[ 64.368560] 0x00c19ed8 0x00025ac6 pu32ctxswitchhdlr+0xe6e/0x1108
[ 64.369031] rcu: Stack dump where RCU GP kthread last ran:
[ 64.369698] Task dump for CPU 1:
[ 64.369893] task:swapper/1 state:R running task stack:
0 pid: 0 ppid: 1 flags:0x00000000
[ 64.370403] stacktrace:
[ 64.370693] 0x00c19d2c 0x000674d4 sched_show_task+0x1ca/0x1e6
[ 64.371120] 0x00c19d4c 0x000682fe dump_cpu_task+0x4a/0x5a
[ 64.371572] 0x00c19d5c 0x000a7a46 rcu_check_gp_kthread_starvation+0x1f6/0x1fa
[ 64.372060] 0x00c19da0 0x000ae47a print_other_cpu_stall+0x39c/0x446
[ 64.372599] 0x00c19dd4 0x000ae70e check_cpu_stall+0x1ea/0x2c2
[ 64.373246] 0x00c19df8 0x000ae81e rcu_pending+0x38/0x1b8
[ 64.373671] 0x00c19e0c 0x000ae9fa rcu_sched_clock_irq+0x5c/0xe6
[ 64.374290] 0x00c19e18 0x000b7e66 update_process_times+0x7e/0xba
[ 64.374857] 0x00c19e20 0x000cf6da tick_sched_handle+0xc8/0xe0
[ 64.375613] 0x00c19e28 0x000cfd70 tick_sched_timer+0x6c/0xe6
[ 64.376458] 0x00c19e54 0x000b8bc4 __run_hrtimer+0x6c/0x12c
[ 64.377078] 0x00c19e70 0x000b8d20 __hrtimer_run_queues+0x9c/0xe4
[ 64.377675] 0x00c19e94 0x000ba460 hrtimer_interrupt+0x1f2/0x556
[ 64.378344] 0x00c19ed4 0x000288fe pu32_timer_intr+0x2a/0x46
[ 64.378934] 0x00c19ed8 0x00025ac6 pu32ctxswitchhdlr+0xe6e/0x1108
[ 64.495067] BUG: scheduling while atomic: swapper/0/0/0x00000002
[ 64.495545] CPU: 0 PID: 0 Comm: swapper/0 Not tainted
5.13.0-00006-g8f51a70fa15c #1
[ 64.495946] stacktrace:
[ 64.496207] 0x006aeebc 0x005ba894 __dump_stack+0x22/0x2c
[ 64.496643] 0x006aeec0 0x005ba91c dump_stack+0x6a/0x9e
[ 64.497074] 0x006aeecc 0x0005df1c __schedule_bug+0x84/0x94
[ 64.497521] 0x006aeee0 0x005dad26 __schedule+0x1c6/0x730
[ 64.498124] 0x006aef14 0x005db420 schedule_idle+0x22/0x7c
[ 64.498609] 0x006aef1c 0x00069854 do_idle+0xf0/0xfe
[ 64.499029] 0x006aef28 0x00069b96 cpu_startup_entry+0x1c/0x20
reply other threads:[~2021-09-11 19:20 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAF8i9mN=eiTWzCij1u=yNA9cg3ThD_VhY8Jc4HpxPdowm68jiw@mail.gmail.com' \
--to=tambewilliam@gmail.com \
--cc=linux-smp@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).