From: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
To: <paulmck@kernel.org>, <frederic@kernel.org>,
<joel@joelfernandes.org>, <josh@joshtriplett.org>,
<boqun.feng@gmail.com>, <rostedt@goodmis.org>,
<mathieu.desnoyers@efficios.com>, <jiangshanlai@gmail.com>,
<qiang.zhang1211@gmail.com>
Cc: <rcu@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
<neeraj.upadhyay@kernel.org>,
Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
Subject: [PATCH] rcu: Reduce synchronize_rcu() delays when all wait heads are in use
Date: Wed, 13 Mar 2024 14:02:28 +0530 [thread overview]
Message-ID: <20240313083228.233247-1-Neeraj.Upadhyay@amd.com> (raw)
When all wait heads are in use, which can happen when
rcu_sr_normal_gp_cleanup_work()'s callback processing
is slow, any new synchronize_rcu() user's rcu_synchronize
node's processing is deferred to future GP periods. This
can result in long list of synchronize_rcu() invocations
waiting for full grace period processing, which can delay
freeing of memory. Mitigate this problem by using first
node in the list as wait tail when all wait heads are in use.
While methods to speed up callback processing would be needed
to recover from this situation, allowing new nodes to complete
their grace period can help prevent delays due to a fixed
number of wait head nodes.
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
kernel/rcu/tree.c | 27 +++++++++++++--------------
1 file changed, 13 insertions(+), 14 deletions(-)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 9fbb5ab57c84..bdccce1ed62f 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -1470,14 +1470,11 @@ static void rcu_poll_gp_seq_end_unlocked(unsigned long *snap)
* for this new grace period. Given that there are a fixed
* number of wait nodes, if all wait nodes are in use
* (which can happen when kworker callback processing
- * is delayed) and additional grace period is requested.
- * This means, a system is slow in processing callbacks.
- *
- * TODO: If a slow processing is detected, a first node
- * in the llist should be used as a wait-tail for this
- * grace period, therefore users which should wait due
- * to a slow process are handled by _this_ grace period
- * and not next.
+ * is delayed), first node in the llist is used as wait
+ * tail for this grace period. This means, the first node
+ * has to go through additional grace periods before it is
+ * part of the wait callbacks. This should be ok, as
+ * the system is slow in processing callbacks anyway.
*
* Below is an illustration of how the done and wait
* tail pointers move from one set of rcu_synchronize nodes
@@ -1725,15 +1722,17 @@ static bool rcu_sr_normal_gp_init(void)
return start_new_poll;
wait_head = rcu_sr_get_wait_head();
- if (!wait_head) {
- // Kick another GP to retry.
+ if (wait_head) {
+ /* Inject a wait-dummy-node. */
+ llist_add(wait_head, &rcu_state.srs_next);
+ } else {
+ // Kick another GP for first node.
start_new_poll = true;
- return start_new_poll;
+ if (first == rcu_state.srs_done_tail)
+ return start_new_poll;
+ wait_head = first;
}
- /* Inject a wait-dummy-node. */
- llist_add(wait_head, &rcu_state.srs_next);
-
/*
* A waiting list of rcu_synchronize nodes should be empty on
* this step, since a GP-kthread, rcu_gp_init() -> gp_cleanup(),
--
2.34.1
next reply other threads:[~2024-03-13 8:33 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-13 8:32 Neeraj Upadhyay [this message]
2024-03-13 14:40 ` [PATCH] rcu: Reduce synchronize_rcu() delays when all wait heads are in use Joel Fernandes
2024-03-13 16:04 ` Neeraj Upadhyay
2024-03-13 16:13 ` Joel Fernandes
2024-03-13 16:19 ` Neeraj Upadhyay
2024-03-13 15:18 ` Frederic Weisbecker
2024-03-13 16:11 ` Neeraj Upadhyay
2024-03-13 16:43 ` Frederic Weisbecker
2024-03-13 16:54 ` Neeraj Upadhyay
2024-03-13 17:15 ` Frederic Weisbecker
2024-03-13 17:26 ` Neeraj Upadhyay
2024-03-13 17:49 ` Neeraj Upadhyay
2024-03-13 23:22 ` Frederic Weisbecker
2024-03-13 17:05 ` Frederic Weisbecker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240313083228.233247-1-Neeraj.Upadhyay@amd.com \
--to=neeraj.upadhyay@amd.com \
--cc=boqun.feng@gmail.com \
--cc=frederic@kernel.org \
--cc=jiangshanlai@gmail.com \
--cc=joel@joelfernandes.org \
--cc=josh@joshtriplett.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=neeraj.upadhyay@kernel.org \
--cc=paulmck@kernel.org \
--cc=qiang.zhang1211@gmail.com \
--cc=rcu@vger.kernel.org \
--cc=rostedt@goodmis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).