From: Alexander Aring <aahringo@redhat.com>
To: teigland@redhat.com
Cc: gfs2@lists.linux.dev, aahringo@redhat.com
Subject: [PATCHv3 v6.8-rc6 18/18] dlm: do dlm message processing in softirq context
Date: Mon, 26 Feb 2024 20:49:09 -0500 [thread overview]
Message-ID: <20240227014909.93945-19-aahringo@redhat.com> (raw)
In-Reply-To: <20240227014909.93945-1-aahringo@redhat.com>
This patch moves the dlm message processing from a ordered workqueue
context to a ordered softirq context. Later we want to call the user
defined ast/bast callbacks directly inside the dlm message processing
context instead of doing an additional context switch to the exisiting
callback workqueue. This should slightly improve the dlm message parsing
behaviour. There are two main reasons why to change to this behaviour:
1.
Allow fewer scheduling possibilities for dlm message parsing context.
This should deliver faster DLM user responses to ast/bast callbacks.
Fewer interrupting of lock requests processing that might trigger a
new lock request avoids situations that we don't finish lock
requests. In future the DLM callback workqueue can be disabled by
a kernel lockspace flag to signal the DLM kernel user is capable
of exectuing the callbacks in softirq context. If this flag is set,
the dlm processing gets rid of an additional queue_work() context
switch that should take more advantage about the new softirq context
because the last preemption possibility is removed from the message
processing context.
2. Bringing the ast/callback callback to softirq context that the use is
aware it should not block into this context. Later patches will
introduce a per lockspace flag to signal that the user is capable to
handling these callbacks in softirq context to solve backwards
compatibility.
3. We can easily switch to a per dlm instance concurrent dlm message
parsing when DLM is ready to handle it. This instance could be in
e.g. per lockspace instance or more fine granulatiry instance such as
per lock instance.
Futher patches will unveil more improvements to switch to a per message
softirq parsing context. Especially if we getting DLM in a state that we
can allow concurrent message parsing.
Signed-off-by: Alexander Aring <aahringo@redhat.com>
---
fs/dlm/lowcomms.c | 34 ++++++++++++----------------------
1 file changed, 12 insertions(+), 22 deletions(-)
diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c
index 444dc858c4a4..303d2837f68b 100644
--- a/fs/dlm/lowcomms.c
+++ b/fs/dlm/lowcomms.c
@@ -183,7 +183,6 @@ static int dlm_local_count;
/* Work queues */
static struct workqueue_struct *io_workqueue;
-static struct workqueue_struct *process_workqueue;
static struct hlist_head connection_hash[CONN_HASH_SIZE];
static DEFINE_SPINLOCK(connections_lock);
@@ -199,9 +198,9 @@ static const struct dlm_proto_ops *dlm_proto_ops;
static void process_recv_sockets(struct work_struct *work);
static void process_send_sockets(struct work_struct *work);
-static void process_dlm_messages(struct work_struct *work);
+static void process_dlm_messages(struct tasklet_struct *tasklet);
-static DECLARE_WORK(process_work, process_dlm_messages);
+static DECLARE_TASKLET_DISABLED(process_tasklet, process_dlm_messages);
static DEFINE_SPINLOCK(processqueue_lock);
static bool process_dlm_messages_pending;
static atomic_t processqueue_count;
@@ -863,7 +862,7 @@ struct dlm_processed_nodes {
struct list_head list;
};
-static void process_dlm_messages(struct work_struct *work)
+static void process_dlm_messages(struct tasklet_struct *tasklet)
{
struct processqueue_entry *pentry;
@@ -971,7 +970,7 @@ static int receive_from_sock(struct connection *con, int buflen)
list_add_tail(&pentry->list, &processqueue);
if (!process_dlm_messages_pending) {
process_dlm_messages_pending = true;
- queue_work(process_workqueue, &process_work);
+ tasklet_schedule(&process_tasklet);
}
spin_unlock_bh(&processqueue_lock);
@@ -1511,7 +1510,8 @@ static void process_recv_sockets(struct work_struct *work)
/* CF_RECV_PENDING cleared */
break;
case DLM_IO_FLUSH:
- flush_workqueue(process_workqueue);
+ tasklet_disable(&process_tasklet);
+ tasklet_enable(&process_tasklet);
fallthrough;
case DLM_IO_RESCHED:
cond_resched();
@@ -1686,10 +1686,7 @@ static void work_stop(void)
io_workqueue = NULL;
}
- if (process_workqueue) {
- destroy_workqueue(process_workqueue);
- process_workqueue = NULL;
- }
+ tasklet_disable(&process_tasklet);
}
static int work_start(void)
@@ -1701,17 +1698,7 @@ static int work_start(void)
return -ENOMEM;
}
- /* ordered dlm message process queue,
- * should be converted to a tasklet
- */
- process_workqueue = alloc_ordered_workqueue("dlm_process",
- WQ_HIGHPRI | WQ_MEM_RECLAIM);
- if (!process_workqueue) {
- log_print("can't start dlm_process");
- destroy_workqueue(io_workqueue);
- io_workqueue = NULL;
- return -ENOMEM;
- }
+ tasklet_enable(&process_tasklet);
return 0;
}
@@ -1734,7 +1721,10 @@ void dlm_lowcomms_shutdown(void)
hlist_for_each_entry_rcu(con, &connection_hash[i], list) {
shutdown_connection(con, true);
stop_connection_io(con);
- flush_workqueue(process_workqueue);
+
+ tasklet_disable(&process_tasklet);
+ tasklet_enable(&process_tasklet);
+
close_connection(con, true);
clean_one_writequeue(con);
--
2.43.0
prev parent reply other threads:[~2024-02-27 1:49 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-27 1:48 [PATCHv3 v6.8-rc6 00/18] dlm: bring message parsing to softirq context Alexander Aring
2024-02-27 1:48 ` [PATCHv3 v6.8-rc6 01/18] fs: dlm: Simplify the allocation of slab caches in dlm_midcomms_cache_create Alexander Aring
2024-02-27 1:48 ` [PATCHv3 v6.8-rc6 02/18] fs: dlm: Simplify the allocation of slab caches in dlm_lowcomms_msg_cache_create Alexander Aring
2024-02-27 1:48 ` [PATCHv3 v6.8-rc6 03/18] dlm: fix off-by-one waiters refcount handling Alexander Aring
2024-02-27 1:48 ` [PATCHv3 v6.8-rc6 04/18] dlm: put lkbs instead of force free Alexander Aring
2024-02-27 1:48 ` [PATCHv3 v6.8-rc6 05/18] dlm: remove allocation parameter in msg allocation Alexander Aring
2024-02-27 1:48 ` [PATCHv3 v6.8-rc6 06/18] dlm: switch to GFP_ATOMIC in dlm allocations Alexander Aring
2024-02-27 1:48 ` [PATCHv3 v6.8-rc6 07/18] dlm: move root_list functionality to recover.c Alexander Aring
2024-02-27 1:48 ` [PATCHv3 v6.8-rc6 08/18] dlm: move master dir dump to own list Alexander Aring
2024-02-27 1:49 ` [PATCHv3 v6.8-rc6 09/18] dlm: move root_list to ls_recover() stack Alexander Aring
2024-02-27 1:49 ` [PATCHv3 v6.8-rc6 10/18] dlm: implement directory dump context Alexander Aring
2024-02-27 1:49 ` [PATCHv3 v6.8-rc6 11/18] dlm: drop holding waiters mutex in waiters recovery Alexander Aring
2024-02-27 1:49 ` [PATCHv3 v6.8-rc6 12/18] dlm: convert ls_waiters_mutex to spinlock Alexander Aring
2024-02-27 1:49 ` [PATCHv3 v6.8-rc6 13/18] dlm: convert res_lock " Alexander Aring
2024-02-27 1:49 ` [PATCHv3 v6.8-rc6 14/18] dlm: make requestqueue handling non sleepable Alexander Aring
2024-02-27 1:49 ` [PATCHv3 v6.8-rc6 15/18] dlm: ls_recv_active semaphore to rwlock Alexander Aring
2024-02-27 1:49 ` [PATCHv3 v6.8-rc6 16/18] dlm: remove schedule in dlm receive path Alexander Aring
2024-02-27 1:49 ` [PATCHv3 v6.8-rc6 17/18] dlm: convert message parsing locks to disable bh Alexander Aring
2024-02-27 1:49 ` Alexander Aring [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240227014909.93945-19-aahringo@redhat.com \
--to=aahringo@redhat.com \
--cc=gfs2@lists.linux.dev \
--cc=teigland@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).