From: Mikulas Patocka <mpatocka@redhat.com>
To: Linus Torvalds <torvalds@linux-foundation.org>,
Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev,
Mike Snitzer <msnitzer@redhat.com>,
Ignat Korchagin <ignat@cloudflare.com>,
Damien Le Moal <damien.lemoal@wdc.com>,
Bob Liu <bob.liu@oracle.com>, Hou Tao <houtao1@huawei.com>,
Nathan Huckleberry <nhuck@google.com>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@elte.hu>
Subject: [PATCH] softirq: fix memory corruption when freeing tasklet_struct
Date: Thu, 25 Jan 2024 19:29:42 +0100 (CET) [thread overview]
Message-ID: <82b964f0-c2c8-a2c6-5b1f-f3145dc2c8e5@redhat.com> (raw)
Hi
There's a problem with the tasklet API - there is no reliable way how to
free a structure that contains tasklet_struct. The problem is that the
function tasklet_action_common calls task_unlock(t) after it called the
callback. If the callback does something that frees tasklet_struct,
task_unlock(t) would write into free memory.
dm-crypt does this - it schedules a tasklet with tasklet_schedule, it does
encryption inside the tasklet handler (because it performs better than
doing the encryption in a workqueue), then it submits a workqueue entry
and calls bio_endio from the workqueue entry.
However, if the workqueue preempts ksoftirqd, this race condition happens:
ksoftirqd:
* tasklet_action_common
* t->func(t->data) (that points to kcryptd_crypt_tasklet)
* kcryptd_crypt_tasklet
* kcryptd_crypt
* kcryptd_crypt_read_convert
* crypt_dec_pending
* queue_work(cc->io_queue, &io->work);
now we switch to the workqueue process:
* kcryptd_io_bio_endio
* bio_endio(io->base_bio) (this calls clone_endio)
* clone_endio
* free_tio
* bio_put(clone) - the bio is freed
now we switch back to ksoftirqd:
* tasklet_action_common calls task_unlock(t)
* task_unlock(t) touches memory that was already freed when the bio was freed
dm-verity has a similar problem.
In order to fix this bug, I am proposing to add a new flag
TASKLET_STATE_ONESHOT. The flag indicates that the tasklet will be
submitted only once and it prevents tasklet_action_common from touching
the tasklet after the callback completed.
If you have another idea how to solve this bug, let me know.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Fixes: 39d42fa96ba1 ("dm crypt: add flags to optionally bypass kcryptd workqueues")
Fixes: 5721d4e5a9cd ("dm verity: Add optional "try_verify_in_tasklet" feature")
Cc: stable@vger.kernel.org # v5.9+
---
drivers/md/dm-crypt.c | 1 +
drivers/md/dm-verity-target.c | 1 +
include/linux/interrupt.h | 9 ++++++++-
kernel/softirq.c | 22 +++++++++++++++-------
4 files changed, 25 insertions(+), 8 deletions(-)
Index: linux-2.6/drivers/md/dm-crypt.c
===================================================================
--- linux-2.6.orig/drivers/md/dm-crypt.c 2024-01-18 19:18:30.000000000 +0100
+++ linux-2.6/drivers/md/dm-crypt.c 2024-01-25 16:42:17.000000000 +0100
@@ -2265,6 +2265,7 @@ static void kcryptd_queue_crypt(struct d
if (in_hardirq() || irqs_disabled()) {
io->in_tasklet = true;
tasklet_init(&io->tasklet, kcryptd_crypt_tasklet, (unsigned long)&io->work);
+ tasklet_set_oneshot(&io->tasklet);
tasklet_schedule(&io->tasklet);
return;
}
Index: linux-2.6/include/linux/interrupt.h
===================================================================
--- linux-2.6.orig/include/linux/interrupt.h 2023-11-13 17:39:50.000000000 +0100
+++ linux-2.6/include/linux/interrupt.h 2024-01-25 16:41:52.000000000 +0100
@@ -684,7 +684,9 @@ struct tasklet_struct name = { \
enum
{
TASKLET_STATE_SCHED, /* Tasklet is scheduled for execution */
- TASKLET_STATE_RUN /* Tasklet is running (SMP only) */
+ TASKLET_STATE_RUN, /* Tasklet is running (SMP only) */
+ TASKLET_STATE_ONESHOT /* Don't unlock the tasklet after the callback
+ to avoid writing to free memory */
};
#if defined(CONFIG_SMP) || defined(CONFIG_PREEMPT_RT)
@@ -756,6 +758,11 @@ extern void tasklet_init(struct tasklet_
extern void tasklet_setup(struct tasklet_struct *t,
void (*callback)(struct tasklet_struct *));
+static inline void tasklet_set_oneshot(struct tasklet_struct *t)
+{
+ __set_bit(TASKLET_STATE_ONESHOT, &t->state);
+}
+
/*
* Autoprobing for irqs:
*
Index: linux-2.6/kernel/softirq.c
===================================================================
--- linux-2.6.orig/kernel/softirq.c 2023-10-31 15:31:42.000000000 +0100
+++ linux-2.6/kernel/softirq.c 2024-01-25 17:10:03.000000000 +0100
@@ -774,18 +774,26 @@ static void tasklet_action_common(struct
if (tasklet_trylock(t)) {
if (!atomic_read(&t->count)) {
+ /*
+ * If oneshot is set, we must not touch the
+ * tasklet after the callback.
+ */
+ bool oneshot = test_bit(TASKLET_STATE_ONESHOT, &t->state);
if (tasklet_clear_sched(t)) {
if (t->use_callback) {
- trace_tasklet_entry(t, t->callback);
- t->callback(t);
- trace_tasklet_exit(t, t->callback);
+ void (*callback)(struct tasklet_struct *) = t->callback;
+ trace_tasklet_entry(t, callback);
+ callback(t);
+ trace_tasklet_exit(t, callback);
} else {
- trace_tasklet_entry(t, t->func);
- t->func(t->data);
- trace_tasklet_exit(t, t->func);
+ void (*func)(unsigned long) = t->func;
+ trace_tasklet_entry(t, func);
+ func(t->data);
+ trace_tasklet_exit(t, func);
}
}
- tasklet_unlock(t);
+ if (!oneshot)
+ tasklet_unlock(t);
continue;
}
tasklet_unlock(t);
Index: linux-2.6/drivers/md/dm-verity-target.c
===================================================================
--- linux-2.6.orig/drivers/md/dm-verity-target.c 2024-01-18 19:18:30.000000000 +0100
+++ linux-2.6/drivers/md/dm-verity-target.c 2024-01-25 18:12:09.000000000 +0100
@@ -676,6 +676,7 @@ static void verity_end_io(struct bio *bi
if (static_branch_unlikely(&use_tasklet_enabled) && io->v->use_tasklet) {
tasklet_init(&io->tasklet, verity_tasklet, (unsigned long)io);
+ tasklet_set_oneshot(&io->tasklet);
tasklet_schedule(&io->tasklet);
} else {
INIT_WORK(&io->work, verity_work);
next reply other threads:[~2024-01-25 18:30 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-25 18:29 Mikulas Patocka [this message]
2024-01-25 19:51 ` [PATCH] softirq: fix memory corruption when freeing tasklet_struct Linus Torvalds
2024-01-25 22:04 ` Mikulas Patocka
2024-01-25 23:15 ` Damien Le Moal
2024-01-26 6:56 ` Hannes Reinecke
2024-01-26 18:09 ` Mikulas Patocka
2024-01-29 8:26 ` request based dm-multipath, was " Christoph Hellwig
2024-01-25 22:51 ` Ignat Korchagin
2024-01-26 0:33 ` Tejun Heo
2024-01-26 18:36 ` Allen
2024-01-26 23:43 ` Tejun Heo
2024-01-27 3:13 ` Allen
2024-01-27 18:37 ` Tejun Heo
2024-01-30 9:19 ` Tejun Heo
2024-01-29 17:00 ` Allen
2024-01-29 17:06 ` Tejun Heo
2024-04-09 14:15 ` Li Lingfeng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=82b964f0-c2c8-a2c6-5b1f-f3145dc2c8e5@redhat.com \
--to=mpatocka@redhat.com \
--cc=bob.liu@oracle.com \
--cc=damien.lemoal@wdc.com \
--cc=dm-devel@lists.linux.dev \
--cc=houtao1@huawei.com \
--cc=ignat@cloudflare.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=msnitzer@redhat.com \
--cc=nhuck@google.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).