From: mwilck@suse.com
To: Christophe Varoqui <christophe.varoqui@opensvc.com>,
Benjamin Marzinski <bmarzins@redhat.com>
Cc: dm-devel@lists.linux.dev, Martin Wilck <mwilck@suse.com>,
Li Xiao Keng <lixiaokeng@huawei.com>,
Miao Guanqin <miaoguanqin@huawei.com>,
Guan Junxiong <guanjunxiong@huawei.com>
Subject: [PATCH v2 06/14] libmultipath: io_err_stat: use higher number of aio slots
Date: Thu, 26 Oct 2023 19:41:45 +0200 [thread overview]
Message-ID: <20231026174153.1133-7-mwilck@suse.com> (raw)
In-Reply-To: <20231026174153.1133-1-mwilck@suse.com>
From: Martin Wilck <mwilck@suse.com>
Currently the number of iocbs per path to test is the same as the
total number of iocbs in the ioctx. This can easily cause iocb
starvation, in particular if some IOs are hanging. In that case
io_submit() will fail, and some paths under test will use much
less IOs as intended, or in the worst case, none at all.
The total number of iocbs reserved in the kernel should be higher.
With this patch, we will be able to run the marginal path test for
at least NR_IOSTAT_PATHS=32 paths at the same time. This is not an upper
limit, because kernel IOCBs can be reused between paths.
Increase the log levels of io_setup and io_submit to make it sure
we catch problems with this approach.
Signed-off-by: Martin Wilck <mwilck@suse.com>
Cc: Li Xiao Keng <lixiaokeng@huawei.com>
Cc: Miao Guanqin <miaoguanqin@huawei.com>
Cc: Guan Junxiong <guanjunxiong@huawei.com>
---
libmultipath/io_err_stat.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/libmultipath/io_err_stat.c b/libmultipath/io_err_stat.c
index 3f32e32..5749003 100644
--- a/libmultipath/io_err_stat.c
+++ b/libmultipath/io_err_stat.c
@@ -38,6 +38,7 @@
#define TIMEOUT_NO_IO_NSEC 10000000 /*10ms = 10000000ns*/
#define FLAKY_PATHFAIL_THRESHOLD 2
#define CONCUR_NR_EVENT 32
+#define NR_IOSTAT_PATHS 32
#define PATH_IO_ERR_IN_CHECKING -1
#define PATH_IO_ERR_WAITING_TO_CHECK -2
@@ -477,7 +478,7 @@ static int send_each_async_io(struct dio_ctx *ct, int fd, char *dev)
get_monotonic_time(&ct->io_starttime);
io_prep_pread(&ct->io, fd, ct->buf, ct->blksize, 0);
if (io_submit(ioctx, 1, ios) != 1) {
- io_err_stat_log(5, "%s: io_submit error %i",
+ io_err_stat_log(2, "%s: io_submit error %i",
dev, errno);
return rc;
}
@@ -703,8 +704,8 @@ int start_io_err_stat_thread(void *data)
if (uatomic_read(&io_err_thread_running) == 1)
return 0;
- if (io_setup(CONCUR_NR_EVENT, &ioctx) != 0) {
- io_err_stat_log(4, "io_setup failed");
+ if (io_setup(CONCUR_NR_EVENT * NR_IOSTAT_PATHS, &ioctx) != 0) {
+ io_err_stat_log(1, "io_setup failed - increase /proc/sys/fs/aio-nr ?");
return 1;
}
--
2.42.0
next prev parent reply other threads:[~2023-10-26 17:42 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-26 17:41 [PATCH v2 00/14] multipath: aio, systemd, and documentation improvements mwilck
2023-10-26 17:41 ` [PATCH v2 01/14] libmultipath: reduce log level of directio messages mwilck
2023-10-26 17:41 ` [PATCH v2 02/14] libmultipath: directio: don't reset ct->running after io_cancel() mwilck
2023-10-27 18:54 ` Benjamin Marzinski
2023-10-26 17:41 ` [PATCH v2 03/14] libmultipath: directio: fix error handling mwilck
2023-10-27 18:54 ` Benjamin Marzinski
2023-10-26 17:41 ` [PATCH v2 04/14] libmultipath: io_err_stat: don't free aio memory before completion mwilck
2023-10-27 18:55 ` Benjamin Marzinski
2023-10-26 17:41 ` [PATCH v2 05/14] libmultipath: io_err_stat: call io_destroy() inside free_io_err_pathvec() mwilck
2023-10-27 18:55 ` Benjamin Marzinski
2023-10-26 17:41 ` mwilck [this message]
2023-10-27 18:56 ` [PATCH v2 06/14] libmultipath: io_err_stat: use higher number of aio slots Benjamin Marzinski
2023-10-26 17:41 ` [PATCH v2 07/14] libmultipath: io_err_stat: fix error handling mwilck
2023-10-27 19:09 ` Benjamin Marzinski
2023-10-26 17:41 ` [PATCH v2 08/14] multipathd.service: require modprobe@dm_multipath.service if available mwilck
2023-10-27 19:09 ` Benjamin Marzinski
2023-10-27 20:05 ` Benjamin Marzinski
2023-10-30 11:45 ` Martin Wilck
2023-10-26 17:41 ` [PATCH v2 09/14] libmpathutil: remove systemd_service_enabled() mwilck
2023-10-26 17:41 ` [PATCH v2 10/14] multipath.conf.5: fix typo mwilck
2023-10-27 19:10 ` Benjamin Marzinski
2023-10-26 17:41 ` [PATCH v2 11/14] Makefile.inc, README.md: fix docs for prefix in split-usr case mwilck
2023-10-27 19:27 ` Benjamin Marzinski
2023-10-26 17:41 ` [PATCH v2 12/14] README.md: update mailing list and contributing information mwilck
2023-10-27 19:29 ` Benjamin Marzinski
2023-10-30 11:41 ` Martin Wilck
2023-10-26 17:41 ` [PATCH v2 13/14] README.md: Extend the section about NVMe mwilck
2023-10-27 19:34 ` Benjamin Marzinski
2023-10-26 17:41 ` [PATCH v2 14/14] README.md: fix formatting of Changelog section mwilck
2023-10-27 19:35 ` Benjamin Marzinski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231026174153.1133-7-mwilck@suse.com \
--to=mwilck@suse.com \
--cc=bmarzins@redhat.com \
--cc=christophe.varoqui@opensvc.com \
--cc=dm-devel@lists.linux.dev \
--cc=guanjunxiong@huawei.com \
--cc=lixiaokeng@huawei.com \
--cc=miaoguanqin@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.