From: "Matthieu Baerts (NGI0)" <matttbe@kernel.org>
To: mptcp@lists.linux.dev
Cc: "Matthieu Baerts (NGI0)" <matttbe@kernel.org>
Subject: [PATCH mptcp-net v3 0/4] selftests: mptcp: mark unstable subtests as flaky
Date: Wed, 22 May 2024 12:47:42 +0200 [thread overview]
Message-ID: <20240522-selftests-mptcp-disable-flaky-v3-0-a10c68bf2680@kernel.org> (raw)
Some subtests can be unstable, failing once every X runs. Fixing them
can take time: there could be an issue in the kernel or in the subtest,
and it is then important to do a proper analysis, not to hide real bugs.
To avoid creating noises on the different CIs, some subtests have been
marked as flaky. As a result, errors with these subtests (if any) are
ignored.
Note that the MPTCP CI will continue to track these flaky subtests. All
these unstable subtests are also tracked by our bug tracker.
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
Changes in v3:
- Patch 1/4:
- use SELFTESTS_MPTCP_LIB_OVERRIDE_FLAKY env var (Mat)
- clearly indicate 'flaky' in the output / TAP format
- Target mptcp-net: these are "fixes" for the selftests. I don't think
we need to add Fixes tag, right?
- Link to v2: https://lore.kernel.org/r/20240521-selftests-mptcp-disable-flaky-v2-0-eba143bcbad0@kernel.org
Changes in v2:
- Patch 2/4: only mark "unbalanced bwidth" subtest as flaky (Paolo)
- Link to v1: https://lore.kernel.org/r/20240520-selftests-mptcp-disable-flaky-v1-0-02e23ba0bc3b@kernel.org
---
Matthieu Baerts (NGI0) (4):
selftests: mptcp: lib: support flaky subtests
selftests: mptcp: simult flows: mark 'unbalanced' tests as flaky
selftests: mptcp: join: mark 'fastclose' tests as flaky
selftests: mptcp: join: mark 'fail' tests as flaky
tools/testing/selftests/net/mptcp/mptcp_join.sh | 10 +++++++-
tools/testing/selftests/net/mptcp/mptcp_lib.sh | 30 +++++++++++++++++++++--
tools/testing/selftests/net/mptcp/simult_flows.sh | 6 ++---
3 files changed, 40 insertions(+), 6 deletions(-)
---
base-commit: d048726b432ccff9b4e3d8b0012a7328c5c7c2f6
change-id: 20240520-selftests-mptcp-disable-flaky-380759d6503c
Best regards,
--
Matthieu Baerts (NGI0) <matttbe@kernel.org>
next reply other threads:[~2024-05-22 10:47 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-22 10:47 Matthieu Baerts (NGI0) [this message]
2024-05-22 10:47 ` [PATCH mptcp-net v3 1/4] selftests: mptcp: lib: support flaky subtests Matthieu Baerts (NGI0)
2024-05-22 10:47 ` [PATCH mptcp-net v3 2/4] selftests: mptcp: simult flows: mark 'unbalanced' tests as flaky Matthieu Baerts (NGI0)
2024-05-22 10:47 ` [PATCH mptcp-net v3 3/4] selftests: mptcp: join: mark 'fastclose' " Matthieu Baerts (NGI0)
2024-05-22 10:47 ` [PATCH mptcp-net v3 4/4] selftests: mptcp: join: mark 'fail' " Matthieu Baerts (NGI0)
2024-05-22 11:42 ` [PATCH mptcp-net v3 0/4] selftests: mptcp: mark unstable subtests " MPTCP CI
2024-05-22 15:38 ` Mat Martineau
2024-05-23 10:10 ` Matthieu Baerts
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240522-selftests-mptcp-disable-flaky-v3-0-a10c68bf2680@kernel.org \
--to=matttbe@kernel.org \
--cc=mptcp@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).