From: Muhammad Usama Anjum <usama.anjum@collabora.com>
To: "Shuah Khan" <shuah@kernel.org>,
gustavo.padovan@collabora.com,
"open list : KERNEL SELFTEST FRAMEWORK"
<linux-kselftest@vger.kernel.org>,
"Laura Nao" <laura.nao@collabora.com>,
"Shreeya Patel" <shreeya.patel@collabora.com>,
"Nícolas F. R. A. Prado" <nfraprado@collabora.com>,
"Helen Mae Koike Fornazier" <helen.koike@collabora.com>
Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>,
automated-testing@lists.yoctoproject.org,
"kernel@collabora.com" <kernel@collabora.com>,
kernelci@lists.linux.dev,
open list <linux-kernel@vger.kernel.org>
Subject: Improving the testing quality of the kernel
Date: Tue, 23 Apr 2024 20:17:18 +0500 [thread overview]
Message-ID: <02c3926c-84fe-4526-a746-6964f5290c14@collabora.com> (raw)
Hi,
I've been working with colleagues at Collabora to improve the testing
quality overall for kselftests since several months. We had identified
following key points to improve:
* Make non-TAP conformant tests, TAP conformant to catch the bugs/failures
in the CI and non-CI environment. Without clear success or failure message,
it is difficult and at times impossible to notice the bugs and which part
of the test or sub-test has failed.
* Add config fragment if it is already isn't present.
* Improve ancient tests to be robust. KernelCI helps a lot in finding
flakiness or non-robust pieces of code by building it in range of
configurations and running it on range of hardware.
As new KernelCI is in works, we are identifying which kselftest suites
could be most suitable to be enabled in the start to keep the noise to a
minimum. The criteria to enable a suite on KernelCI is:
* The test suite is TAP compliant
* The test suite passes in preliminary testing on x86_64 and ARM64 platforms
* The test suite builds fine with Clang in-addition to gcc
To facilitate transparency and track our progress, I am diligently
maintaining a spreadsheet [1] with detailed information about each suite's
status and requirements.
Furthermore, I have commenced exploration into KUnit testing, and initial
results are promising. I plan to delve deeper into KUnit testing and will
provide further updates in the coming days.
Your feedback, insights and collaboration on any aspect of these
initiatives would be highly valuable. We greatly appreciate your
collaboration and support as we continue to enhance the testing quality of
kselftests.
[1]
https://docs.google.com/spreadsheets/d/1XRCgxlY1b74aIOIXQ7qDR-62KTJmuV1ow9st4fpeVhA/edit?usp=sharing
--
BR,
Muhammad Usama Anjum
reply other threads:[~2024-04-23 15:16 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=02c3926c-84fe-4526-a746-6964f5290c14@collabora.com \
--to=usama.anjum@collabora.com \
--cc=automated-testing@lists.yoctoproject.org \
--cc=gustavo.padovan@collabora.com \
--cc=helen.koike@collabora.com \
--cc=kernel@collabora.com \
--cc=kernelci@lists.linux.dev \
--cc=laura.nao@collabora.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=nfraprado@collabora.com \
--cc=shreeya.patel@collabora.com \
--cc=shuah@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).