From: "Wang, Weilin" <weilin.wang@intel.com>
To: Ravi Bangoria <ravi.bangoria@amd.com>
Cc: Kan Liang <kan.liang@linux.intel.com>,
"Alt, Samantha" <samantha.alt@intel.com>,
"Taylor, Perry" <perry.taylor@intel.com>,
"Biggers, Caleb" <caleb.biggers@intel.com>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Jiri Olsa <jolsa@kernel.org>, Namhyung Kim <namhyung@kernel.org>,
"Hunter, Adrian" <adrian.hunter@intel.com>,
Ian Rogers <irogers@google.com>,
"linux-perf-users@vger.kernel.org"
<linux-perf-users@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH v3 1/3] perf test: Add metric value validation test
Date: Sun, 18 Jun 2023 17:43:12 +0000 [thread overview]
Message-ID: <CO6PR11MB56353207BBD7BDA6C530EBDEEE5EA@CO6PR11MB5635.namprd11.prod.outlook.com> (raw)
In-Reply-To: <bc18bc42-3df8-30c9-10ff-8a4ba3d3d9f6@amd.com>
> -----Original Message-----
> From: Ravi Bangoria <ravi.bangoria@amd.com>
> Sent: Friday, June 16, 2023 12:03 AM
> To: Wang, Weilin <weilin.wang@intel.com>
> Cc: Kan Liang <kan.liang@linux.intel.com>; Alt, Samantha
> <samantha.alt@intel.com>; Taylor, Perry <perry.taylor@intel.com>; Biggers,
> Caleb <caleb.biggers@intel.com>; Peter Zijlstra <peterz@infradead.org>; Ingo
> Molnar <mingo@redhat.com>; Arnaldo Carvalho de Melo <acme@kernel.org>;
> Jiri Olsa <jolsa@kernel.org>; Namhyung Kim <namhyung@kernel.org>; Hunter,
> Adrian <adrian.hunter@intel.com>; Ian Rogers <irogers@google.com>; linux-
> perf-users@vger.kernel.org; linux-kernel@vger.kernel.org; Ravi Bangoria
> <ravi.bangoria@amd.com>
> Subject: Re: [PATCH v3 1/3] perf test: Add metric value validation test
>
> Hi,
>
> > diff --git a/tools/perf/tests/shell/stat_metrics_values.sh
> b/tools/perf/tests/shell/stat_metrics_values.sh
> > new file mode 100755
> > index 000000000000..65a15c65eea7
> > --- /dev/null
> > +++ b/tools/perf/tests/shell/stat_metrics_values.sh
> > @@ -0,0 +1,30 @@
> > +#!/bin/bash
> > +# perf metrics value validation
> > +# SPDX-License-Identifier: GPL-2.0
> > +if [ "x$PYTHON" == "x" ]
> > +then
> > + if which python3 > /dev/null
> > + then
> > + PYTHON=python3
> > + else
> > + echo Skipping test, python3 not detected please set
> environment variable PYTHON.
> > + exit 2
> > + fi
> > +fi
> > +
> > +grep -q Intel /proc/cpuinfo || (echo Skipping non-Intel; exit 2)
>
> This check doesn't seem to be working. On my Zen3 AMD machine:
Thanks for reporting this! I've update this search to "GenuineIntel" in v4 to help solve this issue.
Please check it out.
>
> $ sudo ./perf test -vvv 105
> 105: perf metrics value validation :
> --- start ---
> test child forked, pid 518035
> Skipping non-Intel
> Launch python validation script ./tests/shell/lib/perf_metric_validation.py
> Output will be stored in: /tmp/__perf_test.program.tnPoW
> Starting perf collection
> ...
>
> Interestingly, it passes :)
>
> ...
> Test validation finished. Final report:
> [
> {
> "Workload": "perf bench futex hash -r 2 -s",
> "Report": {
> "Metric Validation Statistics": {
> "Total Rule Count": 2,
> "Passed Rule Count": 2
> },
> "Tests in Category": {
> "PositiveValueTest": {
> "Total Tests": 12,
> "Passed Tests": 12,
> "Failed Tests": []
> },
> "RelationshipTest": {
> "Total Tests": 0,
> "Passed Tests": 0,
> "Failed Tests": []
> },
> "SingleMetricTest": {
> "Total Tests": 2,
> "Passed Tests": 2,
> "Failed Tests": []
> }
> },
> "Errors": []
> }
> }
> ]
> test child finished with 0
> ---- end ----
> perf metrics value validation: Ok
>
> I haven't yet investigated whether it's genuine or false positive.
>
Base on this final report, it validated some basic rules like the 12 metrics for positive value test and 2 metrics for single metric value checks.
The test script grabs metrics supported on the platform and generates validation rules that only include metrics in the supported list.
Therefore, it is not surprising that the check passes on your system.
Thanks,
Weilin
> Thanks,
> Ravi
next prev parent reply other threads:[~2023-06-18 17:43 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-09 16:26 [PATCH v2 0/3] Add metric value validation test Weilin Wang
2023-06-09 16:26 ` [PATCH v2 1/3] perf test: " Weilin Wang
2023-06-09 16:26 ` [PATCH v2 2/3] perf test: Add skip list for metrics known would fail Weilin Wang
2023-06-09 16:26 ` [PATCH v2 3/3] perf test: Rerun failed metrics with longer workload Weilin Wang
2023-06-13 0:04 ` [PATCH v2 0/3] Add metric value validation test Ian Rogers
2023-06-14 20:38 ` [PATCH v3 " Weilin Wang
2023-06-14 20:38 ` [PATCH v3 1/3] perf test: " Weilin Wang
2023-06-16 7:02 ` Ravi Bangoria
2023-06-18 17:43 ` Wang, Weilin [this message]
2023-06-19 4:20 ` Ravi Bangoria
2023-06-14 20:38 ` [PATCH v3 2/3] perf test: Add skip list for metrics known would fail Weilin Wang
2023-06-14 20:38 ` [PATCH v3 3/3] perf test: Rerun failed metrics with longer workload Weilin Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CO6PR11MB56353207BBD7BDA6C530EBDEEE5EA@CO6PR11MB5635.namprd11.prod.outlook.com \
--to=weilin.wang@intel.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=caleb.biggers@intel.com \
--cc=irogers@google.com \
--cc=jolsa@kernel.org \
--cc=kan.liang@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=perry.taylor@intel.com \
--cc=peterz@infradead.org \
--cc=ravi.bangoria@amd.com \
--cc=samantha.alt@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).