From: Jonathan Cameron <Jonathan.Cameron@Huawei.com>
To: Dave Jiang <dave.jiang@intel.com>
Cc: <linux-cxl@vger.kernel.org>, <dan.j.williams@intel.com>,
<ira.weiny@intel.com>, <vishal.l.verma@intel.com>,
<alison.schofield@intel.com>, <dave@stgolabs.net>
Subject: Re: [PATCH v7 3/5] cxl: Fix incorrect region perf data calculation
Date: Fri, 5 Apr 2024 14:48:24 +0100 [thread overview]
Message-ID: <20240405144824.00007e34@Huawei.com> (raw)
In-Reply-To: <20240403154844.3403859-4-dave.jiang@intel.com>
On Wed, 3 Apr 2024 08:47:14 -0700
Dave Jiang <dave.jiang@intel.com> wrote:
> Current math in cxl_region_perf_data_calculate divides the latency by 1000
> every time the function gets called. This causes the region latency to be
> divided by 1000 per memory device and the math is incorrect. This is user
> visible as the latency access_coordinate exposed via sysfs will show
> incorrect latency data.
>
> Normalize values from CDAT to nanoseconds. Adjust sub-nanoseconds latency
> to at least 1. Remove adjustment of perf numbers from the generic target
> since hmat handling code has already normalized those numbers. Now all
> computation and stored numbers should be in nanoseconds.
>
> cxl_hb_get_perf_coordinates() is removed and HB coords are calculated
> in the port access_coordinate calculation path since it no longer need
> to be treated special.
>
> Fixes: 3d9f4a197230 ("cxl/region: Calculate performance data for a region")
> Signed-off-by: Dave Jiang <dave.jiang@intel.com>
I should stop reading this code...
What happens with the bandwidth if the minimum point on path to EP is shared?
Gets more complex as maybe the shared bit wasn't the minimum bandwidth previously
but when it's 'split' across multiple paths it becomes so.
E.g. HEre the min on each path is 5, but the bottleneck is actually the RP to
switch at 8 once we are interleaving across EP0 and EP1.
CPU
|
HB
|
RP
|
<min BW here = 8>
|
SWITCH
| |
<each of these BW 5>
EP0 EP1
None of this mattered with traditional HMAT entries because they
are point to point so if such interleaving is going on it was
a problem for the BIOS writer...
Not related to what you are fixing here though so
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> @@ -521,17 +525,13 @@ void cxl_region_perf_data_calculate(struct cxl_region *cxlr,
> struct cxl_endpoint_decoder *cxled)
> {
> struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
> - struct cxl_port *port = cxlmd->endpoint;
> struct cxl_dev_state *cxlds = cxlmd->cxlds;
> struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlds);
> - struct access_coordinate hb_coord[ACCESS_COORDINATE_MAX];
> - struct access_coordinate coord;
> struct range dpa = {
> .start = cxled->dpa_res->start,
> .end = cxled->dpa_res->end,
> };
> struct cxl_dpa_perf *perf;
> - int rc;
>
> switch (cxlr->mode) {
> case CXL_DECODER_RAM:
> @@ -549,35 +549,16 @@ void cxl_region_perf_data_calculate(struct cxl_region *cxlr,
> if (!range_contains(&perf->dpa_range, &dpa))
> return;
>
> - rc = cxl_hb_get_perf_coordinates(port, hb_coord);
> - if (rc) {
> - dev_dbg(&port->dev, "Failed to retrieve hb perf coordinates.\n");
> - return;
> - }
> -
> for (int i = 0; i < ACCESS_COORDINATE_MAX; i++) {
> - /* Pickup the host bridge coords */
> - cxl_coordinates_combine(&coord, &hb_coord[i], &perf->coord);
> -
> /* Get total bandwidth and the worst latency for the cxl region */
Worst latency from what set of choices? Perhaps useful to call that out (multiple EP
paths?)
> cxlr->coord[i].read_latency = max_t(unsigned int,
> cxlr->coord[i].read_latency,
> - coord.read_latency);
> + perf->coord.read_latency);
> cxlr->coord[i].write_latency = max_t(unsigned int,
> cxlr->coord[i].write_latency,
> - coord.write_latency);
> - cxlr->coord[i].read_bandwidth += coord.read_bandwidth;
> - cxlr->coord[i].write_bandwidth += coord.write_bandwidth;
> -
> - /*
> - * Convert latency to nanosec from picosec to be consistent
> - * with the resulting latency coordinates computed by the
> - * HMAT_REPORTING code.
> - */
> - cxlr->coord[i].read_latency =
> - DIV_ROUND_UP(cxlr->coord[i].read_latency, 1000);
> - cxlr->coord[i].write_latency =
> - DIV_ROUND_UP(cxlr->coord[i].write_latency, 1000);
> + perf->coord.write_latency);
> + cxlr->coord[i].read_bandwidth += perf->coord.read_bandwidth;
> + cxlr->coord[i].write_bandwidth += perf->coord.write_bandwidth;
As above, this might be the same bandwidth we are double counting...
> }
> }
>
>
next prev parent reply other threads:[~2024-04-05 13:48 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-03 15:47 [PATCH v7 0/5] cxl: access_coordinate validity fixes for 6.9 Dave Jiang
2024-04-03 15:47 ` [PATCH v7 1/5] cxl: Remove checking of iter in cxl_endpoint_get_perf_coordinates() Dave Jiang
2024-04-26 19:10 ` Robert Richter
2024-04-26 20:13 ` Dave Jiang
2024-04-03 15:47 ` [PATCH v7 2/5] cxl: Fix retrieving of access_coordinates in PCIe path Dave Jiang
2024-04-05 13:32 ` Jonathan Cameron
2024-04-05 22:24 ` Dan Williams
2024-04-03 15:47 ` [PATCH v7 3/5] cxl: Fix incorrect region perf data calculation Dave Jiang
2024-04-05 13:48 ` Jonathan Cameron [this message]
2024-04-05 22:34 ` Dan Williams
2024-04-08 9:54 ` Jonathan Cameron
2024-04-03 15:47 ` [PATCH v7 4/5] cxl: Consolidate dport access_coordinate ->hb_coord and ->sw_coord into ->coord Dave Jiang
2024-04-03 15:47 ` [PATCH v7 5/5] cxl: Add checks to access_coordinate calculation to fail missing data Dave Jiang
2024-04-05 22:36 ` Dan Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240405144824.00007e34@Huawei.com \
--to=jonathan.cameron@huawei.com \
--cc=alison.schofield@intel.com \
--cc=dan.j.williams@intel.com \
--cc=dave.jiang@intel.com \
--cc=dave@stgolabs.net \
--cc=ira.weiny@intel.com \
--cc=linux-cxl@vger.kernel.org \
--cc=vishal.l.verma@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.