From: Alistair Popple <apopple@nvidia.com>
To: "Huang, Ying" <ying.huang@intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev,
linux-acpi@vger.kernel.org,
"Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>,
Wei Xu <weixugc@google.com>,
Dan Williams <dan.j.williams@intel.com>,
Dave Hansen <dave.hansen@intel.com>,
Davidlohr Bueso <dave@stgolabs.net>,
Johannes Weiner <hannes@cmpxchg.org>,
Jonathan Cameron <Jonathan.Cameron@huawei.com>,
Michal Hocko <mhocko@kernel.org>, Yang Shi <shy828301@gmail.com>,
Rafael J Wysocki <rafael.j.wysocki@intel.com>,
Dave Jiang <dave.jiang@intel.com>
Subject: Re: [PATCH RESEND 1/4] memory tiering: add abstract distance calculation algorithms management
Date: Tue, 25 Jul 2023 18:26:15 +1000 [thread overview]
Message-ID: <87sf9cxupz.fsf@nvdebian.thelocal> (raw)
In-Reply-To: <87r0owy95t.fsf@yhuang6-desk2.ccr.corp.intel.com>
"Huang, Ying" <ying.huang@intel.com> writes:
> Hi, Alistair,
>
> Thanks a lot for comments!
>
> Alistair Popple <apopple@nvidia.com> writes:
>
>> Huang Ying <ying.huang@intel.com> writes:
>>
>>> The abstract distance may be calculated by various drivers, such as
>>> ACPI HMAT, CXL CDAT, etc. While it may be used by various code which
>>> hot-add memory node, such as dax/kmem etc. To decouple the algorithm
>>> users and the providers, the abstract distance calculation algorithms
>>> management mechanism is implemented in this patch. It provides
>>> interface for the providers to register the implementation, and
>>> interface for the users.
>>
>> I wonder if we need this level of decoupling though? It seems to me like
>> it would be simpler and better for drivers to calculate the abstract
>> distance directly themselves by calling the desired algorithm (eg. ACPI
>> HMAT) and pass this when creating the nodes rather than having a
>> notifier chain.
>
> Per my understanding, ACPI HMAT and memory device drivers (such as
> dax/kmem) may belong to different subsystems (ACPI vs. dax). It's not
> good to call functions across subsystems directly. So, I think it's
> better to use a general subsystem: memory-tier.c to decouple them. If
> it turns out that a notifier chain is unnecessary, we can use some
> function pointers instead.
>
>> At the moment it seems we've only identified two possible algorithms
>> (ACPI HMAT and CXL CDAT) and I don't think it would make sense for one
>> of those to fallback to the other based on priority, so why not just
>> have drivers call the correct algorithm directly?
>
> For example, we have a system with PMEM (persistent memory, Optane
> DCPMM, or AEP, or something else) in DIMM slots and CXL.mem connected
> via CXL link to a remote memory pool. We will need ACPI HMAT for PMEM
> and CXL CDAT for CXL.mem. One way is to make dax/kmem identify the
> types of the device and call corresponding algorithms.
Yes, that is what I was thinking.
> The other way (suggested by this series) is to make dax/kmem call a
> notifier chain, then CXL CDAT or ACPI HMAT can identify the type of
> device and calculate the distance if the type is correct for them. I
> don't think that it's good to make dax/kem to know every possible
> types of memory devices.
Do we expect there to be lots of different types of memory devices
sharing a common dax/kmem driver though? Must admit I'm coming from a
GPU background where we'd expect each type of device to have it's own
driver anyway so wasn't expecting different types of memory devices to
be handled by the same driver.
>>> Multiple algorithm implementations can cooperate via calculating
>>> abstract distance for different memory nodes. The preference of
>>> algorithm implementations can be specified via
>>> priority (notifier_block.priority).
>>
>> How/what decides the priority though? That seems like something better
>> decided by a device driver than the algorithm driver IMHO.
>
> Do we need the memory device driver specific priority? Or we just share
> a common priority? For example, the priority of CXL CDAT is always
> higher than that of ACPI HMAT? Or architecture specific?
Ok, thanks. Having read the above I think the priority is
unimportant. Algorithms can either decide to return a distance and
NOTIFY_STOP_MASK if they can calculate a distance or NOTIFY_DONE if they
can't for a specific device.
> And, I don't think that we are forced to use the general notifier
> chain interface in all memory device drivers. If the memory device
> driver has better understanding of the memory device, it can use other
> way to determine abstract distance. For example, a CXL memory device
> driver can identify abstract distance by itself. While other memory
> device drivers can use the general notifier chain interface at the
> same time.
Whilst I think personally I would find that flexibility useful I am
concerned it means every driver will just end up divining it's own
distance rather than ensuring data in HMAT/CDAT/etc. is correct. That
would kind of defeat the purpose of it all then.
next prev parent reply other threads:[~2023-07-25 8:26 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-07-21 1:29 [PATCH RESEND 0/4] memory tiering: calculate abstract distance based on ACPI HMAT Huang Ying
2023-07-21 1:29 ` [PATCH RESEND 1/4] memory tiering: add abstract distance calculation algorithms management Huang Ying
2023-07-25 2:13 ` Alistair Popple
2023-07-25 3:14 ` Huang, Ying
2023-07-25 8:26 ` Alistair Popple [this message]
2023-07-26 7:33 ` Huang, Ying
2023-07-27 3:42 ` Alistair Popple
2023-07-27 4:02 ` Huang, Ying
2023-07-27 4:07 ` Alistair Popple
2023-07-27 5:41 ` Huang, Ying
2023-07-28 1:20 ` Alistair Popple
2023-08-11 3:51 ` Huang, Ying
2023-08-21 11:26 ` Alistair Popple
2023-08-21 22:50 ` Huang, Ying
2023-08-21 23:52 ` Alistair Popple
2023-08-22 0:58 ` Huang, Ying
2023-08-22 7:11 ` Alistair Popple
2023-08-23 5:56 ` Huang, Ying
2023-08-25 5:41 ` Alistair Popple
2023-07-21 1:29 ` [PATCH RESEND 2/4] acpi, hmat: refactor hmat_register_target_initiators() Huang Ying
2023-07-25 2:44 ` Alistair Popple
2023-08-07 16:55 ` Jonathan Cameron
2023-08-11 1:13 ` Huang, Ying
2023-07-21 1:29 ` [PATCH RESEND 3/4] acpi, hmat: calculate abstract distance with HMAT Huang Ying
2023-07-25 2:45 ` Alistair Popple
2023-07-25 6:47 ` Huang, Ying
2023-08-21 11:53 ` Alistair Popple
2023-08-21 23:28 ` Huang, Ying
2023-07-21 1:29 ` [PATCH RESEND 4/4] dax, kmem: calculate abstract distance with general interface Huang Ying
2023-07-25 3:11 ` Alistair Popple
2023-07-25 7:02 ` Huang, Ying
2023-08-21 12:03 ` Alistair Popple
2023-08-21 23:33 ` Huang, Ying
2023-08-22 7:36 ` Alistair Popple
2023-08-23 2:13 ` Huang, Ying
2023-08-25 6:00 ` Alistair Popple
2023-07-21 4:15 ` [PATCH RESEND 0/4] memory tiering: calculate abstract distance based on ACPI HMAT Alistair Popple
2023-07-24 17:58 ` Andrew Morton
2023-08-01 2:35 ` Bharata B Rao
2023-08-11 6:26 ` Huang, Ying
2023-08-11 7:49 ` Bharata B Rao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87sf9cxupz.fsf@nvdebian.thelocal \
--to=apopple@nvidia.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=akpm@linux-foundation.org \
--cc=aneesh.kumar@linux.ibm.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@intel.com \
--cc=dave.jiang@intel.com \
--cc=dave@stgolabs.net \
--cc=hannes@cmpxchg.org \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=nvdimm@lists.linux.dev \
--cc=rafael.j.wysocki@intel.com \
--cc=shy828301@gmail.com \
--cc=weixugc@google.com \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).