From: "Huang, Ying" <ying.huang@intel.com>
To: <sthanneeru.opensrc@micron.com>
Cc: <aneesh.kumar@linux.ibm.com>, <hannes@cmpxchg.org>,
<hasanalmaruf@fb.com>, <haowang3@fb.com>,
<gregory.price@memverge.com>, <dan.j.williams@intel.com>,
<mhocko@suse.com>, <tj@kernel.org>,
<hezhongkun.hzk@bytedance.com>, <fvdl@google.com>,
<john@jagalactic.com>, <mirakhur@micron.com>,
<vtavarespetr@micron.com>, <Ravis.OpenSrc@micron.com>,
<Jonathan.Cameron@huawei.com>, <linux-kernel@vger.kernel.org>,
<linux-api@vger.kernel.org>, Johannes Weiner <hannes@cmpxchg.org>
Subject: Re: [RFC PATCH 0/2] Node migration between memory tiers
Date: Mon, 04 Dec 2023 16:52:12 +0800 [thread overview]
Message-ID: <87o7f62vur.fsf@yhuang6-desk2.ccr.corp.intel.com> (raw)
In-Reply-To: <20231130214858.1887-1-sthanneeru.opensrc@micron.com> (sthanneeru opensrc's message of "Fri, 1 Dec 2023 03:18:56 +0530")
<sthanneeru.opensrc@micron.com> writes:
> From: Srinivasulu Thanneeru <sthanneeru.opensrc@micron.com>
>
> The memory tiers feature allows nodes with similar memory types
> or performance characteristics to be grouped together in a
> memory tier. However, there is currently no provision for
> moving a node from one tier to another on demand.
>
> This patch series aims to support node migration between tiers
> on demand by sysadmin/root user using the provided sysfs for
> node migration. Each tier has a start abstract distance(adistance)
> and range.
We have discussed migrating nodes (in fact nodes of a memory type)
between tiers by sysadmin/root before. The basic idea comes from
Johannes. It is summarized in page 11 of [1],
[1] https://lpc.events/event/16/contributions/1209/attachments/1042/1995/Live%20In%20a%20World%20With%20Multiple%20Memory%20Types.pdf
The abstract distance of a memory type (e.g., GPU HBM) can be adjusted
via a sysfs knob (<memory_type>/abstract_distance_offset).
I still think that the memory type is better to be used to change
the abstract distance of nodes. Do you agree?
--
Best Regards,
Huang, Ying
> To migrate a node to a tier, the corresponding node’s sysfs
> adistance_offset is written with a value corresponding to
> the tier’s adistance.
>
> Example: Move node2 to memory tier5 from its default tier(i.e 4)
>
> 1. Check default values:
> $cat /sys/devices/virtual/memory_tiering/memory_tier4/nodelist
> 0-2
>
> $cat /sys/devices/system/node/node0/adistance_offset
> 0
> $cat /sys/devices/system/node/node1/adistance_offset
> 0
> $cat /sys/devices/system/node/node2/adistance_offset
> 0
>
> 2. Move node2 to tier5:
>
> To move node2 from emory_tier4 (adistance=512) to
> emory_tier5 (abstract=640), set the `adistance_offset` of
> node 2 to 128 (i.e., 512 + 128 = 640).
>
> Tier4 adistance start can be derved from tier-id
> (i.e for tier4, 4 << 7 = 512).
>
> $echo 128 > /sys/devices/system/node/node2/adistance_offset
> $cat /sys/devices/system/node/node2/adistance_offset
> 128
>
> 3. Verify node2's tier id:
>
> $cat /sys/devices/virtual/memory_tiering/memory_tier5/nodelist
> 2
> $cat /sys/devices/virtual/memory_tiering/memory_tier4/nodelist
> 0-1
>
> Srinivasulu Thanneeru (2):
> base/node: Add sysfs for adistance_offset
> memory tier: Support node migration between tiers
>
> drivers/base/node.c | 51 +++++++++++++++++++++++
> include/linux/memory-tiers.h | 11 +++++
> include/linux/node.h | 6 +++
> mm/memory-tiers.c | 79 ++++++++++++++++++++----------------
> 4 files changed, 113 insertions(+), 34 deletions(-)
next prev parent reply other threads:[~2023-12-04 8:54 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-11-30 21:48 [RFC PATCH 0/2] Node migration between memory tiers sthanneeru.opensrc
2023-11-30 21:48 ` [RFC PATCH 1/2] base/node: Add sysfs for adistance_offset sthanneeru.opensrc
2023-11-30 21:48 ` [RFC PATCH 2/2] memory tier: Support node migration between tiers sthanneeru.opensrc
2023-12-04 8:52 ` Huang, Ying [this message]
-- strict thread matches above, loose matches on Subject: below --
2023-11-30 22:04 [RFC PATCH 0/2] Node migration between memory tiers sthanneeru.opensrc
2023-12-04 15:43 ` Michal Hocko
2023-12-05 8:51 [EXT] " Michal Hocko
2023-12-05 9:19 ` Ravi Jonnalagadda
2023-12-06 15:22 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87o7f62vur.fsf@yhuang6-desk2.ccr.corp.intel.com \
--to=ying.huang@intel.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=Ravis.OpenSrc@micron.com \
--cc=aneesh.kumar@linux.ibm.com \
--cc=dan.j.williams@intel.com \
--cc=fvdl@google.com \
--cc=gregory.price@memverge.com \
--cc=hannes@cmpxchg.org \
--cc=haowang3@fb.com \
--cc=hasanalmaruf@fb.com \
--cc=hezhongkun.hzk@bytedance.com \
--cc=john@jagalactic.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mhocko@suse.com \
--cc=mirakhur@micron.com \
--cc=sthanneeru.opensrc@micron.com \
--cc=tj@kernel.org \
--cc=vtavarespetr@micron.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).