From: Pasha Tatashin <pasha.tatashin@soleen.com>
To: akpm@linux-foundation.org, alim.akhtar@samsung.com,
alyssa@rosenzweig.io, asahi@lists.linux.dev,
baolu.lu@linux.intel.com, bhelgaas@google.com,
cgroups@vger.kernel.org, corbet@lwn.net, david@redhat.com,
dwmw2@infradead.org, hannes@cmpxchg.org, heiko@sntech.de,
iommu@lists.linux.dev, jernej.skrabec@gmail.com,
jonathanh@nvidia.com, joro@8bytes.org,
krzysztof.kozlowski@linaro.org, linux-doc@vger.kernel.org,
linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-mm@kvack.org, linux-rockchip@lists.infradead.org,
linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev,
linux-tegra@vger.kernel.org, lizefan.x@bytedance.com,
marcan@marcan.st, mhiramat@kernel.org, m.szyprowski@samsung.com,
pasha.tatashin@soleen.com, paulmck@kernel.org,
rdunlap@infradead.org, robin.murphy@arm.com, samuel@sholland.org,
suravee.suthikulpanit@amd.com, sven@svenpeter.dev,
thierry.reding@gmail.com, tj@kernel.org,
tomas.mudrunka@gmail.com, vdumpa@nvidia.com, wens@csie.org,
will@kernel.org, yu-cheng.yu@intel.com, rientjes@google.com,
bagasdotme@gmail.com, mkoutny@suse.com
Subject: [PATCH v5 00/11] IOMMU memory observability
Date: Thu, 22 Feb 2024 17:39:26 +0000 [thread overview]
Message-ID: <20240222173942.1481394-1-pasha.tatashin@soleen.com> (raw)
----------------------------------------------------------------------
Changelog
----------------------------------------------------------------------
v5:
- Synced with v6.8-rc5
- Added: Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
- Added: Acked-by: Jernej Skrabec <jernej.skrabec@gmail.com>
- Addressed review comments from Robin Murphy:
Updated the header comment in iommu-pages.h
Removed __iommu_alloc_pages_node(), invoke
iommu_alloc_pages_node directly.
Removed unused: __iommu_alloc_page_node()
Removed __iommu_free_page()
Renamed: iommu_free_pages_list() -> iommu_put_pages_list()
Added missing iommu_put_pages_list() to dma-iommu.c in
iommu/dma: use iommu_put_pages_list() to releae freelist
v4:
- Synced with v6.8-rc3
- Updated commit log for "iommu: account IOMMU allocated memory" as
suggested by Michal Koutný
- Added more Acked-bys David Rientjes and Thierry Reding
- Added Tested-by Bagas Sanjaya.
v3:
- Sync with v6.7-rc7
- Addressed comments from David Rientjes: s/pages/page/, added
unlikely() into the branches, expanded comment for
iommu_free_pages_list().
- Added Acked-bys: David Rientjes
v2:
- Added Reviewed-by Janne Grunau
- Sync with 6.7.0-rc3
- Separated form the series patches:
vhost-vdpa: account iommu allocations
https://lore.kernel.org/all/20231130200447.2319543-1-pasha.tatashin@soleen.com
vfio: account iommu allocations
https://lore.kernel.org/all/20231130200900.2320829-1-pasha.tatashin@soleen.com
as suggested by Jason Gunthorpe
- Fixed SPARC build issue detected by kernel test robot
- Drop the following patches as they do account iommu page tables:
iommu/dma: use page allocation function provided by iommu-pages.h
iommu/fsl: use page allocation function provided by iommu-pages.h
iommu/iommufd: use page allocation function provided by iommu-pages.h
as suggested by Robin Murphy. These patches are not related to IOMMU
page tables. We might need to do a separate work to support DMA
observability.
- Remove support iommu/io-pgtable-arm-v7s as the 2nd level pages are
under a page size, thanks Robin Murphy for pointing this out.
----------------------------------------------------------------------
Description
----------------------------------------------------------------------
IOMMU subsystem may contain state that is in gigabytes. Majority of that
state is iommu page tables. Yet, there is currently, no way to observe
how much memory is actually used by the iommu subsystem.
This patch series solves this problem by adding both observability to
all pages that are allocated by IOMMU, and also accountability, so
admins can limit the amount if via cgroups.
The system-wide observability is using /proc/meminfo:
SecPageTables: 438176 kB
Contains IOMMU and KVM memory.
Per-node observability:
/sys/devices/system/node/nodeN/meminfo
Node N SecPageTables: 422204 kB
Contains IOMMU and KVM memory in the given NUMA node.
Per-node IOMMU only observability:
/sys/devices/system/node/nodeN/vmstat
nr_iommu_pages 105555
Contains number of pages IOMMU allocated in the given node.
Accountability: using sec_pagetables cgroup-v2 memory.stat entry.
With the change, iova_stress[1] stops as limit is reached:
$ ./iova_stress
iova space: 0T free memory: 497G
iova space: 1T free memory: 495G
iova space: 2T free memory: 493G
iova space: 3T free memory: 491G
stops as limit is reached.
This series encorporates suggestions that came from the discussion
at LPC [2].
----------------------------------------------------------------------
[1] https://github.com/soleen/iova_stress
[2] https://lpc.events/event/17/contributions/1466
----------------------------------------------------------------------
Previous versions
v1: https://lore.kernel.org/all/20231128204938.1453583-1-pasha.tatashin@soleen.com
v2: https://lore.kernel.org/linux-mm/20231130201504.2322355-1-pasha.tatashin@soleen.com
v3: https://lore.kernel.org/all/20231226200205.562565-1-pasha.tatashin@soleen.com
v4: https://lore.kernel.org/all/20240207174102.1486130-1-pasha.tatashin@soleen.com
----------------------------------------------------------------------
Pasha Tatashin (11):
iommu/vt-d: add wrapper functions for page allocations
iommu/dma: use iommu_put_pages_list() to releae freelist
iommu/amd: use page allocation function provided by iommu-pages.h
iommu/io-pgtable-arm: use page allocation function provided by
iommu-pages.h
iommu/io-pgtable-dart: use page allocation function provided by
iommu-pages.h
iommu/exynos: use page allocation function provided by iommu-pages.h
iommu/rockchip: use page allocation function provided by iommu-pages.h
iommu/sun50i: use page allocation function provided by iommu-pages.h
iommu/tegra-smmu: use page allocation function provided by
iommu-pages.h
iommu: observability of the IOMMU allocations
iommu: account IOMMU allocated memory
Documentation/admin-guide/cgroup-v2.rst | 2 +-
Documentation/filesystems/proc.rst | 4 +-
drivers/iommu/amd/amd_iommu.h | 8 -
drivers/iommu/amd/init.c | 91 ++++++------
drivers/iommu/amd/io_pgtable.c | 13 +-
drivers/iommu/amd/io_pgtable_v2.c | 20 +--
drivers/iommu/amd/iommu.c | 13 +-
drivers/iommu/dma-iommu.c | 7 +-
drivers/iommu/exynos-iommu.c | 14 +-
drivers/iommu/intel/dmar.c | 16 +-
drivers/iommu/intel/iommu.c | 47 ++----
drivers/iommu/intel/iommu.h | 2 -
drivers/iommu/intel/irq_remapping.c | 16 +-
drivers/iommu/intel/pasid.c | 18 +--
drivers/iommu/intel/svm.c | 11 +-
drivers/iommu/io-pgtable-arm.c | 15 +-
drivers/iommu/io-pgtable-dart.c | 37 ++---
drivers/iommu/iommu-pages.h | 186 ++++++++++++++++++++++++
drivers/iommu/rockchip-iommu.c | 14 +-
drivers/iommu/sun50i-iommu.c | 7 +-
drivers/iommu/tegra-smmu.c | 18 ++-
include/linux/mmzone.h | 5 +-
mm/vmstat.c | 3 +
23 files changed, 361 insertions(+), 206 deletions(-)
create mode 100644 drivers/iommu/iommu-pages.h
--
2.44.0.rc0.258.g7320e95886-goog
next reply other threads:[~2024-02-22 17:39 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-22 17:39 Pasha Tatashin [this message]
2024-02-22 17:39 ` [PATCH v5 01/11] iommu/vt-d: add wrapper functions for page allocations Pasha Tatashin
2024-04-04 12:16 ` Linu Cherian
2024-04-04 13:56 ` Pasha Tatashin
2024-04-08 2:17 ` Linu Cherian
2024-02-22 17:39 ` [PATCH v5 02/11] iommu/dma: use iommu_put_pages_list() to releae freelist Pasha Tatashin
2024-03-15 21:28 ` David Rientjes
2024-02-22 17:39 ` [PATCH v5 03/11] iommu/amd: use page allocation function provided by iommu-pages.h Pasha Tatashin
2024-02-22 17:39 ` [PATCH v5 04/11] iommu/io-pgtable-arm: " Pasha Tatashin
2024-02-22 17:39 ` [PATCH v5 05/11] iommu/io-pgtable-dart: " Pasha Tatashin
2024-02-22 17:39 ` [PATCH v5 06/11] iommu/exynos: " Pasha Tatashin
2024-02-22 17:39 ` [PATCH v5 07/11] iommu/rockchip: " Pasha Tatashin
2024-02-22 17:39 ` [PATCH v5 08/11] iommu/sun50i: " Pasha Tatashin
2024-02-22 17:39 ` [PATCH v5 09/11] iommu/tegra-smmu: " Pasha Tatashin
2024-02-22 17:39 ` [PATCH v5 10/11] iommu: observability of the IOMMU allocations Pasha Tatashin
2024-02-22 17:39 ` [PATCH v5 11/11] iommu: account IOMMU allocated memory Pasha Tatashin
2024-03-15 21:33 ` [PATCH v5 00/11] IOMMU memory observability David Rientjes
2024-03-15 21:53 ` Joerg Roedel
2024-04-03 13:18 ` Pasha Tatashin
2024-04-04 0:58 ` Linu Cherian
2024-04-04 13:50 ` Pasha Tatashin
2024-04-12 10:10 ` Joerg Roedel
2024-04-12 17:06 ` Pasha Tatashin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240222173942.1481394-1-pasha.tatashin@soleen.com \
--to=pasha.tatashin@soleen.com \
--cc=akpm@linux-foundation.org \
--cc=alim.akhtar@samsung.com \
--cc=alyssa@rosenzweig.io \
--cc=asahi@lists.linux.dev \
--cc=bagasdotme@gmail.com \
--cc=baolu.lu@linux.intel.com \
--cc=bhelgaas@google.com \
--cc=cgroups@vger.kernel.org \
--cc=corbet@lwn.net \
--cc=david@redhat.com \
--cc=dwmw2@infradead.org \
--cc=hannes@cmpxchg.org \
--cc=heiko@sntech.de \
--cc=iommu@lists.linux.dev \
--cc=jernej.skrabec@gmail.com \
--cc=jonathanh@nvidia.com \
--cc=joro@8bytes.org \
--cc=krzysztof.kozlowski@linaro.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-rockchip@lists.infradead.org \
--cc=linux-samsung-soc@vger.kernel.org \
--cc=linux-sunxi@lists.linux.dev \
--cc=linux-tegra@vger.kernel.org \
--cc=lizefan.x@bytedance.com \
--cc=m.szyprowski@samsung.com \
--cc=marcan@marcan.st \
--cc=mhiramat@kernel.org \
--cc=mkoutny@suse.com \
--cc=paulmck@kernel.org \
--cc=rdunlap@infradead.org \
--cc=rientjes@google.com \
--cc=robin.murphy@arm.com \
--cc=samuel@sholland.org \
--cc=suravee.suthikulpanit@amd.com \
--cc=sven@svenpeter.dev \
--cc=thierry.reding@gmail.com \
--cc=tj@kernel.org \
--cc=tomas.mudrunka@gmail.com \
--cc=vdumpa@nvidia.com \
--cc=wens@csie.org \
--cc=will@kernel.org \
--cc=yu-cheng.yu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).