From: Yan Zhao <yan.y.zhao@intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Christoph Hellwig <hch@infradead.org>, <kvm@vger.kernel.org>,
<linux-kernel@vger.kernel.org>, <x86@kernel.org>,
<alex.williamson@redhat.com>, <kevin.tian@intel.com>,
<iommu@lists.linux.dev>, <pbonzini@redhat.com>,
<seanjc@google.com>, <dave.hansen@linux.intel.com>,
<luto@kernel.org>, <peterz@infradead.org>, <tglx@linutronix.de>,
<mingo@redhat.com>, <bp@alien8.de>, <hpa@zytor.com>,
<corbet@lwn.net>, <joro@8bytes.org>, <will@kernel.org>,
<robin.murphy@arm.com>, <baolu.lu@linux.intel.com>,
<yi.l.liu@intel.com>, Russell King <linux@armlinux.org.uk>
Subject: Re: [PATCH 3/5] x86/mm: Introduce and export interface arch_clean_nonsnoop_dma()
Date: Fri, 7 Jun 2024 17:39:21 +0800 [thread overview]
Message-ID: <ZmLVSU39rJBzQFX0@yzhao56-desk.sh.intel.com> (raw)
In-Reply-To: <20240606115503.GZ19897@nvidia.com>
On Thu, Jun 06, 2024 at 08:55:03AM -0300, Jason Gunthorpe wrote:
> On Thu, Jun 06, 2024 at 10:48:10AM +0800, Yan Zhao wrote:
> > On Sat, Jun 01, 2024 at 04:46:14PM -0300, Jason Gunthorpe wrote:
> > > On Mon, May 27, 2024 at 11:37:34PM -0700, Christoph Hellwig wrote:
> > > > On Tue, May 21, 2024 at 01:00:16PM -0300, Jason Gunthorpe wrote:
> > > > > > > Err, no. There should really be no exported cache manipulation macros,
> > > > > > > as drivers are almost guaranteed to get this wrong. I've added
> > > > > > > Russell to the Cc list who has been extremtly vocal about this at least
> > > > > > > for arm.
> > > > > >
> > > > > > We could possibly move this under some IOMMU core API (ie flush and
> > > > > > map, unmap and flush), the iommu APIs are non-modular so this could
> > > > > > avoid the exported symbol.
> > > > >
> > > > > Though this would be pretty difficult for unmap as we don't have the
> > > > > pfns in the core code to flush. I don't think we have alot of good
> > > > > options but to make iommufd & VFIO handle this directly as they have
> > > > > the list of pages to flush on the unmap side. Use a namespace?
> > > >
> > > > Just have a unmap version that also takes a list of PFNs that you'd
> > > > need for non-coherent mappings?
> > >
> > > VFIO has never supported that so nothing like that exists yet.. This
> > > is sort of the first steps to some very basic support for a
> > > non-coherent cache flush in a limited case of a VM that can do its own
> > > cache flushing through kvm.
> > >
> > > The pfn list is needed for unpin_user_pages() and it has an ugly
> > > design where vfio/iommufd read back the pfns seperately from unmap,
> > > and they both do it differently without a common range list
> > > datastructure here.
> > >
> > > So, we'd need to build some new unmap function that returns a pfn list
> > > that it internally fetches via the read ops. Then it can do the read,
> > > unmap, flush iotlb, flush cache in core code.
> > Would the core code flush CPU caches by providing page physical address?
>
> Physical address is all we will have in the core code..
>
> > If yes, do you think it's still necessary to export arch_flush_cache_phys()
> > (as what's implemented in this patch)?
>
> Christoph is asking not to export it, that would mean relying on the
> iommu core to be non-modulare and putting the arch calls there with a
> more restricted exported API - ie based on unmap.
Got it. Thanks for explanation!
>
> > > I've been working towards this very slowly as I want to push this
> > > stuff down into the io page table walk and remove the significant
> > > inefficiency, so it is not throw away work, but it is certainly some
> > > notable amount of work to do.
> > Will VFIO also be switched to this new unmap interface? Do we need to care
> > about backporting?
>
> I don't know :)
>
> > And is it possible for VFIO alone to implement in the current proposed way
> > in this series as the first step for easier backport?
>
> I think this series is the best option we have right now, but make the
> EXPORT a NS export to try to discourage abuse of it while we continue
> working
Will do. Thanks!
next prev parent reply other threads:[~2024-06-07 9:40 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-07 6:18 [PATCH 0/5] Enforce CPU cache flush for non-coherent device assignment Yan Zhao
2024-05-07 6:19 ` [PATCH 1/5] x86/pat: Let pat_pfn_immune_to_uc_mtrr() check MTRR for untracked PAT range Yan Zhao
2024-05-07 8:26 ` Tian, Kevin
2024-05-07 9:12 ` Yan Zhao
2024-05-08 22:14 ` Alex Williamson
2024-05-09 3:36 ` Yan Zhao
2024-05-16 7:42 ` Tian, Kevin
2024-05-16 14:07 ` Sean Christopherson
2024-05-20 2:36 ` Tian, Kevin
2024-05-07 6:20 ` [PATCH 2/5] KVM: x86/mmu: Fine-grained check of whether a invalid & RAM PFN is MMIO Yan Zhao
2024-05-07 8:39 ` Tian, Kevin
2024-05-07 9:19 ` Yan Zhao
2024-05-07 6:20 ` [PATCH 3/5] x86/mm: Introduce and export interface arch_clean_nonsnoop_dma() Yan Zhao
2024-05-07 8:51 ` Tian, Kevin
2024-05-07 9:40 ` Yan Zhao
2024-05-20 14:07 ` Christoph Hellwig
2024-05-21 15:49 ` Jason Gunthorpe
2024-05-21 16:00 ` Jason Gunthorpe
2024-05-22 3:41 ` Yan Zhao
2024-05-28 6:37 ` Christoph Hellwig
2024-06-01 19:46 ` Jason Gunthorpe
2024-06-06 2:48 ` Yan Zhao
2024-06-06 11:55 ` Jason Gunthorpe
2024-06-07 9:39 ` Yan Zhao [this message]
2024-05-07 6:21 ` [PATCH 4/5] vfio/type1: Flush CPU caches on DMA pages in non-coherent domains Yan Zhao
2024-05-09 18:10 ` Alex Williamson
2024-05-10 10:31 ` Yan Zhao
2024-05-10 16:57 ` Alex Williamson
2024-05-13 7:11 ` Yan Zhao
2024-05-16 7:53 ` Tian, Kevin
2024-05-16 8:34 ` Tian, Kevin
2024-05-16 20:31 ` Alex Williamson
2024-05-17 17:11 ` Jason Gunthorpe
2024-05-20 2:52 ` Tian, Kevin
2024-05-21 16:07 ` Jason Gunthorpe
2024-05-21 16:21 ` Alex Williamson
2024-05-21 16:34 ` Jason Gunthorpe
2024-05-21 18:19 ` Alex Williamson
2024-05-21 18:37 ` Jason Gunthorpe
2024-05-22 6:24 ` Tian, Kevin
2024-05-22 12:29 ` Jason Gunthorpe
2024-05-22 14:43 ` Alex Williamson
2024-05-22 16:52 ` Jason Gunthorpe
2024-05-22 18:22 ` Alex Williamson
2024-05-22 23:26 ` Tian, Kevin
2024-05-22 23:32 ` Jason Gunthorpe
2024-05-22 23:40 ` Tian, Kevin
2024-05-23 14:58 ` Jason Gunthorpe
2024-05-23 22:47 ` Alex Williamson
2024-05-24 0:30 ` Tian, Kevin
2024-05-24 13:50 ` Jason Gunthorpe
2024-05-22 3:33 ` Yan Zhao
2024-05-22 3:24 ` Yan Zhao
2024-05-22 12:26 ` Jason Gunthorpe
2024-05-24 3:07 ` Yan Zhao
2024-05-16 20:50 ` Alex Williamson
2024-05-17 3:11 ` Yan Zhao
2024-05-17 4:44 ` Alex Williamson
2024-05-17 5:00 ` Yan Zhao
2024-05-07 6:22 ` [PATCH 5/5] iommufd: " Yan Zhao
2024-05-09 14:13 ` Jason Gunthorpe
2024-05-10 8:03 ` Yan Zhao
2024-05-10 13:29 ` Jason Gunthorpe
2024-05-13 7:43 ` Yan Zhao
2024-05-14 15:11 ` Jason Gunthorpe
2024-05-15 7:06 ` Yan Zhao
2024-05-15 20:43 ` Jason Gunthorpe
2024-05-16 2:32 ` Yan Zhao
2024-05-16 8:38 ` Tian, Kevin
2024-05-16 9:48 ` Yan Zhao
2024-05-17 17:04 ` Jason Gunthorpe
2024-05-20 2:45 ` Yan Zhao
2024-05-21 16:04 ` Jason Gunthorpe
2024-05-22 3:17 ` Yan Zhao
2024-05-22 6:29 ` Yan Zhao
2024-05-22 17:01 ` Jason Gunthorpe
2024-05-27 7:15 ` Yan Zhao
2024-06-01 16:48 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZmLVSU39rJBzQFX0@yzhao56-desk.sh.intel.com \
--to=yan.y.zhao@intel.com \
--cc=alex.williamson@redhat.com \
--cc=baolu.lu@linux.intel.com \
--cc=bp@alien8.de \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=hch@infradead.org \
--cc=hpa@zytor.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=robin.murphy@arm.com \
--cc=seanjc@google.com \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
--cc=x86@kernel.org \
--cc=yi.l.liu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).