From: Tomasz Jeznach <tjeznach@rivosinc.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
Robin Murphy <robin.murphy@arm.com>,
Paul Walmsley <paul.walmsley@sifive.com>,
Palmer Dabbelt <palmer@dabbelt.com>,
Albert Ou <aou@eecs.berkeley.edu>,
Anup Patel <apatel@ventanamicro.com>,
Sunil V L <sunilvl@ventanamicro.com>,
Nick Kossifidis <mick@ics.forth.gr>,
Sebastien Boeuf <seb@rivosinc.com>,
Rob Herring <robh+dt@kernel.org>,
Krzysztof Kozlowski <krzk+dt@kernel.org>,
Conor Dooley <conor+dt@kernel.org>,
devicetree@vger.kernel.org, iommu@lists.linux.dev,
linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
linux@rivosinc.com
Subject: Re: [PATCH v2 7/7] iommu/riscv: Paging domain support
Date: Wed, 24 Apr 2024 16:54:01 -0700 [thread overview]
Message-ID: <CAH2o1u4eZ-mRO7hiJzA-pwYDOo0+3vObpBZT3_MXW=zC9mXRbA@mail.gmail.com> (raw)
In-Reply-To: <20240424233950.GJ231144@ziepe.ca>
On Wed, Apr 24, 2024 at 4:39 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Wed, Apr 24, 2024 at 04:30:45PM -0700, Tomasz Jeznach wrote:
> > > > @@ -46,6 +46,10 @@ MODULE_LICENSE("GPL");
> > > > #define dev_to_iommu(dev) \
> > > > container_of((dev)->iommu->iommu_dev, struct riscv_iommu_device, iommu)
> > > >
> > > > +/* IOMMU PSCID allocation namespace. */
> > > > +static DEFINE_IDA(riscv_iommu_pscids);
> > > > +#define RISCV_IOMMU_MAX_PSCID BIT(20)
> > > > +
> > >
> > > You may consider putting this IDA in the riscv_iommu_device() and move
> > > the pscid from the domain to the bond?
> > >
> >
> > I've been considering containing IDA inside riscv_iommu_device at some
> > point, but it made PCSID management more complicated. In the follow
> > up patches it is desired for PSCID to be unique across all IOMMUs in
> > the system (within guest's GSCID), as the protection domains might
> > (and will) be shared between more than single IOMMU device.
>
> The PCSID isn't scoped under the GSCID? That doesn't sound very good,
> it means VM's can't direct issue invalidation with their local view of
> the PCSID space?
>
To clarify: PSCID namespace is per GSCID.
However there might be more than one IOMMU in a single system sharing
the same GSCID, and with e.g. SVA domains attached to more than one
IOMMU. It was simpler to manage PCSID globally.
PSCID management for the VM assigned GSCID will be the VM's responsibility.
> > > This seems suboptimal, you probably want to copy the new design that
> > > Intel is doing where you allocate "bonds" that are already
> > > de-duplicated. Ie if I have 10 devices on the same iommu sharing the
> > > domain the above will invalidate the PSCID 10 times. It should only be
> > > done once.
> > >
> > > ie add a "bond" for the (iommu,pscid) and refcount that based on how
> > > many devices are used. Then another "bond" for the ATS stuff eventually.
> > >
> >
> > Agree, not perfect to send duplicate invalidations.
> > This should improve with follow up patchsets introducing of SVA
> > (reusing the same, extended bond structure) and update to send IOTLB
> > range invalidations.
> >
> > For this change I've decided to go with as simple as possible
> > implementation and over-invalidate for domains with multiple devices
> > attached. Hope this makes sense.
>
> It is fine as long as you do fix it..
>
SG. I'll have a second look if it can be fixed sooner.
> Jason
Best,
- Tomasz
next prev parent reply other threads:[~2024-04-24 23:54 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-18 16:32 [PATCH v2 0/7] Linux RISC-V IOMMU Support Tomasz Jeznach
2024-04-18 16:32 ` [PATCH v2 1/7] dt-bindings: iommu: riscv: Add bindings for RISC-V IOMMU Tomasz Jeznach
2024-04-18 17:04 ` Conor Dooley
2024-04-24 22:37 ` Tomasz Jeznach
2024-04-25 17:11 ` Conor Dooley
2024-04-22 14:04 ` Rob Herring
2024-04-18 16:32 ` [PATCH v2 2/7] iommu/riscv: Add RISC-V IOMMU platform device driver Tomasz Jeznach
2024-04-18 21:22 ` Robin Murphy
2024-04-24 21:59 ` Tomasz Jeznach
2024-04-25 11:23 ` Robin Murphy
2024-04-18 16:32 ` [PATCH v2 3/7] iommu/riscv: Add RISC-V IOMMU PCIe " Tomasz Jeznach
2024-04-18 22:07 ` Robin Murphy
2024-04-18 16:32 ` [PATCH v2 4/7] iommu/riscv: Enable IOMMU registration and device probe Tomasz Jeznach
2024-04-18 16:32 ` [PATCH v2 5/7] iommu/riscv: Device directory management Tomasz Jeznach
2024-04-19 12:40 ` Jason Gunthorpe
2024-04-24 23:01 ` Tomasz Jeznach
2024-04-24 23:07 ` Jason Gunthorpe
2024-04-22 5:11 ` Baolu Lu
2024-04-24 23:07 ` Tomasz Jeznach
2024-04-18 16:32 ` [PATCH v2 6/7] iommu/riscv: Command and fault queue support Tomasz Jeznach
2024-04-18 16:32 ` [PATCH v2 7/7] iommu/riscv: Paging domain support Tomasz Jeznach
2024-04-19 12:56 ` Jason Gunthorpe
2024-04-22 7:40 ` Baolu Lu
2024-04-24 23:30 ` Tomasz Jeznach
2024-04-24 23:39 ` Jason Gunthorpe
2024-04-24 23:54 ` Tomasz Jeznach [this message]
2024-04-25 0:48 ` Jason Gunthorpe
2024-04-22 5:21 ` Baolu Lu
2024-04-22 19:30 ` Jason Gunthorpe
2024-04-23 17:00 ` Andrew Jones
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAH2o1u4eZ-mRO7hiJzA-pwYDOo0+3vObpBZT3_MXW=zC9mXRbA@mail.gmail.com' \
--to=tjeznach@rivosinc.com \
--cc=aou@eecs.berkeley.edu \
--cc=apatel@ventanamicro.com \
--cc=conor+dt@kernel.org \
--cc=devicetree@vger.kernel.org \
--cc=iommu@lists.linux.dev \
--cc=jgg@ziepe.ca \
--cc=joro@8bytes.org \
--cc=krzk+dt@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=linux@rivosinc.com \
--cc=mick@ics.forth.gr \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=robh+dt@kernel.org \
--cc=robin.murphy@arm.com \
--cc=seb@rivosinc.com \
--cc=sunilvl@ventanamicro.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).