From: Anup Patel <apatel@ventanamicro.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>,
Paul Walmsley <paul.walmsley@sifive.com>,
Thomas Gleixner <tglx@linutronix.de>,
Marc Zyngier <maz@kernel.org>, Rob Herring <robh+dt@kernel.org>,
Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>,
Robin Murphy <robin.murphy@arm.com>,
Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
Frank Rowand <frowand.list@gmail.com>,
Atish Patra <atishp@atishpatra.org>,
Andrew Jones <ajones@ventanamicro.com>,
Anup Patel <anup@brainfault.org>,
linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
devicetree@vger.kernel.org, iommu@lists.linux.dev,
Vincent Chen <vincent.chen@sifive.com>
Subject: Re: [PATCH v3 07/11] irqchip/riscv-imsic: Improve IOMMU DMA support
Date: Tue, 13 Jun 2023 13:25:41 +0530 [thread overview]
Message-ID: <CAK9=C2V9o4FsFXAfRHLGSuJaDubwM_HSL7keTYeWb_vFDNLd7g@mail.gmail.com> (raw)
In-Reply-To: <ZGIrOKIT8uHcNqbQ@nvidia.com>
On Mon, May 15, 2023 at 6:23 PM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Mon, May 08, 2023 at 07:58:38PM +0530, Anup Patel wrote:
> > We have a separate RISC-V IMSIC MSI address for each CPU so changing
> > MSI (or IRQ) affinity results in re-programming of MSI address in
> > the PCIe (or platform) device.
> >
> > Currently, the iommu_dma_prepare_msi() is called only once at the
> > time of IRQ allocation so IOMMU DMA domain will only have mapping
> > for one MSI page. This means iommu_dma_compose_msi_msg() called
> > by imsic_irq_compose_msi_msg() will always use the same MSI page
> > irrespective to target CPU MSI address. In other words, changing
> > MSI (or IRQ) affinity for device using IOMMU DMA domain will not
> > work.
> >
> > To address above issue, we do the following:
> > 1) Map MSI pages for all CPUs in imsic_irq_domain_alloc()
> > using iommu_dma_prepare_msi().
> > 2) Add a new iommu_dma_select_msi() API to select a specific
> > MSI page from a set of already mapped MSI pages.
> > 3) Use iommu_dma_select_msi() to select a specific MSI page
> > before calling iommu_dma_compose_msi_msg() in
> > imsic_irq_compose_msi_msg().
>
> Is there an iommu driver somewhere in all this? I don't obviously see
> one?
Sorry for the delayed response.
The RISC-V IOMMU specification is frozen and will be ratified/released
anytime this month or next.
https://github.com/riscv-non-isa/riscv-iommu/releases/download/v1.0-rc6/riscv-iommu.pdf
The RISC-V IOMMU driver will be send-out on LKML pretty soon
https://github.com/tjeznach/linux/tree/tjeznach/riscv-iommu
which can be tested on QEMU
https://github.com/tjeznach/qemu/tree/tjeznach/riscv-iommu
>
> There should be no reason to use the dma-iommu.c stuff just to make
> interrupts work, that is only necessary if there is an iommu, and the
> platform architecture requires the iommu to have the MSI region
> programmed into IOPTEs.
>
> And I'd be much happier if we could clean this design up before risc-v
> starts using it too :\
>
Sure, I will send-out v4 in the next few days.
Regards,
Anup
next prev parent reply other threads:[~2023-06-13 7:56 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-08 14:28 [PATCH v3 00/11] Linux RISC-V AIA Support Anup Patel
2023-05-08 14:28 ` [PATCH v3 01/11] RISC-V: Add riscv_fw_parent_hartid() function Anup Patel
2023-05-10 12:45 ` Conor Dooley
2023-06-13 8:05 ` Anup Patel
2023-05-08 14:28 ` [PATCH v3 02/11] of/irq: Set FWNODE_FLAG_BEST_EFFORT for the interrupt controller DT nodes Anup Patel
2023-06-08 18:28 ` Rob Herring
2023-06-08 20:04 ` Saravana Kannan
2023-06-09 11:40 ` Anup Patel
2023-06-09 21:17 ` Saravana Kannan
2023-06-13 4:42 ` Anup Patel
2023-05-08 14:28 ` [PATCH v3 03/11] irqchip/riscv-intc: Add support for RISC-V AIA Anup Patel
2023-05-08 14:28 ` [PATCH v3 04/11] dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller Anup Patel
2023-05-10 12:16 ` Conor Dooley
2023-06-13 8:18 ` Anup Patel
2023-05-11 9:49 ` Krzysztof Kozlowski
2023-05-08 14:28 ` [PATCH v3 05/11] irqchip: Add RISC-V incoming MSI controller driver Anup Patel
2023-05-08 14:28 ` [PATCH v3 06/11] irqchip/riscv-imsic: Add support for PCI MSI irqdomain Anup Patel
2023-05-08 14:28 ` [PATCH v3 07/11] irqchip/riscv-imsic: Improve IOMMU DMA support Anup Patel
2023-05-10 10:48 ` Robin Murphy
2023-05-10 15:12 ` Anup Patel
2023-05-15 12:53 ` Jason Gunthorpe
2023-06-13 7:55 ` Anup Patel [this message]
2023-05-08 14:28 ` [PATCH v3 08/11] dt-bindings: interrupt-controller: Add RISC-V advanced PLIC Anup Patel
2023-05-10 15:41 ` Conor Dooley
2023-06-13 10:37 ` Anup Patel
2023-05-08 14:28 ` [PATCH v3 09/11] irqchip: Add RISC-V advanced PLIC driver Anup Patel
2023-05-08 14:28 ` [PATCH v3 10/11] RISC-V: Select APLIC and IMSIC drivers Anup Patel
2023-05-10 12:20 ` Conor Dooley
2023-05-08 14:28 ` [PATCH v3 11/11] MAINTAINERS: Add entry for RISC-V AIA drivers Anup Patel
2023-05-10 10:24 ` [PATCH v3 00/11] Linux RISC-V AIA Support Conor Dooley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAK9=C2V9o4FsFXAfRHLGSuJaDubwM_HSL7keTYeWb_vFDNLd7g@mail.gmail.com' \
--to=apatel@ventanamicro.com \
--cc=ajones@ventanamicro.com \
--cc=anup@brainfault.org \
--cc=atishp@atishpatra.org \
--cc=devicetree@vger.kernel.org \
--cc=frowand.list@gmail.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=krzysztof.kozlowski+dt@linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-riscv@lists.infradead.org \
--cc=maz@kernel.org \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=robh+dt@kernel.org \
--cc=robin.murphy@arm.com \
--cc=tglx@linutronix.de \
--cc=vincent.chen@sifive.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).