Re: [PATCH v2 7/7] iommu/riscv: Paging domain support

From: Baolu Lu <baolu.lu@linux.intel.com>
To: Jason Gunthorpe <jgg@ziepe.ca>, Tomasz Jeznach <tjeznach@rivosinc.com>
Cc: baolu.lu@linux.intel.com, Joerg Roedel <joro@8bytes.org>,
	Will Deacon <will@kernel.org>,
	Robin Murphy <robin.murphy@arm.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Anup Patel <apatel@ventanamicro.com>,
	Sunil V L <sunilvl@ventanamicro.com>,
	Nick Kossifidis <mick@ics.forth.gr>,
	Sebastien Boeuf <seb@rivosinc.com>,
	Rob Herring <robh+dt@kernel.org>,
	Krzysztof Kozlowski <krzk+dt@kernel.org>,
	Conor Dooley <conor+dt@kernel.org>,
	devicetree@vger.kernel.org, iommu@lists.linux.dev,
	linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
	linux@rivosinc.com
Subject: Re: [PATCH v2 7/7] iommu/riscv: Paging domain support
Date: Mon, 22 Apr 2024 15:40:51 +0800	[thread overview]
Message-ID: <23ef7fbd-7498-4fb0-895b-81c618acedc1@linux.intel.com> (raw)
In-Reply-To: <20240419125627.GD223006@ziepe.ca>

On 2024/4/19 20:56, Jason Gunthorpe wrote:
>> +		riscv_iommu_cmd_inval_vma(&cmd);
>> +		riscv_iommu_cmd_inval_set_pscid(&cmd, domain->pscid);
>> +		if (len > 0 && len < RISCV_IOMMU_IOTLB_INVAL_LIMIT) {
>> +			for (iova = start; iova < end; iova += PAGE_SIZE) {
>> +				riscv_iommu_cmd_inval_set_addr(&cmd, iova);
>> +				riscv_iommu_cmd_send(iommu, &cmd, 0);
>> +			}
>> +		} else {
>> +			riscv_iommu_cmd_send(iommu, &cmd, 0);
>> +		}
>> +	}
> This seems suboptimal, you probably want to copy the new design that
> Intel is doing where you allocate "bonds" that are already
> de-duplicated. Ie if I have 10 devices on the same iommu sharing the
> domain the above will invalidate the PSCID 10 times. It should only be
> done once.
> 
> ie add a "bond" for the (iommu,pscid) and refcount that based on how
> many devices are used. Then another "bond" for the ATS stuff eventually.

The latest version is under discussion here.

https://lore.kernel.org/linux-iommu/20240416080656.60968-1-baolu.lu@linux.intel.com/

Supposedly, you can make such optimization after the base code is landed
in the mainline tree if the change is big.

Best regards,
baolu