From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B86A5C433EF for ; Tue, 28 Dec 2021 12:23:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231158AbhL1MXS (ORCPT ); Tue, 28 Dec 2021 07:23:18 -0500 Received: from ams.source.kernel.org ([145.40.68.75]:33498 "EHLO ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230290AbhL1MXS (ORCPT ); Tue, 28 Dec 2021 07:23:18 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id ADFBFB811BD; Tue, 28 Dec 2021 12:23:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3CA1EC36AE8; Tue, 28 Dec 2021 12:23:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1640694195; bh=ctzCAJML9qMvzhn8zf3X+uHDG/LU7mhRfvDCv/DU5+c=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Hsfy1FCV6/kLsTWdRLKKONenuyoAGROPdbP4BiXY3md0f0fAHSml4MZyk5eeTiZAH WtM62Xt0aIc5SnGUDyv62QwtCMFhCfuWKfqLHvudKSV4SvnuVx9gAMXPew7+UQxoVX E+6Uh/qqqgW+1OgG8lfrfv2VvTOc/dhSOSjmTP5L9OuJLS4TnlKuWmZi+lOGny4QGf 97iyywzeeW9wr1YxBLaOHMd1djkQOSE43hWX9kEsXt7QFt4qN7iyyfBB2j5+GEDIZL 6ct1aQ0h5FMRqsCC8OC0w3PNlgWmc6oZ7gKO2c7PttC6f7QByamDpT5LGxHMfK6Gl1 JuvHHWcyM/BbQ== Received: from cfbb000407.r.cam.camfibre.uk ([185.219.108.64] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1n2BVZ-00EiPi-8y; Tue, 28 Dec 2021 12:23:13 +0000 Date: Tue, 28 Dec 2021 12:23:12 +0000 Message-ID: <87ilv8zznz.wl-maz@kernel.org> From: Marc Zyngier To: "Michael Kelley (LINUX)" Cc: Sunil Muthuswamy , KY Srinivasan , Haiyang Zhang , Stephen Hemminger , "wei.liu@kernel.org" , Dexuan Cui , "tglx@linutronix.de" , "mingo@redhat.com" , "bp@alien8.de" , "hpa@zytor.com" , "lorenzo.pieralisi@arm.com" , "robh@kernel.org" , "kw@linux.com" , "bhelgaas@google.com" , "arnd@arndb.de" , "x86@kernel.org" , "linux-kernel@vger.kernel.org" , "linux-hyperv@vger.kernel.org" , "linux-pci@vger.kernel.org" , "linux-arch@vger.kernel.org" , Sunil Muthuswamy Subject: Re: [PATCH v7 2/2] PCI: hv: Add arm64 Hyper-V vPCI support In-Reply-To: References: <1639767121-22007-1-git-send-email-sunilmut@linux.microsoft.com> <1639767121-22007-3-git-send-email-sunilmut@linux.microsoft.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: mikelley@microsoft.com, sunilmut@linux.microsoft.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, lorenzo.pieralisi@arm.com, robh@kernel.org, kw@linux.com, bhelgaas@google.com, arnd@arndb.de, x86@kernel.org, linux-kernel@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-pci@vger.kernel.org, linux-arch@vger.kernel.org, sunilmut@microsoft.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-arch@vger.kernel.org On Mon, 27 Dec 2021 17:38:07 +0000, "Michael Kelley (LINUX)" wrote: > > From: Sunil Muthuswamy Sent: Friday, December 17, 2021 10:52 AM > > > > Add arm64 Hyper-V vPCI support by implementing the arch specific > > interfaces. Introduce an IRQ domain and chip specific to Hyper-v vPCI that > > is based on SPIs. The IRQ domain parents itself to the arch GIC IRQ domain > > for basic vector management. > > > > Signed-off-by: Sunil Muthuswamy > > --- > > In v2, v3, v4, v5, v6 & v7: > > Changes are described in the cover letter. > > > > arch/arm64/include/asm/hyperv-tlfs.h | 9 + > > drivers/pci/Kconfig | 2 +- > > drivers/pci/controller/Kconfig | 2 +- > > drivers/pci/controller/pci-hyperv.c | 241 ++++++++++++++++++++++++++- > > 4 files changed, 251 insertions(+), 3 deletions(-) > > > > diff --git a/arch/arm64/include/asm/hyperv-tlfs.h b/arch/arm64/include/asm/hyperv-tlfs.h > > index 4d964a7f02ee..bc6c7ac934a1 100644 > > --- a/arch/arm64/include/asm/hyperv-tlfs.h > > +++ b/arch/arm64/include/asm/hyperv-tlfs.h > > @@ -64,6 +64,15 @@ > > #define HV_REGISTER_STIMER0_CONFIG 0x000B0000 > > #define HV_REGISTER_STIMER0_COUNT 0x000B0001 > > > > +union hv_msi_entry { > > + u64 as_uint64[2]; > > + struct { > > + u64 address; > > + u32 data; > > + u32 reserved; > > + } __packed; > > +}; > > + > > #include > > > > #endif > > diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig > > index 43e615aa12ff..d98fafdd0f99 100644 > > --- a/drivers/pci/Kconfig > > +++ b/drivers/pci/Kconfig > > @@ -184,7 +184,7 @@ config PCI_LABEL > > > > config PCI_HYPERV > > tristate "Hyper-V PCI Frontend" > > - depends on X86_64 && HYPERV && PCI_MSI && PCI_MSI_IRQ_DOMAIN && SYSFS > > + depends on ((X86 && X86_64) || ARM64) && HYPERV && PCI_MSI && PCI_MSI_IRQ_DOMAIN && SYSFS > > select PCI_HYPERV_INTERFACE > > help > > The PCI device frontend driver allows the kernel to import arbitrary > > diff --git a/drivers/pci/controller/Kconfig b/drivers/pci/controller/Kconfig > > index 93b141110537..2536abcc045a 100644 > > --- a/drivers/pci/controller/Kconfig > > +++ b/drivers/pci/controller/Kconfig > > @@ -281,7 +281,7 @@ config PCIE_BRCMSTB > > > > config PCI_HYPERV_INTERFACE > > tristate "Hyper-V PCI Interface" > > - depends on X86 && HYPERV && PCI_MSI && PCI_MSI_IRQ_DOMAIN && X86_64 > > + depends on ((X86 && X86_64) || ARM64) && HYPERV && PCI_MSI && PCI_MSI_IRQ_DOMAIN > > help > > The Hyper-V PCI Interface is a helper driver allows other drivers to > > have a common interface with the Hyper-V PCI frontend driver. > > diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c > > index ead7d6cb6bf1..02ba2e7e2618 100644 > > --- a/drivers/pci/controller/pci-hyperv.c > > +++ b/drivers/pci/controller/pci-hyperv.c > > @@ -47,6 +47,8 @@ > > #include > > #include > > #include > > +#include > > +#include > > #include > > > > /* > > @@ -614,7 +616,236 @@ static int hv_msi_prepare(struct irq_domain *domain, struct device *dev, > > { > > return pci_msi_prepare(domain, dev, nvec, info); > > } > > -#endif /* CONFIG_X86 */ > > +#elif defined(CONFIG_ARM64) > > +/* > > + * SPI vectors to use for vPCI; arch SPIs range is [32, 1019], but leaving a bit > > + * of room at the start to allow for SPIs to be specified through ACPI and > > + * starting with a power of two to satisfy power of 2 multi-MSI requirement. > > + */ > > +#define HV_PCI_MSI_SPI_START 64 > > +#define HV_PCI_MSI_SPI_NR (1020 - HV_PCI_MSI_SPI_START) > > +#define DELIVERY_MODE 0 > > +#define FLOW_HANDLER NULL > > +#define FLOW_NAME NULL > > +#define hv_msi_prepare NULL > > + > > +struct hv_pci_chip_data { > > + DECLARE_BITMAP(spi_map, HV_PCI_MSI_SPI_NR); > > + struct mutex map_lock; > > +}; > > + > > +/* Hyper-V vPCI MSI GIC IRQ domain */ > > +static struct irq_domain *hv_msi_gic_irq_domain; > > + > > +/* Hyper-V PCI MSI IRQ chip */ > > +static struct irq_chip hv_arm64_msi_irq_chip = { > > + .name = "MSI", > > + .irq_set_affinity = irq_chip_set_affinity_parent, > > + .irq_eoi = irq_chip_eoi_parent, > > + .irq_mask = irq_chip_mask_parent, > > + .irq_unmask = irq_chip_unmask_parent > > +}; > > + > > +static unsigned int hv_msi_get_int_vector(struct irq_data *irqd) > > +{ > > + return irqd->parent_data->hwirq; > > +} > > + > > +static void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry, > > + struct msi_desc *msi_desc) > > +{ > > + msi_entry->address = ((u64)msi_desc->msg.address_hi << 32) | > > + msi_desc->msg.address_lo; > > + msi_entry->data = msi_desc->msg.data; > > +} > > + > > +/* > > + * @nr_bm_irqs: Indicates the number of IRQs that were allocated from > > + * the bitmap. > > + * @nr_dom_irqs: Indicates the number of IRQs that were allocated from > > + * the parent domain. > > + */ > > +static void hv_pci_vec_irq_free(struct irq_domain *domain, > > + unsigned int virq, > > + unsigned int nr_bm_irqs, > > + unsigned int nr_dom_irqs) > > +{ > > + struct hv_pci_chip_data *chip_data = domain->host_data; > > + struct irq_data *d = irq_domain_get_irq_data(domain, virq); > > FWIW, irq_domain_get_irq_data() can return NULL. Maybe that's an > error in the "should never happen" category. Throughout kernel code, > some callers check for a NULL result, but a lot do not. irq_domain_get_irq_data() returns NULL when there is no mapping. If this happens here, then the allocation tracking has gone horribly wrong, and I certainly want to see the resulting Oops rather than papering over it. M. -- Without deviation from the norm, progress is not possible.