From: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
To: Niklas Cassel <cassel@kernel.org>
Cc: "Jingoo Han" <jingoohan1@gmail.com>,
"Gustavo Pimentel" <gustavo.pimentel@synopsys.com>,
"Lorenzo Pieralisi" <lpieralisi@kernel.org>,
"Krzysztof Wilczyński" <kw@linux.com>,
"Rob Herring" <robh@kernel.org>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Marek Vasut" <marek.vasut+renesas@gmail.com>,
"Yoshihiro Shimoda" <yoshihiro.shimoda.uh@renesas.com>,
"Thierry Reding" <thierry.reding@gmail.com>,
"Jonathan Hunter" <jonathanh@nvidia.com>,
"Kishon Vijay Abraham I" <kishon@ti.com>,
"Vidya Sagar" <vidyas@nvidia.com>,
"Vignesh Raghavendra" <vigneshr@ti.com>,
"Richard Zhu" <hongxing.zhu@nxp.com>,
"Lucas Stach" <l.stach@pengutronix.de>,
"Shawn Guo" <shawnguo@kernel.org>,
"Sascha Hauer" <s.hauer@pengutronix.de>,
"Pengutronix Kernel Team" <kernel@pengutronix.de>,
"Fabio Estevam" <festevam@gmail.com>,
"NXP Linux Team" <linux-imx@nxp.com>,
"Minghuan Lian" <minghuan.Lian@nxp.com>,
"Mingkai Hu" <mingkai.hu@nxp.com>, "Roy Zang" <roy.zang@nxp.com>,
"Kunihiko Hayashi" <hayashi.kunihiko@socionext.com>,
"Masami Hiramatsu" <mhiramat@kernel.org>,
"Kishon Vijay Abraham I" <kishon@kernel.org>,
"Jesper Nilsson" <jesper.nilsson@axis.com>,
"Srikanth Thokala" <srikanth.thokala@intel.com>,
linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-renesas-soc@vger.kernel.org, linux-arm-msm@vger.kernel.org,
linux-tegra@vger.kernel.org, linux-omap@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linuxppc-dev@lists.ozlabs.org, linux-arm-kernel@axis.com
Subject: Re: [PATCH v10 8/8] PCI: dwc: ep: Remove "core_init_notifier" flag
Date: Mon, 25 Mar 2024 19:47:06 +0530 [thread overview]
Message-ID: <20240325141706.GD2938@thinkpad> (raw)
In-Reply-To: <Zf1xTkuK8yBZXmQ0@ryzen>
On Fri, Mar 22, 2024 at 12:53:50PM +0100, Niklas Cassel wrote:
> On Thu, Mar 14, 2024 at 01:18:06PM +0530, Manivannan Sadhasivam wrote:
> > "core_init_notifier" flag is set by the glue drivers requiring refclk from
> > the host to complete the DWC core initialization. Also, those drivers will
> > send a notification to the EPF drivers once the initialization is fully
> > completed using the pci_epc_init_notify() API. Only then, the EPF drivers
> > will start functioning.
> >
> > For the rest of the drivers generating refclk locally, EPF drivers will
> > start functioning post binding with them. EPF drivers rely on the
> > 'core_init_notifier' flag to differentiate between the drivers.
> > Unfortunately, this creates two different flows for the EPF drivers.
> >
> > So to avoid that, let's get rid of the "core_init_notifier" flag and follow
> > a single initialization flow for the EPF drivers. This is done by calling
> > the dw_pcie_ep_init_notify() from all glue drivers after the completion of
> > dw_pcie_ep_init_registers() API. This will allow all the glue drivers to
> > send the notification to the EPF drivers once the initialization is fully
> > completed.
> >
> > Only difference here is that, the drivers requiring refclk from host will
> > send the notification once refclk is received, while others will send it
> > during probe time itself.
> >
> > But this also requires the EPC core driver to deliver the notification
> > after EPF driver bind. Because, the glue driver can send the notification
> > before the EPF drivers bind() and in those cases the EPF drivers will miss
> > the event. To accommodate this, EPC core is now caching the state of the
> > EPC initialization in 'init_complete' flag and pci-ep-cfs driver sends the
> > notification to EPF drivers based on that after each EPF driver bind.
> >
> > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > ---
> > drivers/pci/controller/dwc/pci-dra7xx.c | 2 ++
> > drivers/pci/controller/dwc/pci-imx6.c | 2 ++
> > drivers/pci/controller/dwc/pci-keystone.c | 2 ++
> > drivers/pci/controller/dwc/pci-layerscape-ep.c | 2 ++
> > drivers/pci/controller/dwc/pcie-artpec6.c | 2 ++
> > drivers/pci/controller/dwc/pcie-designware-plat.c | 2 ++
> > drivers/pci/controller/dwc/pcie-keembay.c | 2 ++
> > drivers/pci/controller/dwc/pcie-qcom-ep.c | 1 -
> > drivers/pci/controller/dwc/pcie-rcar-gen4.c | 2 ++
> > drivers/pci/controller/dwc/pcie-tegra194.c | 1 -
> > drivers/pci/controller/dwc/pcie-uniphier-ep.c | 2 ++
> > drivers/pci/endpoint/functions/pci-epf-test.c | 18 +++++-------------
> > drivers/pci/endpoint/pci-ep-cfs.c | 9 +++++++++
> > drivers/pci/endpoint/pci-epc-core.c | 22 ++++++++++++++++++++++
> > include/linux/pci-epc.h | 7 ++++---
> > 15 files changed, 58 insertions(+), 18 deletions(-)
>
> FWIW:
> Tested-by: Niklas Cassel <cassel@kernel.org>
>
>
>
> However, when looking at this, I was surprised that you never call something
> that will set:
> init_complete = false;
> from e.g. dw_pcie_ep_deinit() or dw_pcie_ep_cleanup().
>
> I saw that you do seem to set
> init_complete = false;
> in your other follow up series that is based on this one.
>
> What will happen if you run with only this series merged (without your
> follow up series), on a platform that used to have .core_init_notifier?
>
> If you do remove and recreate the symlink on a platform with external
> refclk, since you never set init_complete to false, you could trigger
> EPF core_init callback, e.g. pci_epf_test_core_init() to be called,
> which will do DBI writes even when there is no refclk.
>
> E.g. (on a platform with external refclk):
> 1) Create symlink to pci-epf-test in configfs.
> 2) Start RC, your EPC driver will call ep_init_notifiy() when perst
> deasserts.
> 3) Run pci-epf-test.
> 4) Remove the pci-epf-test symlink
> 5) Shutdown RC
> 6) Create symlink to pci-epf-test in configfs.
> This will see that init_complete is true, and will do DBI writes
> which will crash your system, since you don't have a refclk.
>
> Perhaps you should move the patch that calls a function that sets
> init_complete = false;
> to this series, so that this crash is not possible?
>
Good catch! But moving that patch to this series requires moving some other
patches as well. So in the meantime, I'll set this flag to false in
dw_pcie_ep_cleanup().
[...]
> > diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
> > index 18c80002d3bd..fc0282b0d626 100644
> > --- a/drivers/pci/endpoint/functions/pci-epf-test.c
> > +++ b/drivers/pci/endpoint/functions/pci-epf-test.c
[...]
> > - if (!core_init_notifier) {
> > - ret = pci_epf_test_core_init(epf);
> > - if (ret)
> > - return ret;
> > - }
> > -
>
> While you did fix up all DWC based drivers, the non-DWC EPC drivers that
> did not have epc_features->core_init_notifier before this patch:
>
> drivers/pci/controller/cadence/pcie-cadence-ep.c:#include <linux/pci-epc.h>
> drivers/pci/controller/pcie-rcar-ep.c:#include <linux/pci-epc.h>
> drivers/pci/controller/pcie-rockchip-ep.c:#include <linux/pci-epc.h>
>
> I don't think that they will work with pci-epf-test anymore, since AFAICT,
> you did not add a call to: pci_epc_init_notify() or similar in these EPC drivers.
> (Like this patch does to all the DWC-based drivers without a core_init_notifier.)
>
Doh, yeah I completely missed these. Thanks for pointing out. Will add the
notify_init call in next version.
- Mani
--
மணிவண்ணன் சதாசிவம்
prev parent reply other threads:[~2024-03-25 14:17 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-14 7:47 [PATCH v10 0/8] PCI: dwc: ep: Fix DBI access failure for drivers requiring refclk from host Manivannan Sadhasivam
2024-03-14 7:47 ` [PATCH v10 1/8] " Manivannan Sadhasivam
2024-03-14 7:48 ` [PATCH v10 2/8] PCI: dwc: ep: Add Kernel-doc comments for APIs Manivannan Sadhasivam
2024-03-22 11:02 ` Niklas Cassel
2024-03-14 7:48 ` [PATCH v10 3/8] PCI: dwc: ep: Remove deinit() callback from struct dw_pcie_ep_ops Manivannan Sadhasivam
2024-03-14 7:48 ` [PATCH v10 4/8] PCI: dwc: ep: Rename dw_pcie_ep_exit() to dw_pcie_ep_deinit() Manivannan Sadhasivam
2024-03-14 7:48 ` [PATCH v10 5/8] PCI: dwc: ep: Introduce dw_pcie_ep_cleanup() API for drivers supporting PERST# Manivannan Sadhasivam
2024-03-14 7:48 ` [PATCH v10 6/8] PCI: dwc: ep: Rename dw_pcie_ep_init_complete() to dw_pcie_ep_init_registers() Manivannan Sadhasivam
2024-03-14 7:48 ` [PATCH v10 7/8] PCI: dwc: ep: Call dw_pcie_ep_init_registers() API directly from all glue drivers Manivannan Sadhasivam
2024-03-22 11:03 ` Niklas Cassel
2024-03-14 7:48 ` [PATCH v10 8/8] PCI: dwc: ep: Remove "core_init_notifier" flag Manivannan Sadhasivam
2024-03-22 11:53 ` Niklas Cassel
2024-03-25 14:17 ` Manivannan Sadhasivam [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240325141706.GD2938@thinkpad \
--to=manivannan.sadhasivam@linaro.org \
--cc=bhelgaas@google.com \
--cc=cassel@kernel.org \
--cc=festevam@gmail.com \
--cc=gustavo.pimentel@synopsys.com \
--cc=hayashi.kunihiko@socionext.com \
--cc=hongxing.zhu@nxp.com \
--cc=jesper.nilsson@axis.com \
--cc=jingoohan1@gmail.com \
--cc=jonathanh@nvidia.com \
--cc=kernel@pengutronix.de \
--cc=kishon@kernel.org \
--cc=kishon@ti.com \
--cc=kw@linux.com \
--cc=l.stach@pengutronix.de \
--cc=linux-arm-kernel@axis.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-imx@nxp.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-omap@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-renesas-soc@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=lpieralisi@kernel.org \
--cc=marek.vasut+renesas@gmail.com \
--cc=mhiramat@kernel.org \
--cc=minghuan.Lian@nxp.com \
--cc=mingkai.hu@nxp.com \
--cc=robh@kernel.org \
--cc=roy.zang@nxp.com \
--cc=s.hauer@pengutronix.de \
--cc=shawnguo@kernel.org \
--cc=srikanth.thokala@intel.com \
--cc=thierry.reding@gmail.com \
--cc=vidyas@nvidia.com \
--cc=vigneshr@ti.com \
--cc=yoshihiro.shimoda.uh@renesas.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).