From: Bjorn Helgaas <helgaas@kernel.org>
To: "Bowman, Terry" <terry.bowman@amd.com>
Cc: dave@stgolabs.net, jonathan.cameron@huawei.com,
dave.jiang@intel.com, alison.schofield@intel.com,
dan.j.williams@intel.com, bhelgaas@google.com,
shiju.jose@huawei.com, ming.li@zohomail.com,
Smita.KoralahalliChannabasappa@amd.com, rrichter@amd.com,
dan.carpenter@linaro.org, PradeepVineshReddy.Kodamati@amd.com,
lukas@wunner.de, Benjamin.Cheatham@amd.com,
sathyanarayanan.kuppuswamy@linux.intel.com,
linux-cxl@vger.kernel.org, alucerop@amd.com, ira.weiny@intel.com,
linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org
Subject: Re: [PATCH v13 02/25] PCI/CXL: Introduce pcie_is_cxl()
Date: Mon, 8 Dec 2025 12:33:56 -0600 [thread overview]
Message-ID: <20251208183356.GA3416294@bhelgaas> (raw)
In-Reply-To: <a5d4f527-1a66-4c1e-9311-b8f3fe3badf1@amd.com>
On Mon, Dec 08, 2025 at 09:26:29AM -0600, Bowman, Terry wrote:
> On 12/5/2025 6:45 PM, Bjorn Helgaas wrote:
> > On Mon, Nov 03, 2025 at 06:09:38PM -0600, Terry Bowman wrote:
> >> CXL and AER drivers need the ability to identify CXL devices.
> >>
> >> Introduce set_pcie_cxl() with logic checking for CXL.mem or CXL.cache
> >> status in the CXL Flexbus DVSEC status register. The CXL Flexbus DVSEC
> >> presence is used because it is required for all the CXL PCIe devices.[1]
Spec usage seems to be "Flex Bus", which would help searches.
> >> Add boolean 'struct pci_dev::is_cxl' with the purpose to cache the CXL
> >> CXL.cache and CXl.mem status.
> >>
> >> In the case the device is an EP or USP, call set_pcie_cxl() on behalf of
> >> the parent downstream device. Once a device is created there is
> >> possibilty the parent training or CXL state was updated as well. This
> >> will make certain the correct parent CXL state is cached.
> ...
> >> + pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_FLEXBUS_STATUS_OFFSET, &cap);
> >> +
> >> + dev->is_cxl = FIELD_GET(PCI_DVSEC_CXL_FLEXBUS_STATUS_CACHE_MASK, cap) ||
> >> + FIELD_GET(PCI_DVSEC_CXL_FLEXBUS_STATUS_MEM_MASK, cap);
> > Not sure the "_MASK" and "_OFFSET" on the end of all these #defines is
> > really needed. Other items in pci_regs.h typically don't include
> > them, and these names get really long.
>
> These were moved over from local CXL header. As a result they are not
> consistent in the usage of offset and mask. Would you like for this to
> be made the same ? This would be to change all points where used.
I think it's worth dropping "_MASK" and "_OFFSET" to reduce the
length.
> >> + * Update parent's CXL state because alternate protocol training
> >> + * may have changed
> >
> > What is the event that changes alternate protocol training? The
> > commit log says "once a device is created", but I don't know what that
> > means in terms of hardware.
>
> There is potential an upstream device (switch) was hotpluged and in
> the case of the alt training retries may not be correctly cached in
> pci_dev::is_cxl.
I assume this refers to the Alternate Protocol Negotiation (PCIe r7.0,
seg 4.2.5.2), which happens during link training.
IIUC, the problem here is that we may enumerate a bridge before its
secondary link has trained. In that case, the Cache_Enabled and
Mem_Enabled bits in the bridge's Flex Bus Port Status may be zero.
After the secondary link has trained, those bits may be set based on
the alternate protocol negotiation, so we need to re-read the bridges
Port Status. Annoying that these are documented as RO when they don't
really seem to be read-only.
Maybe there's no point in reading the bridge Flex Bus Port Status at
all until we enumerate a device *below* the bridge, e.g, something
like this:
if (!pci_is_pcie(dev))
return;
if (!(pci_pcie_type(dev) == PCI_EXP_TYPE_ENDPOINT ||
pci_pcie_type(dev) == PCI_EXP_TYPE_UPSTREAM))
return;
/* Parent's CXL status only valid when link to child has trained */
bridge = pci_upstream_bridge(dev);
dvsec = pci_find_dvsec_capability(bridge, PCI_VENDOR_ID_CXL,
PCI_DVSEC_CXL_FLEXBUS_PORT);
if (!dvsec)
return;
pci_read_config_word(bridge, dvsec + PCI_DVSEC_CXL_FLEXBUS_STATUS_OFFSET, &cap);
...
next prev parent reply other threads:[~2025-12-08 18:33 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-11-04 0:09 [PATCH v13 00/25] Enable CXL PCIe Port Protocol Error handling and logging Terry Bowman
2025-11-04 0:09 ` [PATCH v13 01/25] CXL/PCI: Move CXL DVSEC definitions into uapi/linux/pci_regs.h Terry Bowman
2025-12-06 0:31 ` Bjorn Helgaas
2025-12-06 0:41 ` dan.j.williams
2025-12-06 1:56 ` Bjorn Helgaas
2025-12-06 4:56 ` dan.j.williams
2025-11-04 0:09 ` [PATCH v13 02/25] PCI/CXL: Introduce pcie_is_cxl() Terry Bowman
2025-12-06 0:45 ` Bjorn Helgaas
2025-12-08 15:26 ` Bowman, Terry
2025-12-08 18:33 ` Bjorn Helgaas [this message]
2025-11-04 0:09 ` [PATCH v13 03/25] cxl/pci: Remove unnecessary CXL Endpoint handling helper functions Terry Bowman
2025-11-04 0:09 ` [PATCH v13 04/25] cxl/pci: Remove unnecessary CXL RCH " Terry Bowman
2025-11-04 0:09 ` [PATCH v13 05/25] cxl: Remove CXL VH handling in CONFIG_PCIEAER_CXL conditional blocks from core/pci.c Terry Bowman
2025-11-04 0:09 ` [PATCH v13 06/25] cxl: Move CXL driver's RCH error handling into core/ras_rch.c Terry Bowman
2025-11-04 0:09 ` [PATCH v13 07/25] CXL/AER: Replace device_lock() in cxl_rch_handle_error_iter() with guard() lock Terry Bowman
2025-12-08 18:05 ` Bjorn Helgaas
2025-11-04 0:09 ` [PATCH v13 08/25] CXL/AER: Move AER drivers RCH error handling into pcie/aer_cxl_rch.c Terry Bowman
2025-12-08 18:06 ` Bjorn Helgaas
2025-12-08 21:28 ` Bowman, Terry
2025-12-10 21:57 ` Bowman, Terry
2025-12-10 23:12 ` Bjorn Helgaas
2025-11-04 0:09 ` [PATCH v13 09/25] PCI/AER: Report CXL or PCIe bus error type in trace logging Terry Bowman
2025-11-04 0:09 ` [PATCH v13 10/25] cxl/pci: Update RAS handler interfaces to also support CXL Ports Terry Bowman
2025-11-04 0:09 ` [PATCH v13 11/25] cxl/pci: Log message if RAS registers are unmapped Terry Bowman
2025-11-04 0:09 ` [PATCH v13 12/25] cxl/pci: Unify CXL trace logging for CXL Endpoints and CXL Ports Terry Bowman
2025-11-04 0:09 ` [PATCH v13 13/25] cxl/pci: Update cxl_handle_cor_ras() to return early if no RAS errors Terry Bowman
2025-11-04 0:09 ` [PATCH v13 14/25] cxl/pci: Map CXL Endpoint Port and CXL Switch Port RAS registers Terry Bowman
2025-11-04 0:09 ` [PATCH v13 15/25] CXL/PCI: Introduce PCI_ERS_RESULT_PANIC Terry Bowman
2025-11-04 0:09 ` [PATCH v13 16/25] CXL/AER: Introduce pcie/aer_cxl_vh.c in AER driver for forwarding CXL errors Terry Bowman
2025-12-08 18:36 ` Bjorn Helgaas
2025-11-04 0:09 ` [PATCH v13 17/25] cxl: Introduce cxl_pci_drv_bound() to check for bound driver Terry Bowman
2025-11-04 0:09 ` [PATCH v13 18/25] cxl: Change CXL handlers to use guard() instead of scoped_guard() Terry Bowman
2025-11-04 0:09 ` [PATCH v13 19/25] cxl/pci: Introduce CXL protocol error handlers for Endpoints Terry Bowman
2025-11-04 0:09 ` [PATCH v13 20/25] CXL/PCI: Introduce CXL Port protocol error handlers Terry Bowman
2025-12-08 18:37 ` Bjorn Helgaas
2025-12-09 15:17 ` Bowman, Terry
2025-11-04 0:09 ` [PATCH v13 21/25] PCI/AER: Dequeue forwarded CXL error Terry Bowman
2025-11-04 0:09 ` [PATCH v13 22/25] CXL/PCI: Export and rename merge_result() to pci_ers_merge_result() Terry Bowman
2025-12-08 18:38 ` Bjorn Helgaas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20251208183356.GA3416294@bhelgaas \
--to=helgaas@kernel.org \
--cc=Benjamin.Cheatham@amd.com \
--cc=PradeepVineshReddy.Kodamati@amd.com \
--cc=Smita.KoralahalliChannabasappa@amd.com \
--cc=alison.schofield@intel.com \
--cc=alucerop@amd.com \
--cc=bhelgaas@google.com \
--cc=dan.carpenter@linaro.org \
--cc=dan.j.williams@intel.com \
--cc=dave.jiang@intel.com \
--cc=dave@stgolabs.net \
--cc=ira.weiny@intel.com \
--cc=jonathan.cameron@huawei.com \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=lukas@wunner.de \
--cc=ming.li@zohomail.com \
--cc=rrichter@amd.com \
--cc=sathyanarayanan.kuppuswamy@linux.intel.com \
--cc=shiju.jose@huawei.com \
--cc=terry.bowman@amd.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).