From: Koichiro Den <den@valinux.co.jp>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: "Jon Mason" <jdmason@kudzu.us>,
"Dave Jiang" <dave.jiang@intel.com>,
"Allen Hubbe" <allenbh@gmail.com>,
"Manivannan Sadhasivam" <mani@kernel.org>,
"Krzysztof Wilczyński" <kwilczynski@kernel.org>,
"Kishon Vijay Abraham I" <kishon@kernel.org>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Frank Li" <Frank.Li@nxp.com>,
"Jerome Brunet" <jbrunet@baylibre.com>,
"Lorenzo Pieralisi" <lpieralisi@kernel.org>,
"Niklas Cassel" <cassel@kernel.org>,
linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,
ntb@lists.linux.dev
Subject: Re: [PATCH v4 12/12] NTB: epf: Implement db_vector_count/mask for doorbells
Date: Wed, 24 Jun 2026 10:35:43 +0900 [thread overview]
Message-ID: <zo4ujr5n2rdlkhnssq4b23bbeobz3yyuugchghkszyifw4dj2w@5qsvn5vhdedj> (raw)
In-Reply-To: <20260623163134.GA813775@bhelgaas>
On Tue, Jun 23, 2026 at 11:31:34AM -0500, Bjorn Helgaas wrote:
> On Wed, May 13, 2026 at 11:49:23AM +0900, Koichiro Den wrote:
> > Implement .db_vector_count and .db_vector_mask so ntb core/clients can
> > map doorbell events to per-vector work.
> >
> > Report vectors as 0..(db_count - 2) (skipping the unused slot) and return
> > BIT_ULL(db_vector) for the corresponding doorbell bit. Use
> > ntb_epf_db_vector_count() for bounds checks in ntb_epf_db_vector_mask(), so
> > the same lower-bound guard is applied before building the bitmask.
> >
> > Signed-off-by: Koichiro Den <den@valinux.co.jp>
> > ---
> > Changes since v3:
> > - Reuse ntb_epf_db_vector_count() from ntb_epf_db_vector_mask() for bounds.
> > - Return 0 when db_count is below NTB_EPF_MIN_DB_COUNT.
> > - Drop Reviewed-by tags due to the changes.
> >
> > drivers/ntb/hw/epf/ntb_hw_epf.c | 32 ++++++++++++++++++++++++++++++++
> > 1 file changed, 32 insertions(+)
> >
> > diff --git a/drivers/ntb/hw/epf/ntb_hw_epf.c b/drivers/ntb/hw/epf/ntb_hw_epf.c
> > index 10618e462229..af5755472842 100644
> > --- a/drivers/ntb/hw/epf/ntb_hw_epf.c
> > +++ b/drivers/ntb/hw/epf/ntb_hw_epf.c
> > @@ -434,6 +434,36 @@ static u64 ntb_epf_db_valid_mask(struct ntb_dev *ntb)
> > return ntb_ndev(ntb)->db_valid_mask;
> > }
> >
> > +static int ntb_epf_db_vector_count(struct ntb_dev *ntb)
> > +{
> > + struct ntb_epf_dev *ndev = ntb_ndev(ntb);
> > + unsigned int db_count = ndev->db_count;
> > +
> > + /*
> > + * db_count includes an extra skipped slot due to the legacy
> > + * doorbell layout. Expose only the real doorbell vectors.
> > + */
> > + if (db_count < NTB_EPF_MIN_DB_COUNT)
> > + return 0;
> > +
> > + return db_count - 1;
> > +}
> > +
> > +static u64 ntb_epf_db_vector_mask(struct ntb_dev *ntb, int db_vector)
> > +{
> > + int nr_vec;
> > +
> > + /*
> > + * db_count includes one skipped slot in the legacy layout. Valid
> > + * doorbell vectors are therefore [0 .. (db_count - 2)].
> > + */
> > + nr_vec = ntb_epf_db_vector_count(ntb);
> > + if (db_vector < 0 || db_vector >= nr_vec)
> > + return 0;
> > +
> > + return BIT_ULL(db_vector);
>
Hi Bjorn,
> This all ends up looking slightly different from pci-epf-vntb.c, and
> it's not obvious whether the differences are essential or superfluous.
I don't think there is an actual bug here.
The thing is that "db_count" has meant slightly different things in
pci-epf-vntb.c and ntb_hw_epf.c since their introduction.
In pci-epf-vntb.c, db_count is the total number including the link slot and the
historically unused/skipped slot. It is also exposed via configfs, so changing
that meaning now would affect existing users' configfs settings.
So vntb_epf_db_vector_count() subtracts EPF_IRQ_DB_START because the NTB core
should only see the usable doorbell vectors via .db_vector_count().
>
> pci-epf-vntb.c:
>
> enum EPF_IRQ_DB_START # value 2
> #define MIN_DB_COUNT (EPF_IRQ_DB_START + 1) # value 3
>
> vntb_epf_db_vector_count() comment says "db_count is total number of
> doorbell slots exposed to peer, including reserved ones". But it
> returns "db_count - EPF_IRQ_DB_START", which suggests that the
> reserves slots are not exposed?
>
> vntb_epf_db_vector_count() comment lists two reserved slots
>
> vntb_epf_db_vector_count() returns "db_count - EPF_IRQ_DB_START"
>
> vntb_epf_db_vector_mask() comment mentions "two reserved slots"
>
> ntb_hw_epf.c:
>
> enum EPF_IRQ_DB_START # value 2
> #define NTB_EPF_MIN_DB_COUNT 3 # value 3, not based on EPF_IRQ_DB_START
>
> ntb_epf_db_vector_count() comment says "expose only real doorbell
> vectors". I guess this *excludes* the reserved ones?
Yes.
>
> ntb_epf_db_vector_count() comment mentions "an extra skipped slot"
> but apparently actually accounts for *two* slots
> (NTB_EPF_MIN_DB_COUNT==3)
>
> ntb_epf_db_vector_count() returns "db_count - 1" # should it be -2?
This is intentionally -1.
ntb_hw_epf sets "ndev->db_count = irq - 1" where the "-1" accounts for the link
slot. That goes back to the initial ntb_hw_epf support commit 812ce2f8d14ea. So
the number of usable doorbell vectors is "db_count - 1", subtracting the
remaining unused/skipped slot.
>
> ntb_epf_db_vector_mask() comment mentions "one skipped slot"
>
> I don't want to change anything during the v7.2 merge window, but if
> there are any actual bugs here, they could be fixed after v7.2-rc1.
> If there are no bugs but this could be clarified to reduce confusion,
> we could do that for v7.3.
If preferred, I can send a follow-up patch that adds comments to make the
distinction clearer.
Best regards,
Koichiro
next prev parent reply other threads:[~2026-06-24 1:35 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-05-13 2:49 [PATCH v4 00/12] PCI: endpoint: pci-epf-vntb / NTB: epf: Enable per-doorbell bit handling Koichiro Den
2026-05-13 2:49 ` [PATCH v4 01/12] PCI: endpoint: pci-epf-vntb: Document legacy MSI doorbell offset Koichiro Den
2026-05-13 2:49 ` [PATCH v4 02/12] PCI: endpoint: pci-epf-vntb: Defer pci_epc_raise_irq() out of atomic context Koichiro Den
2026-05-14 18:53 ` Frank Li
2026-05-13 2:49 ` [PATCH v4 03/12] PCI: endpoint: pci-epf-vntb: Report 0-based doorbell vector via ntb_db_event() Koichiro Den
2026-05-13 2:49 ` [PATCH v4 04/12] PCI: endpoint: pci-epf-vntb: Reject unusable doorbell counts Koichiro Den
2026-05-14 18:55 ` Frank Li
2026-05-13 2:49 ` [PATCH v4 05/12] PCI: endpoint: pci-epf-vntb: Guard configfs writes after EPC attach Koichiro Den
2026-05-14 18:57 ` Frank Li
2026-05-13 2:49 ` [PATCH v4 06/12] PCI: endpoint: pci-epf-vntb: Exclude reserved slots from db_valid_mask Koichiro Den
2026-05-13 2:49 ` [PATCH v4 07/12] PCI: endpoint: pci-epf-vntb: Implement db_vector_count/mask for doorbells Koichiro Den
2026-05-14 19:02 ` Frank Li
2026-05-13 2:49 ` [PATCH v4 08/12] NTB: epf: Document legacy doorbell slot offset in ntb_epf_peer_db_set() Koichiro Den
2026-05-13 2:49 ` [PATCH v4 09/12] NTB: epf: Make db_valid_mask cover only real doorbell bits Koichiro Den
2026-05-26 22:21 ` Dave Jiang
2026-05-13 2:49 ` [PATCH v4 10/12] NTB: epf: Report 0-based doorbell vector via ntb_db_event() Koichiro Den
2026-05-14 19:03 ` Frank Li
2026-05-26 22:28 ` Dave Jiang
2026-05-13 2:49 ` [PATCH v4 11/12] NTB: epf: Fix doorbell bitmask and IRQ vector handling Koichiro Den
2026-05-14 19:06 ` Frank Li
2026-05-26 22:36 ` Dave Jiang
2026-05-13 2:49 ` [PATCH v4 12/12] NTB: epf: Implement db_vector_count/mask for doorbells Koichiro Den
2026-05-26 22:47 ` Dave Jiang
2026-06-23 16:31 ` Bjorn Helgaas
2026-06-24 1:35 ` Koichiro Den [this message]
2026-05-19 14:42 ` [PATCH v4 00/12] PCI: endpoint: pci-epf-vntb / NTB: epf: Enable per-doorbell bit handling Manivannan Sadhasivam
2026-05-19 21:29 ` Dave Jiang
2026-05-21 6:29 ` Koichiro Den
2026-05-26 14:45 ` Dave Jiang
2026-05-26 15:56 ` Koichiro Den
2026-05-26 16:27 ` Dave Jiang
2026-05-27 9:09 ` Manivannan Sadhasivam
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=zo4ujr5n2rdlkhnssq4b23bbeobz3yyuugchghkszyifw4dj2w@5qsvn5vhdedj \
--to=den@valinux.co.jp \
--cc=Frank.Li@nxp.com \
--cc=allenbh@gmail.com \
--cc=bhelgaas@google.com \
--cc=cassel@kernel.org \
--cc=dave.jiang@intel.com \
--cc=helgaas@kernel.org \
--cc=jbrunet@baylibre.com \
--cc=jdmason@kudzu.us \
--cc=kishon@kernel.org \
--cc=kwilczynski@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=lpieralisi@kernel.org \
--cc=mani@kernel.org \
--cc=ntb@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).