From: Joe Damato <jdamato@fastly.com>
To: Tariq Toukan <ttoukan.linux@gmail.com>
Cc: Jakub Kicinski <kuba@kernel.org>,
Zhu Yanjun <zyjzyj2000@gmail.com>,
linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
saeedm@nvidia.com, gal@nvidia.com, nalramli@fastly.com,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Leon Romanovsky <leon@kernel.org>,
"open list:MELLANOX MLX5 core VPI driver"
<linux-rdma@vger.kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Tariq Toukan <tariqt@nvidia.com>
Subject: Re: [PATCH net-next 0/1] mlx5: Add netdev-genl queue stats
Date: Thu, 9 May 2024 21:27:11 -0700 [thread overview]
Message-ID: <Zj2iHwEO4NLvCT06@LQ3V64L9R2> (raw)
In-Reply-To: <Zj1qxgElgHhtxj4h@LQ3V64L9R2>
On Thu, May 09, 2024 at 05:31:06PM -0700, Joe Damato wrote:
> On Thu, May 09, 2024 at 01:16:15PM +0300, Tariq Toukan wrote:
> >
> >
> > On 09/05/2024 9:30, Joe Damato wrote:
> > > On Wed, May 08, 2024 at 07:08:39PM -0700, Jakub Kicinski wrote:
> > > > On Thu, 9 May 2024 01:57:52 +0000 Joe Damato wrote:
> > > > > If I'm following that right and understanding mlx5 (two things I am
> > > > > unlikely to do simultaneously), that sounds to me like:
> > > > >
> > > > > - mlx5e_get_queue_stats_rx and mlx5e_get_queue_stats_tx check if i <
> > > > > priv->channels.params.num_channels (instead of priv->stats_nch),
> > > >
> > > > Yes, tho, not sure whether the "if i < ...num_channels" is even
> > > > necessary, as core already checks against real_num_rx_queues.
> > > >
> > > > > and when
> > > > > summing mlx5e_sq_stats in the latter function, it's up to
> > > > > priv->channels.params.mqprio.num_tc instead of priv->max_opened_tc.
> > > > >
> > > > > - mlx5e_get_base_stats accumulates and outputs stats for everything from
> > > > > priv->channels.params.num_channels to priv->stats_nch, and
> > > >
> > > > I'm not sure num_channels gets set to 0 when device is down so possibly
> > > > from "0 if down else ...num_channels" to stats_nch.
> > >
> > > Yea, you were right:
> > >
> > > if (priv->channels.num == 0)
> > > i = 0;
> > > else
> > > i = priv->channels.params.num_channels;
> > > for (; i < priv->stats_nch; i++) {
> > >
> > > Seems to be working now when I adjust the queue count and the test is
> > > passing as I adjust the queue count up or down. Cool.
> > >
> >
> > I agree that get_base should include all inactive queues stats.
> > But it's not straight forward to implement.
> >
> > A few guiding points:
> >
> > Use mlx5e_get_dcb_num_tc(params) for current num_tc.
> >
> > txq_ix (within the real_num_tx_queues) is calculated by c->ix + tc *
> > params->num_channels.
> >
> > The txqsq stats struct is chosen by channel_stats[c->ix]->sq[tc].
> >
> > It means, in the base stats you should include SQ stats for:
> > 1. all SQs of non-active channels, i.e. ch in [params.num_channels,
> > priv->stats_nch), tc in [0, priv->max_opened_tc).
> > 2. all SQs of non-active TCs in active channels [0, params.num_channels), tc
> > in [mlx5e_get_dcb_num_tc(params), priv->max_opened_tc).
> >
> > Now I actually see that the patch has issues in mlx5e_get_queue_stats_tx.
> > You should not loop over all TCs of channel index i.
> > You must do a reverse mapping from "i" to the pair/tuple [ch_ix, tc], and
> > then access a single TXQ stats by priv->channel_stats[ch_ix].sq[tc].
>
> It looks like txq2sq probably will help with this?
>
> Something like:
>
> for (j = 0; j < mlx5e_get_dcb_num_tc(); j++) {
> sq = priv->txq2sq[j];
> if (sq->ch_ix == i) {
> /* this sq->stats is what I need */
> }
> }
>
> Is that right?
This was incorrect, but I think I got it in the v2 I just sent out. When
you have the time, please take a look at that version.
Thanks for the guidance, it was very helpful.
> Not sure if I'm missing something obvious here, sorry, I've been puzzling
> over the mlx5 source for a bit.
>
> BTW: kind of related but in mlx5e_alloc_txqsq the int tc param is unused (I
> think). It might be helpful to struct mlx5e_txqsq to have a tc field and
> then in mlx5e_alloc_txqsq:
>
> sq->tc = tc;
>
> Not sure if that'd be helpful in general, but I could send that as a
> separate patch.
next prev parent reply other threads:[~2024-05-10 4:27 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-03 2:25 [PATCH net-next 0/1] mlx5: Add netdev-genl queue stats Joe Damato
2024-05-03 2:25 ` [PATCH net-next 1/1] net/mlx5e: Add per queue netdev-genl stats Joe Damato
2024-05-03 10:55 ` [PATCH net-next 0/1] mlx5: Add netdev-genl queue stats Zhu Yanjun
2024-05-03 18:43 ` Joe Damato
2024-05-03 21:58 ` Jakub Kicinski
2024-05-03 23:53 ` Joe Damato
2024-05-04 0:34 ` Jakub Kicinski
2024-05-06 18:04 ` Joe Damato
2024-05-08 21:40 ` Tariq Toukan
2024-05-08 23:24 ` Joe Damato
2024-05-09 0:56 ` Jakub Kicinski
2024-05-09 1:57 ` Joe Damato
2024-05-09 2:08 ` Jakub Kicinski
2024-05-09 4:11 ` Joe Damato
2024-05-09 6:30 ` Joe Damato
2024-05-09 10:16 ` Tariq Toukan
2024-05-09 23:14 ` Joe Damato
2024-05-10 0:31 ` Joe Damato
2024-05-10 4:27 ` Joe Damato [this message]
2024-05-09 9:42 ` Tariq Toukan
2024-05-09 23:06 ` Joe Damato
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Zj2iHwEO4NLvCT06@LQ3V64L9R2 \
--to=jdamato@fastly.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=gal@nvidia.com \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=nalramli@fastly.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=saeedm@nvidia.com \
--cc=tariqt@nvidia.com \
--cc=ttoukan.linux@gmail.com \
--cc=zyjzyj2000@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).