From: Willem de Bruijn <willemdebruijn.kernel@gmail.com>
To: Richard Gobert <richardbgobert@gmail.com>, richardbgobert@gmail.com
Cc: alexander.duyck@gmail.com, davem@davemloft.net,
dsahern@kernel.org, edumazet@google.com, kuba@kernel.org,
linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
netdev@vger.kernel.org, pabeni@redhat.com, shuah@kernel.org,
willemdebruijn.kernel@gmail.com
Subject: Re: [PATCH net-next v9 2/3] net: gro: move L3 flush checks to tcp_gro_receive and udp_gro_receive_segment
Date: Thu, 09 May 2024 10:24:55 -0400 [thread overview]
Message-ID: <663cdcb73953_126914294b5@willemb.c.googlers.com.notmuch> (raw)
In-Reply-To: <20240507163021.130466-1-richardbgobert@gmail.com>
Richard Gobert wrote:
> {inet,ipv6}_gro_receive functions perform flush checks (ttl, flags,
> iph->id, ...) against all packets in a loop. These flush checks are used in
> all merging UDP and TCP flows.
>
> These checks need to be done only once and only against the found p skb,
> since they only affect flush and not same_flow.
>
> This patch leverages correct network header offsets from the cb for both
> outer and inner network headers - allowing these checks to be done only
> once, in tcp_gro_receive and udp_gro_receive_segment. As a result,
> NAPI_GRO_CB(p)->flush is not used at all. In addition, flush_id checks are
> more declarative and contained in inet_gro_flush, thus removing the need
> for flush_id in napi_gro_cb.
>
> This results in less parsing code for non-loop flush tests for TCP and UDP
> flows.
>
> To make sure results are not within noise range - I've made netfilter drop
> all TCP packets, and measured CPU performance in GRO (in this case GRO is
> responsible for about 50% of the CPU utilization).
>
> perf top while replaying 64 parallel IP/TCP streams merging in GRO:
> (gro_receive_network_flush is compiled inline to tcp_gro_receive)
> net-next:
> 6.94% [kernel] [k] inet_gro_receive
> 3.02% [kernel] [k] tcp_gro_receive
>
> patch applied:
> 4.27% [kernel] [k] tcp_gro_receive
> 4.22% [kernel] [k] inet_gro_receive
>
> perf top while replaying 64 parallel IP/IP/TCP streams merging in GRO (same
> results for any encapsulation, in this case inet_gro_receive is top
> offender in net-next)
> net-next:
> 10.09% [kernel] [k] inet_gro_receive
> 2.08% [kernel] [k] tcp_gro_receive
>
> patch applied:
> 6.97% [kernel] [k] inet_gro_receive
> 3.68% [kernel] [k] tcp_gro_receive
>
> Signed-off-by: Richard Gobert <richardbgobert@gmail.com>
> +static inline int inet_gro_flush(const struct iphdr *iph, const struct iphdr *iph2,
> + struct sk_buff *p, bool outer)
> +{
> + const u32 id = ntohl(*(__be32 *)&iph->id);
> + const u32 id2 = ntohl(*(__be32 *)&iph2->id);
> + const u16 ipid_offset = (id >> 16) - (id2 >> 16);
> + const u16 count = NAPI_GRO_CB(p)->count;
> + const u32 df = id & IP_DF;
> + int flush;
> +
> + /* All fields must match except length and checksum. */
> + flush = (iph->ttl ^ iph2->ttl) | (iph->tos ^ iph2->tos) | (df ^ (id2 & IP_DF));
> +
> + if (outer && df)
> + return flush;
if (flush)
return 1;
To be able to avoid the two flush | below?
Or to avoid adding a branch
if (flush | (outer && df))
return 1;
> +
> + /* When we receive our second frame we can make a decision on if we
> + * continue this flow as an atomic flow with a fixed ID or if we use
> + * an incrementing ID.
> + */
> + if (count == 1 && df && !ipid_offset)
> + NAPI_GRO_CB(p)->ip_fixedid = true;
> +
> + if (NAPI_GRO_CB(p)->ip_fixedid && df)
> + return flush | ipid_offset;
> +
> + return flush | (ipid_offset ^ count);
And then simply
if (NAPI_GRO_CB(p)->ip_fixedid)
return ipid_offset;
else
return ipid_offset ^ count;
Since NAPI_GRO_CB(p)->ip_fixedid is only set if DF is set on the first
two segments, and df ^ id2 & IP_DF is tested above, no need to test
that again.
> +}
next prev parent reply other threads:[~2024-05-09 14:24 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-07 16:23 [PATCH net-next v9 0/3] net: gro: remove network_header use, move p->{flush/flush_id} calculations to L4 Richard Gobert
2024-05-07 16:28 ` [PATCH net-next v9 1/3] net: gro: use cb instead of skb->network_header Richard Gobert
2024-05-07 16:30 ` [PATCH net-next v9 2/3] net: gro: move L3 flush checks to tcp_gro_receive and udp_gro_receive_segment Richard Gobert
2024-05-07 18:08 ` Eric Dumazet
2024-05-09 18:58 ` Richard Gobert
2024-05-10 7:01 ` Eric Dumazet
2024-05-10 20:20 ` Richard Gobert
2024-05-09 14:24 ` Willem de Bruijn [this message]
2024-05-09 19:02 ` Richard Gobert
2024-05-07 16:31 ` [PATCH net-next v9 3/3] selftests/net: add flush id selftests Richard Gobert
2024-05-08 0:34 ` [PATCH net-next v9 0/3] net: gro: remove network_header use, move p->{flush/flush_id} calculations to L4 Jakub Kicinski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=663cdcb73953_126914294b5@willemb.c.googlers.com.notmuch \
--to=willemdebruijn.kernel@gmail.com \
--cc=alexander.duyck@gmail.com \
--cc=davem@davemloft.net \
--cc=dsahern@kernel.org \
--cc=edumazet@google.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=richardbgobert@gmail.com \
--cc=shuah@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).