From: "Július Milan" <julius.milan.22@gmail.com>
To: Magnus Karlsson <magnus.karlsson@gmail.com>
Cc: xdp-newbies@vger.kernel.org
Subject: Re: thread safe, lock free AF_XDP setup
Date: Mon, 10 Jul 2023 16:39:25 +0200 [thread overview]
Message-ID: <CANaGk7tV=uj1cm_xROj6HRH=D9BtstmXUu-8ZeGBrX8_LG+u6w@mail.gmail.com> (raw)
In-Reply-To: <CAJ8uoz3YrC9W10rZM_qJivW51Ly+dBb5_UZ2jHgeFC4+1jzAww@mail.gmail.com>
Thanks a lot Magnus
Now I fully understand.
Julius
On Mon, Jul 10, 2023 at 3:57 PM Magnus Karlsson
<magnus.karlsson@gmail.com> wrote:
>
> On Mon, 10 Jul 2023 at 15:51, Július Milan <julius.milan.22@gmail.com> wrote:
> >
> > Hi all
> >
> > I am missing a full understanding of what really is thread safe and
> > what is not, when using AF_XDP.
> >
> > RX and TX operations look obvious as every xsk socket has its own RX
> > and TX rings and so I can have multiple threads, as long as they don't
> > try to use the same ring of the same socket concurrently, that's the
> > only limitation (Am I correct yet?). So I can have for example 1:1
> > mapping of thread to xsk socket to avoid locking.
>
> That is correct.
>
> > However what about UMEM rings, i.e. Completion and Fill queue?
> > About ring operations as __peek, __reserve ..., it is written (for ex.
> > here: https://www.kernel.org/doc/html/latest/networking/af_xdp.html#rings),
> > that they are not thread safe.
> > So it seems that I should use mutex to access them.
> > But I can see in:
> > https://github.com/xdp-project/bpf-examples/blob/master/AF_XDP-forwarding/xsk_fwd.c#L526
> > That they are filled / drained without locking, how is it then?
>
> The documentation above mentions that there are two shared_umem modes:
> one if you want to share the same netdev/queue_id and another one for
> all the other cases. The forwarding example uses the latter which has
> one fill ring and one completion ring per socket. So no locking
> needed. The former only has one fill and completion ring pair per
> umem, so you need to take care of mutual exclusion yourself in this
> case.
>
> > Does it make a difference if the socket is created by
> > xsk_socket__create_shared or xsk_socket__create?
>
> The former one sets the XDP_SHARED_UMEM flag. If you want to share the
> umem between several sockets, you need to use this one.
>
> > What is the limitation that cannot be broken, as I described it for RX
> > / TX? (if correctly)
> >
> > Best Regards
> > Julius
prev parent reply other threads:[~2023-07-10 14:39 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-07-10 13:43 thread safe, lock free AF_XDP setup Július Milan
2023-07-10 13:57 ` Magnus Karlsson
2023-07-10 14:39 ` Július Milan [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CANaGk7tV=uj1cm_xROj6HRH=D9BtstmXUu-8ZeGBrX8_LG+u6w@mail.gmail.com' \
--to=julius.milan.22@gmail.com \
--cc=magnus.karlsson@gmail.com \
--cc=xdp-newbies@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).