From: Federico Parola <federico.parola@polito.it>
To: Magnus Karlsson <magnus.karlsson@gmail.com>,
Andrey Slastenov <a.slastenov@gmail.com>
Cc: xdp-newbies@vger.kernel.org
Subject: Re: Why choosing AF_XDP (busy polling) over DPDK
Date: Mon, 19 Dec 2022 08:41:23 +0100 [thread overview]
Message-ID: <86f02c52-48bb-6128-fd26-b332dc4769ed@polito.it> (raw)
In-Reply-To: <CAJ8uoz0zo3zH8a_Zq+L5WzDYjgZh7-pU+ubo9MSbmk-NbxKoiw@mail.gmail.com>
On 12/14/22 16:31, Magnus Karlsson wrote:
> On Wed, Dec 14, 2022 at 4:16 PM Federico Parola
> <federico.parola@polito.it> wrote:
>>
>> On 12/14/22 10:00, Magnus Karlsson wrote:
>>> On Wed, Dec 14, 2022 at 9:16 AM Federico Parola
>>> <federico.parola@polito.it> wrote:
>>>>
>>>> Hello everyone,
>>>> in a context in which all traffic is sent to user space, and the main
>>>> concern is performance (so AF_XDP busy polling is the best candidate),
>>>> is there a reason to choose AF_XDP instead of DPDK for packet I/O, given
>>>> that the latter is still much faster?
>>>
>>> The choice is not AF_XDP vs DPDK, since DPDK can run on top of AF_XDP.
>>> The choice is do you want to use user-space drivers or kernel space
>>> drivers. There are many pros and cons with both approaches. The two
>>> main advantages of user-space drivers in DPDK are that they are faster
>>> than their kernel counterparts and you get access to many NIC features
>>> that are not available right now from user-space if you are using a
>>> kernel driver. Kernel drivers, on the other hand, enables you to
>>> deploy your application on any Linux system (from a certain version)
>>> and does not force you into the strict execution model of DPDK (which
>>> is what most people use with user-space drivers). Everything in Linux
>>> is still usable and it is easy to have a system that mixes data-plane
>>> processing with other types of workloads. If the only thing you care
>>> about is max performance of a pure packet processing workload on a
>>> system you have complete control over, the choice is easy: go with a
>>> user-space driver and DPDK. If this is not your ultimate goal, then
>>> XDP and AF_XDP might be something to consider.
>> Thanks for your answer Magnus.
>>
>> - Concerning deploying an application on any Linux system, what could be
>> the limitations of DPDK userspace drivers that AF_XDP doesn't have? The
>> use of hugepages?
>
> That would be one. The requirement of shared memory, forced unity
> mmap() calls that might fail, lack of binary forward compatibility,
> etc.
>
>> - Concerning a system that mixes data-plane processing with other types
>> of workloads, do you refer only to the capability of XDP to steer
>> traffic either towards userspace with AF_XDP or let it proceed to the
>> kernel? Or are there other features of DPDK userspace drivers that would
>> impact this kind of system? Maybe the use of busy polling is too
>> aggressive to share the CPU with other workloads?
>
> Yes, busy-polling and sharing a core is not a good idea in any system.
> But there is a good solution to this in DPDK that uses capabilities in
> newer CPUs. Basically, if you use DPDK you should adhere to its strict
> model of one process per core, otherwise you will likely get into
> trouble. Either by performance dropping significantly because you
> enable things like interrupts or 4K pages in DPDK, or because you do
> things like trying to run to DPDK polling threads on the same core,
> which might result in deadlock. If I am not mistaken, there is a
> concept in DPDK called service cores in which you can share a core
> between a normal process and a DPDK data plane thread. But it requires
> you to program specifically for it.
>
> If you want to steer part of the traffic to the kernel, you are better
> off using XDP. Bouncing things through user-space is expensive, DPDK
> or not.
>
>>>
>>> There are likely some good write-ups on the Internet about this.
>>>
>>> /Magnus
>>>
>>>> Best regards,
>>>> Federico Parola
Thank you very much for this information Magnus.
On 12/14/22 20:46, Andrey Slastenov wrote:> Hello everyone
>
> Check out this short article about why XDP was chosen over DPDK. The
> reasoning for this decision is also explained.
>
>
https://gcore.com/blog/how-we-use-regular-expressions-in-xdp-for-packet-filtering/
>
>
> BR, Andrey
Thanks for sharing this article Andrey.
Best regards,
Federico
next prev parent reply other threads:[~2022-12-19 7:41 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-14 8:08 Why choosing AF_XDP (busy polling) over DPDK Federico Parola
2022-12-14 9:00 ` Magnus Karlsson
2022-12-14 15:16 ` Federico Parola
2022-12-14 15:31 ` Magnus Karlsson
2022-12-19 7:41 ` Federico Parola [this message]
2022-12-14 19:46 ` Andrey Slastenov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=86f02c52-48bb-6128-fd26-b332dc4769ed@polito.it \
--to=federico.parola@polito.it \
--cc=a.slastenov@gmail.com \
--cc=magnus.karlsson@gmail.com \
--cc=xdp-newbies@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).