From: Alexander Lobakin <aleksander.lobakin@intel.com>
To: Robin Murphy <robin.murphy@arm.com>,
Steven Price <steven.price@arm.com>,
Christoph Hellwig <hch@lst.de>
Cc: Eric Dumazet <edumazet@google.com>,
Jakub Kicinski <kuba@kernel.org>,
Marek Szyprowski <m.szyprowski@samsung.com>,
Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
"Rafael J. Wysocki" <rafael@kernel.org>,
Magnus Karlsson <magnus.karlsson@intel.com>,
<nex.sw.ncis.osdt.itp.upstreaming@intel.com>,
<bpf@vger.kernel.org>, <netdev@vger.kernel.org>,
<iommu@lists.linux.dev>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v6 2/7] dma: avoid redundant calls for sync operations
Date: Thu, 9 May 2024 16:43:00 +0200 [thread overview]
Message-ID: <f207da41-5222-4d1d-897d-9b288e33a547@intel.com> (raw)
In-Reply-To: <5d6a26f5-5acb-4c5d-aa11-724399d1348b@arm.com>
From: Robin Murphy <robin.murphy@arm.com>
Date: Thu, 9 May 2024 15:33:13 +0100
> On 09/05/2024 2:43 pm, Steven Price wrote:
>> On 07/05/2024 12:20, Alexander Lobakin wrote:
>>> Quite often, devices do not need dma_sync operations on x86_64 at least.
>>> Indeed, when dev_is_dma_coherent(dev) is true and
>>> dev_use_swiotlb(dev) is false, iommu_dma_sync_single_for_cpu()
>>> and friends do nothing.
>>>
>>> However, indirectly calling them when CONFIG_RETPOLINE=y consumes about
>>> 10% of cycles on a cpu receiving packets from softirq at ~100Gbit rate.
>>> Even if/when CONFIG_RETPOLINE is not set, there is a cost of about 3%.
>>>
>>> Add dev->need_dma_sync boolean and turn it off during the device
>>> initialization (dma_set_mask()) depending on the setup:
>>> dev_is_dma_coherent() for the direct DMA, !(sync_single_for_device ||
>>> sync_single_for_cpu) or the new dma_map_ops flag, %DMA_F_CAN_SKIP_SYNC,
>>> advertised for non-NULL DMA ops.
>>> Then later, if/when swiotlb is used for the first time, the flag
>>> is reset back to on, from swiotlb_tbl_map_single().
>>>
>>> On iavf, the UDP trafficgen with XDP_DROP in skb mode test shows
>>> +3-5% increase for direct DMA.
>>>
>>> Suggested-by: Christoph Hellwig <hch@lst.de> # direct DMA shortcut
>>> Co-developed-by: Eric Dumazet <edumazet@google.com>
>>> Signed-off-by: Eric Dumazet <edumazet@google.com>
>>> Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com>
>>
>> I've bisected a boot failure (on a Firefly RK3288) to this commit.
>> AFAICT the problem is that I have (at least) two drivers which don't
>> call dma_set_mask() and therefore never initialise the new dma_need_sync
>> variable.
>>
>> The specific drivers are "rockchip-drm" and "rk_gmac-dwmac". Is it a
>> requirement that all drivers engaging in DMA should call dma_set_mask()
>> - and therefore this has uncovered a bug in those drivers. Or is the
>> assumption that all drivers call dma_set_mask() faulty?
>
> Historically it's long been documented (at least in DMA-API-HOWTO) that
> a 32-bit DMA mask is assumed by default, so as much as we would prefer
> to shift expectations, there are still going to be a great many drivers
> relying on that :(
>
> Perhaps its time for dma-debug to start warning about implicit mask
> usage, maybe that might help push the agenda a bit?
I also thought of this, but currently don't know how to detect whether a
driver has called dma_set_mask*().
The fix will arrive in several minutes.
>
> Thanks,
> Robin.
Thanks,
Olek
next prev parent reply other threads:[~2024-05-09 14:43 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-07 11:20 [PATCH v6 0/7] dma: skip calling no-op sync ops when possible Alexander Lobakin
2024-05-07 11:20 ` [PATCH v6 1/7] dma: compile-out DMA sync op calls when not used Alexander Lobakin
2024-05-07 11:20 ` [PATCH v6 2/7] dma: avoid redundant calls for sync operations Alexander Lobakin
2024-05-09 11:41 ` Marek Szyprowski
2024-05-09 11:44 ` Alexander Lobakin
2024-05-09 11:59 ` Alexander Lobakin
2024-05-09 12:02 ` Christoph Hellwig
2024-05-09 12:01 ` Christoph Hellwig
2024-05-09 13:43 ` Steven Price
2024-05-09 13:49 ` Christoph Hellwig
2024-05-09 14:33 ` Robin Murphy
2024-05-09 14:43 ` Alexander Lobakin [this message]
2024-05-07 11:20 ` [PATCH v6 3/7] iommu/dma: avoid expensive indirect " Alexander Lobakin
2024-05-07 11:20 ` [PATCH v6 4/7] page_pool: make sure frag API fields don't span between cachelines Alexander Lobakin
2024-05-07 11:20 ` [PATCH v6 5/7] page_pool: don't use driver-set flags field directly Alexander Lobakin
2024-05-07 11:20 ` [PATCH v6 6/7] page_pool: check for DMA sync shortcut earlier Alexander Lobakin
2024-05-07 11:20 ` [PATCH v6 7/7] xsk: use generic DMA sync shortcut instead of a custom one Alexander Lobakin
2024-05-07 11:30 ` [PATCH v6 0/7] dma: skip calling no-op sync ops when possible Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f207da41-5222-4d1d-897d-9b288e33a547@intel.com \
--to=aleksander.lobakin@intel.com \
--cc=bpf@vger.kernel.org \
--cc=edumazet@google.com \
--cc=hch@lst.de \
--cc=iommu@lists.linux.dev \
--cc=joro@8bytes.org \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=m.szyprowski@samsung.com \
--cc=magnus.karlsson@intel.com \
--cc=netdev@vger.kernel.org \
--cc=nex.sw.ncis.osdt.itp.upstreaming@intel.com \
--cc=rafael@kernel.org \
--cc=robin.murphy@arm.com \
--cc=steven.price@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).