From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
To: Daniel Vetter <daniel@ffwll.ch>
Cc: "Bryan O'Donoghue" <bryan.odonoghue@linaro.org>,
"Dmitry Baryshkov" <dmitry.baryshkov@linaro.org>,
"Hans de Goede" <hdegoede@redhat.com>,
"Sumit Semwal" <sumit.semwal@linaro.org>,
"Benjamin Gaignard" <benjamin.gaignard@collabora.com>,
"Brian Starkey" <Brian.Starkey@arm.com>,
"John Stultz" <jstultz@google.com>,
"T.J. Mercier" <tjmercier@google.com>,
"Christian König" <christian.koenig@amd.com>,
"Lennart Poettering" <mzxreary@0pointer.de>,
"Robert Mader" <robert.mader@collabora.com>,
"Sebastien Bacher" <sebastien.bacher@canonical.com>,
"Linux Media Mailing List" <linux-media@vger.kernel.org>,
"dri-devel@lists.freedesktop.org"
<dri-devel@lists.freedesktop.org>,
linaro-mm-sig@lists.linaro.org,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
"Milan Zamazal" <mzamazal@redhat.com>,
"Maxime Ripard" <mripard@redhat.com>,
"Andrey Konovalov" <andrey.konovalov.ynk@gmail.com>
Subject: Re: Safety of opening up /dev/dma_heap/* to physically present users (udev uaccess tag) ?
Date: Tue, 7 May 2024 21:36:13 +0300 [thread overview]
Message-ID: <20240507183613.GB20390@pendragon.ideasonboard.com> (raw)
In-Reply-To: <Zjpmu_Xj6BPdkDPa@phenom.ffwll.local>
On Tue, May 07, 2024 at 07:36:59PM +0200, Daniel Vetter wrote:
> On Tue, May 07, 2024 at 04:15:05PM +0100, Bryan O'Donoghue wrote:
> > On 07/05/2024 16:09, Dmitry Baryshkov wrote:
> > > Ah, I see. Then why do you require the DMA-ble buffer at all? If you are
> > > providing data to VPU or DRM, then you should be able to get the buffer
> > > from the data-consuming device.
> >
> > Because we don't necessarily know what the consuming device is, if any.
>
> Well ... that's an entirely different issue. And it's unsolved.
>
> Currently the approach is to allocate where the constraints are usually
> most severe (like display, if you need that, or the camera module for
> input) and then just pray the stack works out without too much copying.
> All userspace (whether the generic glue or the userspace driver depends a
> bit upon the exact api) does need to have a copy fallback for these
> sharing cases, ideally with the copying accelerated by hw.
>
> If you try to solve this by just preemptive allocating everything as cma
> buffers, then you'll make the situation substantially worse (because now
> you're wasting tons of cma memory where you might not even need it).
> And without really solving the problem, since for some gpus that memory
> might be unusable (because you cannot scan that out on any discrete gpu,
> and sometimes not even on an integrated one).
I think we have a general agreement that the proposed solution is a
stop-gap measure for an unsolved issue.
Note that libcamera is already designed that way. The API is designed to
import buffers, using dma-buf file handles. If an application has a way
to allocate dma-buf instances through another means (from the display or
from a video encoder for instance), it should do so, and use those
buffers with libcamera.
For applications that don't have an easy way to get hold of dma-buf
instances, we have a buffer allocator helper as a side component. That
allocator uses the underlying camera capture device, and allocates
buffers from the V4L2 video device. It's only on platforms where we have
no hardware camera processing (or, rather, platforms where the hardware
vendors doesn't give us access to the camera hardware, such as recent
Intel SoCs, or Qualcomm SoCs used in ARM laptops) that we need to
allocate memory elsewhere.
In the long run, I want a centralized memory allocator accessible by
userspace applications (something similar in purpose to gralloc on
Android), and I want to get rid of buffer allocation in libcamera (and
even in V4L2, in the even longer term). That's the long run.
Shorter term, we have a problem to solve, and the best option we have
found so far is to rely on dma-buf heaps as a backend for the frame
buffer allocatro helper in libcamera for the use case described above.
This won't work in 100% of the cases, clearly. It's a stop-gap measure
until we can do better.
> > Could be VPU, could be Zoom/Hangouts via pipewire, could for argument sake
> > be GPU or DSP.
> >
> > Also if we introduce a dependency on another device to allocate the output
> > buffers - say always taking the output buffer from the GPU, then we've added
> > another dependency which is more difficult to guarantee across different
> > arches.
--
Regards,
Laurent Pinchart
next prev parent reply other threads:[~2024-05-07 18:36 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-06 11:49 Safety of opening up /dev/dma_heap/* to physically present users (udev uaccess tag) ? Hans de Goede
2024-05-06 12:05 ` Maxime Ripard
2024-05-06 12:11 ` Hans de Goede
2024-05-06 13:38 ` Daniel Vetter
2024-05-06 14:01 ` Hans de Goede
2024-05-07 11:15 ` Daniel Vetter
2024-05-08 5:46 ` Daniel Stone
2024-05-08 8:33 ` Daniel Vetter
2024-05-08 8:38 ` Daniel Stone
2024-05-08 15:49 ` Daniel Vetter
2024-05-09 9:23 ` Daniel Stone
2024-05-16 10:18 ` Daniel Vetter
2024-05-13 13:51 ` Simon Ser
2024-05-16 10:13 ` Daniel Vetter
2024-05-16 11:49 ` Christian König
2024-05-07 13:34 ` Dmitry Baryshkov
2024-05-07 18:59 ` Laurent Pinchart
2024-05-22 13:34 ` Maxime Ripard
2024-05-23 9:41 ` Daniel Vetter
2024-05-07 13:32 ` Dmitry Baryshkov
2024-05-07 14:34 ` Hans de Goede
2024-05-07 15:09 ` Dmitry Baryshkov
2024-05-07 15:15 ` Bryan O'Donoghue
2024-05-07 15:19 ` Dmitry Baryshkov
2024-05-07 18:40 ` Laurent Pinchart
2024-05-07 19:59 ` Dmitry Baryshkov
2024-05-07 20:15 ` Laurent Pinchart
2024-05-08 8:39 ` Daniel Vetter
2024-05-08 21:54 ` Laurent Pinchart
2024-05-13 8:39 ` Maxime Ripard
2024-05-07 17:36 ` Daniel Vetter
2024-05-07 18:36 ` Laurent Pinchart [this message]
2024-05-07 20:07 ` Nicolas Dufresne
2024-05-08 8:36 ` Daniel Vetter
2024-05-08 21:51 ` Laurent Pinchart
2024-05-08 21:52 ` Laurent Pinchart
2024-05-13 8:29 ` Maxime Ripard
2024-05-13 8:34 ` Laurent Pinchart
2024-05-13 15:10 ` Nicolas Dufresne
2024-05-14 20:42 ` Laurent Pinchart
2024-05-15 17:43 ` nicolas.dufresne
2024-05-16 11:27 ` Laurent Pinchart
2024-05-16 17:11 ` nicolas.dufresne
2024-05-21 8:43 ` Maxime Ripard
2024-05-21 14:18 ` nicolas.dufresne
2024-05-16 7:00 ` Simon Ser
2024-05-16 11:20 ` Laurent Pinchart
2024-05-22 13:02 ` Nicolas Dufresne
2024-05-13 13:42 ` Nicolas Dufresne
2024-05-13 13:51 ` Maxime Ripard
2024-05-13 15:06 ` Nicolas Dufresne
2024-05-14 20:45 ` Laurent Pinchart
2024-05-14 20:52 ` Nicolas Dufresne
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240507183613.GB20390@pendragon.ideasonboard.com \
--to=laurent.pinchart@ideasonboard.com \
--cc=Brian.Starkey@arm.com \
--cc=andrey.konovalov.ynk@gmail.com \
--cc=benjamin.gaignard@collabora.com \
--cc=bryan.odonoghue@linaro.org \
--cc=christian.koenig@amd.com \
--cc=daniel@ffwll.ch \
--cc=dmitry.baryshkov@linaro.org \
--cc=dri-devel@lists.freedesktop.org \
--cc=hdegoede@redhat.com \
--cc=jstultz@google.com \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=mripard@redhat.com \
--cc=mzamazal@redhat.com \
--cc=mzxreary@0pointer.de \
--cc=robert.mader@collabora.com \
--cc=sebastien.bacher@canonical.com \
--cc=sumit.semwal@linaro.org \
--cc=tjmercier@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).