From: Simon Maurer <mail@maurer.systems>
To: linux-remoteproc@vger.kernel.org
Subject: Re: remoteproc over PCIe
Date: Wed, 10 May 2023 15:06:23 +0200 [thread overview]
Message-ID: <8a49666b-4509-1210-f8d5-fffd0a4cf855@maurer.systems> (raw)
In-Reply-To: <86dc39b0-61bf-5e83-e65d-1283fdf382b4@foss.st.com>
Hi Arnaud,
Thank you for your response.
On 5/9/23 19:08, Arnaud POULIQUEN wrote:
> Hi Simon,
>
> On 5/9/23 15:42, Simon Maurer wrote:
>> Hi,
>> I've got a "Zynq PCIe FMC Carrier Evaluation Board" with a x86 host. The Zynq
>> 7000 is an FPGA with 2x Cortex-A9, on which ZephyrOS with the OpenAMP framework
>> is running. The VirtIO ring and the buffer for the RPMSGs are located in the
>> nocache memory section of the ZephyrOS. The card DDR-RAM and the CPU control
>> registers are mapped into PCIe BARs. On the FPGA the "AXI Memory Mapped To PCI
>> Express" IP core is used, so the kernel has mmio access to the card DDR-RAM.
>> Besides the kernel module, I had to do a few modifications elsewhere in the
>> kernel. In remoteproc_virtio.c I implemented the get_shm_region function for the
>> rproc_virtio_config_ops. This gives access to the rpmsg buffer, that is already
>> mapped.
>
> + christopher Hellwig.
>
> Nice proposal!
>
> I've also had the same approach in mind for a while, to remove the use of
> dma_declare_coherent_memory in remoteproc_virtio[1][2].
> But I haven't found the time to implement it yet.
>
> [1] https://lkml.org/lkml/2021/6/23/607
> [2]
> https://patchwork.kernel.org/project/linux-remoteproc/patch/AOKowLclCbOCKxyiJ71WeNyuAAj2q8EUtxrXbyky5E@cp7-web-042.plabs.ch/
>
> It would be nice to ensure same approach for the different virtio
> backend (remoteproc_virtio, virtio_mmio, ...) to be able to reuse the virtio
> drivers on top of.
>
> The virtio_mmio seems the reference...
>
>> In virtio_rpmsg_bus.c this function is used instead of allocating a new
>> region.
>
> The DT seems to me a valid place to define memory regions associated.
> It can be one for vrings and buffers, or one unique pool.
> One reason is that the main processor can have to specify the memory mapping
> and to specify associated access right.
>
I created a section in the linker script for ZephyrOS that is reserved
for the host. The kernel module parses the ELF file and creates the
vrings and the buffers in it. There is no device tree on my x86 host,
there is just BIOS and ACPI magic.
>> This is just a proof of concept, but it seems to be working, ttyRPMSG is
>> created and I can send and receive messages.
>
> Great!
>
>> But what would be the clean way to do this? I'm thinking about implementing
>> dma_map_ops for the vdev, but maybe there is a better solution?
>
> Good question...I don't have the answer.
> I have implemented something different but not fully satisfied. For instance how
> to manage the remoteproc address (DA) and CPU address(PA) conversion for vring
> descriptor?
>
I forgot to mention this, the PA address (host) of the shared memory is
written to a variable in the shared memory, so ZephyrOS/OpenAMP can
translate back the buffer addresses to DA. But this is a hack I would
really like to get rid of. This would be the reason for the dma_map_ops
implementation, so the driver on the host has full control what
addresses the vring writes into the descriptors.
> You can have a look to the work I started, described in the cover letter[3]] with
> links to my github for the code source.
> I have only up-streamed the step 1 yet.
> We have some discussions in the OpenAMP project to reuse the virtio MMIO. So I
> put on hold waiting more clues.
>
> [3]https://www.spinics.net/lists/linux-remoteproc/msg13680.html
>
>
Thanks, I will take a look at it.
Best Regards
Simon
next prev parent reply other threads:[~2023-05-10 13:06 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-09 13:42 remoteproc over PCIe Simon Maurer
2023-05-09 17:08 ` Arnaud POULIQUEN
2023-05-10 13:06 ` Simon Maurer [this message]
-- strict thread matches above, loose matches on Subject: below --
2025-01-22 15:59 [PATCH] remoteproc: add optional hook for resource checking Simon Maurer
2025-01-22 18:53 ` RemoteProc over PCIe Simon Maurer
2025-01-27 16:27 ` Mathieu Poirier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8a49666b-4509-1210-f8d5-fffd0a4cf855@maurer.systems \
--to=mail@maurer.systems \
--cc=linux-remoteproc@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).