* [PATCH] remoteproc: add optional hook for resource checking
@ 2025-01-22 15:59 Simon Maurer
2025-01-22 18:53 ` RemoteProc over PCIe Simon Maurer
0 siblings, 1 reply; 6+ messages in thread
From: Simon Maurer @ 2025-01-22 15:59 UTC (permalink / raw)
To: linux-remoteproc
This optional hook gives the remoteproc driver the opportunity to check
the resource table after parsing and to replace the allocator in
the carveouts before resource allocation. This is needed for RemoteProc
over PCIe where the carveouts are already mapped by the driver.
Signed-off-by: Simon Maurer <mail@maurer.systems>
---
drivers/remoteproc/remoteproc_core.c | 6 ++++++
include/linux/remoteproc.h | 3 +++
2 files changed, 9 insertions(+)
diff --git a/drivers/remoteproc/remoteproc_core.c
b/drivers/remoteproc/remoteproc_core.c
index c2cf0d277729..3129c5d68770 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -1423,6 +1423,12 @@ static int rproc_fw_boot(struct rproc *rproc,
const struct firmware *fw)
goto clean_up_resources;
}
+ if (rproc->ops->check_rsc) {
+ ret = rproc->ops->check_rsc(rproc);
+ if (ret)
+ goto clean_up_resources;
+ }
+
/* Allocate carveout resources associated to rproc */
ret = rproc_alloc_registered_carveouts(rproc);
if (ret) {
diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
index b4795698d8c2..6d7d35d24dad 100644
--- a/include/linux/remoteproc.h
+++ b/include/linux/remoteproc.h
@@ -371,6 +371,8 @@ enum rsc_handling_status {
* @handle_rsc: optional platform hook to handle vendor resources.
Should return
* RSC_HANDLED if resource was handled, RSC_IGNORED if not handled
* and a negative value on error
+ * @hcheck_rsc: optional platform hook to check resources before
allocation.
+ * Should return 0 if resources are OK and a negative value on error
* @find_loaded_rsc_table: find the loaded resource table from
firmware image
* @get_loaded_rsc_table: get resource table installed in memory
* by external entity
@@ -394,6 +396,7 @@ struct rproc_ops {
int (*parse_fw)(struct rproc *rproc, const struct firmware *fw);
int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, void *rsc,
int offset, int avail);
+ int (*check_rsc)(struct rproc *rproc);
struct resource_table *(*find_loaded_rsc_table)(
struct rproc *rproc, const struct firmware *fw);
struct resource_table *(*get_loaded_rsc_table)(
--
2.46.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* RemoteProc over PCIe
2025-01-22 15:59 [PATCH] remoteproc: add optional hook for resource checking Simon Maurer
@ 2025-01-22 18:53 ` Simon Maurer
2025-01-27 16:27 ` Mathieu Poirier
0 siblings, 1 reply; 6+ messages in thread
From: Simon Maurer @ 2025-01-22 18:53 UTC (permalink / raw)
To: linux-remoteproc
The last patch would be the first step for RemoteProc over PCIe. I use a
Xilinx Zynq 7000 PCIe card as my remote CPU and a x86 PC as the the
host. Both vrings and RPMSG-buffer are in the remote (PCIe-Card) SRAM.
So the RPMSG-buffer isn't actually DMA memory, but the SRAM on the Zynq
is mapped in a PCIe-bar. I'm working now on a patch, that transfers the
ownership of the RPMSG-buffer form virtio_rpmsg_bus to
remoteproc_virtio. So instead of virtio_rpmsg_bus calling
dma_alloc_coherent for buffer allocation, it would call
virtio_get_shm_region to get TX/RX buffers and on rpmsg_remove it would
call virtio_release_shm_regions (new in virtio_config_ops), analogue to
find_vqs/del_vqs.
Thoughts?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: RemoteProc over PCIe
2025-01-22 18:53 ` RemoteProc over PCIe Simon Maurer
@ 2025-01-27 16:27 ` Mathieu Poirier
0 siblings, 0 replies; 6+ messages in thread
From: Mathieu Poirier @ 2025-01-27 16:27 UTC (permalink / raw)
To: Simon Maurer; +Cc: linux-remoteproc
On Wed, 22 Jan 2025 at 11:53, Simon Maurer <mail@maurer.systems> wrote:
>
> The last patch would be the first step for RemoteProc over PCIe. I use a
> Xilinx Zynq 7000 PCIe card as my remote CPU and a x86 PC as the the
> host. Both vrings and RPMSG-buffer are in the remote (PCIe-Card) SRAM.
> So the RPMSG-buffer isn't actually DMA memory, but the SRAM on the Zynq
> is mapped in a PCIe-bar. I'm working now on a patch, that transfers the
> ownership of the RPMSG-buffer form virtio_rpmsg_bus to
> remoteproc_virtio. So instead of virtio_rpmsg_bus calling
> dma_alloc_coherent for buffer allocation, it would call
> virtio_get_shm_region to get TX/RX buffers and on rpmsg_remove it would
> call virtio_release_shm_regions (new in virtio_config_ops), analogue to
> find_vqs/del_vqs.
> Thoughts?
>
Hi Simon,
I don't know anything about the Zynq 7000 but I assume it needs to be
explicitly told to act as an endpoint rather than a PCI bus master.
I can't say much about the above without looking at real code so I
will wait for your patches before spending more time on the design you
are putting forward.
As for your patch adding a new operation to the rproc_ops structure,
please include in the submission for the work you presented above. It
can't be merged without an actual customer.
Thanks,
Mathieu
^ permalink raw reply [flat|nested] 6+ messages in thread
* remoteproc over PCIe
@ 2023-05-09 13:42 Simon Maurer
2023-05-09 17:08 ` Arnaud POULIQUEN
0 siblings, 1 reply; 6+ messages in thread
From: Simon Maurer @ 2023-05-09 13:42 UTC (permalink / raw)
To: linux-remoteproc
Hi,
I've got a "Zynq PCIe FMC Carrier Evaluation Board" with a x86 host. The
Zynq 7000 is an FPGA with 2x Cortex-A9, on which ZephyrOS with the
OpenAMP framework is running. The VirtIO ring and the buffer for the
RPMSGs are located in the nocache memory section of the ZephyrOS. The
card DDR-RAM and the CPU control registers are mapped into PCIe BARs. On
the FPGA the "AXI Memory Mapped To PCI Express" IP core is used, so the
kernel has mmio access to the card DDR-RAM.
Besides the kernel module, I had to do a few modifications elsewhere in
the kernel. In remoteproc_virtio.c I implemented the get_shm_region
function for the rproc_virtio_config_ops. This gives access to the rpmsg
buffer, that is already mapped. In virtio_rpmsg_bus.c this function is
used instead of allocating a new region. This is just a proof of
concept, but it seems to be working, ttyRPMSG is created and I can send
and receive messages.
But what would be the clean way to do this? I'm thinking about
implementing dma_map_ops for the vdev, but maybe there is a better solution?
Best regards,
Simon
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: remoteproc over PCIe
2023-05-09 13:42 remoteproc " Simon Maurer
@ 2023-05-09 17:08 ` Arnaud POULIQUEN
2023-05-10 13:06 ` Simon Maurer
0 siblings, 1 reply; 6+ messages in thread
From: Arnaud POULIQUEN @ 2023-05-09 17:08 UTC (permalink / raw)
To: Simon Maurer, linux-remoteproc, Christoph Hellwig
Hi Simon,
On 5/9/23 15:42, Simon Maurer wrote:
> Hi,
> I've got a "Zynq PCIe FMC Carrier Evaluation Board" with a x86 host. The Zynq
> 7000 is an FPGA with 2x Cortex-A9, on which ZephyrOS with the OpenAMP framework
> is running. The VirtIO ring and the buffer for the RPMSGs are located in the
> nocache memory section of the ZephyrOS. The card DDR-RAM and the CPU control
> registers are mapped into PCIe BARs. On the FPGA the "AXI Memory Mapped To PCI
> Express" IP core is used, so the kernel has mmio access to the card DDR-RAM.
> Besides the kernel module, I had to do a few modifications elsewhere in the
> kernel. In remoteproc_virtio.c I implemented the get_shm_region function for the
> rproc_virtio_config_ops. This gives access to the rpmsg buffer, that is already
> mapped.
+ christopher Hellwig.
Nice proposal!
I've also had the same approach in mind for a while, to remove the use of
dma_declare_coherent_memory in remoteproc_virtio[1][2].
But I haven't found the time to implement it yet.
[1] https://lkml.org/lkml/2021/6/23/607
[2]
https://patchwork.kernel.org/project/linux-remoteproc/patch/AOKowLclCbOCKxyiJ71WeNyuAAj2q8EUtxrXbyky5E@cp7-web-042.plabs.ch/
It would be nice to ensure same approach for the different virtio
backend (remoteproc_virtio, virtio_mmio, ...) to be able to reuse the virtio
drivers on top of.
The virtio_mmio seems the reference...
> In virtio_rpmsg_bus.c this function is used instead of allocating a new
> region.
The DT seems to me a valid place to define memory regions associated.
It can be one for vrings and buffers, or one unique pool.
One reason is that the main processor can have to specify the memory mapping
and to specify associated access right.
> This is just a proof of concept, but it seems to be working, ttyRPMSG is
> created and I can send and receive messages.
Great!
> But what would be the clean way to do this? I'm thinking about implementing
> dma_map_ops for the vdev, but maybe there is a better solution?
Good question...I don't have the answer.
I have implemented something different but not fully satisfied. For instance how
to manage the remoteproc address (DA) and CPU address(PA) conversion for vring
descriptor?
You can have a look to the work I started, described in the cover letter[3]] with
links to my github for the code source.
I have only up-streamed the step 1 yet.
We have some discussions in the OpenAMP project to reuse the virtio MMIO. So I
put on hold waiting more clues.
[3]https://www.spinics.net/lists/linux-remoteproc/msg13680.html
Regards,
Arnaud
>
> Best regards,
> Simon
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: remoteproc over PCIe
2023-05-09 17:08 ` Arnaud POULIQUEN
@ 2023-05-10 13:06 ` Simon Maurer
0 siblings, 0 replies; 6+ messages in thread
From: Simon Maurer @ 2023-05-10 13:06 UTC (permalink / raw)
To: linux-remoteproc
Hi Arnaud,
Thank you for your response.
On 5/9/23 19:08, Arnaud POULIQUEN wrote:
> Hi Simon,
>
> On 5/9/23 15:42, Simon Maurer wrote:
>> Hi,
>> I've got a "Zynq PCIe FMC Carrier Evaluation Board" with a x86 host. The Zynq
>> 7000 is an FPGA with 2x Cortex-A9, on which ZephyrOS with the OpenAMP framework
>> is running. The VirtIO ring and the buffer for the RPMSGs are located in the
>> nocache memory section of the ZephyrOS. The card DDR-RAM and the CPU control
>> registers are mapped into PCIe BARs. On the FPGA the "AXI Memory Mapped To PCI
>> Express" IP core is used, so the kernel has mmio access to the card DDR-RAM.
>> Besides the kernel module, I had to do a few modifications elsewhere in the
>> kernel. In remoteproc_virtio.c I implemented the get_shm_region function for the
>> rproc_virtio_config_ops. This gives access to the rpmsg buffer, that is already
>> mapped.
>
> + christopher Hellwig.
>
> Nice proposal!
>
> I've also had the same approach in mind for a while, to remove the use of
> dma_declare_coherent_memory in remoteproc_virtio[1][2].
> But I haven't found the time to implement it yet.
>
> [1] https://lkml.org/lkml/2021/6/23/607
> [2]
> https://patchwork.kernel.org/project/linux-remoteproc/patch/AOKowLclCbOCKxyiJ71WeNyuAAj2q8EUtxrXbyky5E@cp7-web-042.plabs.ch/
>
> It would be nice to ensure same approach for the different virtio
> backend (remoteproc_virtio, virtio_mmio, ...) to be able to reuse the virtio
> drivers on top of.
>
> The virtio_mmio seems the reference...
>
>> In virtio_rpmsg_bus.c this function is used instead of allocating a new
>> region.
>
> The DT seems to me a valid place to define memory regions associated.
> It can be one for vrings and buffers, or one unique pool.
> One reason is that the main processor can have to specify the memory mapping
> and to specify associated access right.
>
I created a section in the linker script for ZephyrOS that is reserved
for the host. The kernel module parses the ELF file and creates the
vrings and the buffers in it. There is no device tree on my x86 host,
there is just BIOS and ACPI magic.
>> This is just a proof of concept, but it seems to be working, ttyRPMSG is
>> created and I can send and receive messages.
>
> Great!
>
>> But what would be the clean way to do this? I'm thinking about implementing
>> dma_map_ops for the vdev, but maybe there is a better solution?
>
> Good question...I don't have the answer.
> I have implemented something different but not fully satisfied. For instance how
> to manage the remoteproc address (DA) and CPU address(PA) conversion for vring
> descriptor?
>
I forgot to mention this, the PA address (host) of the shared memory is
written to a variable in the shared memory, so ZephyrOS/OpenAMP can
translate back the buffer addresses to DA. But this is a hack I would
really like to get rid of. This would be the reason for the dma_map_ops
implementation, so the driver on the host has full control what
addresses the vring writes into the descriptors.
> You can have a look to the work I started, described in the cover letter[3]] with
> links to my github for the code source.
> I have only up-streamed the step 1 yet.
> We have some discussions in the OpenAMP project to reuse the virtio MMIO. So I
> put on hold waiting more clues.
>
> [3]https://www.spinics.net/lists/linux-remoteproc/msg13680.html
>
>
Thanks, I will take a look at it.
Best Regards
Simon
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-01-27 16:27 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-01-22 15:59 [PATCH] remoteproc: add optional hook for resource checking Simon Maurer
2025-01-22 18:53 ` RemoteProc over PCIe Simon Maurer
2025-01-27 16:27 ` Mathieu Poirier
-- strict thread matches above, loose matches on Subject: below --
2023-05-09 13:42 remoteproc " Simon Maurer
2023-05-09 17:08 ` Arnaud POULIQUEN
2023-05-10 13:06 ` Simon Maurer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).