* [PATCH 0/1] remoteproc documentation changes
@ 2024-10-23 5:33 anish kumar
2024-10-23 5:33 ` [PATCH V4 1/7] Remoteproc: Documentation: add the main index anish kumar
` (7 more replies)
0 siblings, 8 replies; 18+ messages in thread
From: anish kumar @ 2024-10-23 5:33 UTC (permalink / raw)
To: ohad, bjorn.andersson, mathieu.poirier, corbet
Cc: linux-remoteproc, linux-doc, linux-kernel, kernel test robot
This patch series transitions the documentation
for remoteproc from the staging directory to the
mainline kernel. It introduces both kernel and
user-space APIs, enhancing the overall documentation
quality.
V4:
Fixed compilation errors and moved documentation to
driver-api directory.
V3:
Seperated out the patches further to make the intention
clear for each patch.
V2:
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202410161444.jOKMsoGS-lkp@intel.com/
^ permalink raw reply [flat|nested] 18+ messages in thread* [PATCH V4 1/7] Remoteproc: Documentation: add the main index 2024-10-23 5:33 [PATCH 0/1] remoteproc documentation changes anish kumar @ 2024-10-23 5:33 ` anish kumar 2024-10-24 15:42 ` Mathieu Poirier 2024-10-23 5:33 ` [PATCH V4 2/7] Documentation: remoteproc: move from staging anish kumar ` (6 subsequent siblings) 7 siblings, 1 reply; 18+ messages in thread From: anish kumar @ 2024-10-23 5:33 UTC (permalink / raw) To: ohad, bjorn.andersson, mathieu.poirier, corbet Cc: linux-remoteproc, linux-doc, linux-kernel, anish kumar In prepration of moving the remoteproc documentation from staging to mainline and also for enhancing the documentation. Signed-off-by: anish kumar <yesanishhere@gmail.com> --- Documentation/driver-api/index.rst | 1 + Documentation/driver-api/remoteproc/index.rst | 20 +++++++++++++++++++ 2 files changed, 21 insertions(+) create mode 100644 Documentation/driver-api/remoteproc/index.rst diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst index c57c609ad2eb..ec9d669b92e5 100644 --- a/Documentation/driver-api/index.rst +++ b/Documentation/driver-api/index.rst @@ -30,6 +30,7 @@ available subsections can be seen below. frame-buffer regulator reset + remoteproc/index iio/index input usb/index diff --git a/Documentation/driver-api/remoteproc/index.rst b/Documentation/driver-api/remoteproc/index.rst new file mode 100644 index 000000000000..8b8c8f512e2b --- /dev/null +++ b/Documentation/driver-api/remoteproc/index.rst @@ -0,0 +1,20 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================================================================== +remoteproc - remote processor subsystem in Linux(TM) kernel +======================================================================== + +Authors: + - anish kumar <yesanishhere@gmail.com> + + remote processor subsystem is a way to manage the lifecycle of + a subsytem that is external to the Linux. The remoteproc framework + allows different platforms/architectures to control (power on, + load firmware, power off) those remote processors while abstracting + the hardware differences, so the entire driver doesn't need to be + duplicated. + +Mailing List +------------ +To post a message, send an email to +linux-remoteproc@vger.kernel.org -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH V4 1/7] Remoteproc: Documentation: add the main index 2024-10-23 5:33 ` [PATCH V4 1/7] Remoteproc: Documentation: add the main index anish kumar @ 2024-10-24 15:42 ` Mathieu Poirier 2024-10-24 17:51 ` anish kumar 0 siblings, 1 reply; 18+ messages in thread From: Mathieu Poirier @ 2024-10-24 15:42 UTC (permalink / raw) To: anish kumar Cc: ohad, bjorn.andersson, corbet, linux-remoteproc, linux-doc, linux-kernel On Tue, Oct 22, 2024 at 10:33:51PM -0700, anish kumar wrote: > In prepration of moving the remoteproc documentation > from staging to mainline and also for enhancing > the documentation. > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > --- > Documentation/driver-api/index.rst | 1 + > Documentation/driver-api/remoteproc/index.rst | 20 +++++++++++++++++++ > 2 files changed, 21 insertions(+) > create mode 100644 Documentation/driver-api/remoteproc/index.rst > > diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst > index c57c609ad2eb..ec9d669b92e5 100644 > --- a/Documentation/driver-api/index.rst > +++ b/Documentation/driver-api/index.rst > @@ -30,6 +30,7 @@ available subsections can be seen below. > frame-buffer > regulator > reset > + remoteproc/index This patch doesn't apply to rproc-next and I'm pretty sure it is the same for Jonathan's tree. > iio/index > input > usb/index > diff --git a/Documentation/driver-api/remoteproc/index.rst b/Documentation/driver-api/remoteproc/index.rst > new file mode 100644 > index 000000000000..8b8c8f512e2b > --- /dev/null > +++ b/Documentation/driver-api/remoteproc/index.rst > @@ -0,0 +1,20 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > +======================================================================== > +remoteproc - remote processor subsystem in Linux(TM) kernel > +======================================================================== > + > +Authors: > + - anish kumar <yesanishhere@gmail.com> > + > + remote processor subsystem is a way to manage the lifecycle of > + a subsytem that is external to the Linux. The remoteproc framework ... external to the Linux kernel. > + allows different platforms/architectures to control (power on, > + load firmware, power off) those remote processors while abstracting > + the hardware differences, so the entire driver doesn't need to be > + duplicated. > + > +Mailing List > +------------ > +To post a message, send an email to > +linux-remoteproc@vger.kernel.org > -- > 2.39.3 (Apple Git-146) > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V4 1/7] Remoteproc: Documentation: add the main index 2024-10-24 15:42 ` Mathieu Poirier @ 2024-10-24 17:51 ` anish kumar 0 siblings, 0 replies; 18+ messages in thread From: anish kumar @ 2024-10-24 17:51 UTC (permalink / raw) To: Mathieu Poirier Cc: ohad, bjorn.andersson, corbet, linux-remoteproc, linux-doc, linux-kernel On Thu, Oct 24, 2024 at 8:42 AM Mathieu Poirier <mathieu.poirier@linaro.org> wrote: > > On Tue, Oct 22, 2024 at 10:33:51PM -0700, anish kumar wrote: > > In prepration of moving the remoteproc documentation > > from staging to mainline and also for enhancing > > the documentation. > > > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > > --- > > Documentation/driver-api/index.rst | 1 + > > Documentation/driver-api/remoteproc/index.rst | 20 +++++++++++++++++++ > > 2 files changed, 21 insertions(+) > > create mode 100644 Documentation/driver-api/remoteproc/index.rst > > > > diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst > > index c57c609ad2eb..ec9d669b92e5 100644 > > --- a/Documentation/driver-api/index.rst > > +++ b/Documentation/driver-api/index.rst > > @@ -30,6 +30,7 @@ available subsections can be seen below. > > frame-buffer > > regulator > > reset > > + remoteproc/index > > This patch doesn't apply to rproc-next and I'm pretty sure it is the same for > Jonathan's tree. Sorry, I cloned rproc instead of rproc-next. Will resend the patch series after rebasing on that. > > > iio/index > > input > > usb/index > > diff --git a/Documentation/driver-api/remoteproc/index.rst b/Documentation/driver-api/remoteproc/index.rst > > new file mode 100644 > > index 000000000000..8b8c8f512e2b > > --- /dev/null > > +++ b/Documentation/driver-api/remoteproc/index.rst > > @@ -0,0 +1,20 @@ > > +.. SPDX-License-Identifier: GPL-2.0 > > + > > +======================================================================== > > +remoteproc - remote processor subsystem in Linux(TM) kernel > > +======================================================================== > > + > > +Authors: > > + - anish kumar <yesanishhere@gmail.com> > > + > > + remote processor subsystem is a way to manage the lifecycle of > > + a subsytem that is external to the Linux. The remoteproc framework > > ... external to the Linux kernel. > > > + allows different platforms/architectures to control (power on, > > + load firmware, power off) those remote processors while abstracting > > + the hardware differences, so the entire driver doesn't need to be > > + duplicated. > > + > > +Mailing List > > +------------ > > +To post a message, send an email to > > +linux-remoteproc@vger.kernel.org > > -- > > 2.39.3 (Apple Git-146) > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH V4 2/7] Documentation: remoteproc: move from staging 2024-10-23 5:33 [PATCH 0/1] remoteproc documentation changes anish kumar 2024-10-23 5:33 ` [PATCH V4 1/7] Remoteproc: Documentation: add the main index anish kumar @ 2024-10-23 5:33 ` anish kumar 2024-10-23 5:33 ` [PATCH V4 3/7] Documentation: remoteproc: add more information anish kumar ` (5 subsequent siblings) 7 siblings, 0 replies; 18+ messages in thread From: anish kumar @ 2024-10-23 5:33 UTC (permalink / raw) To: ohad, bjorn.andersson, mathieu.poirier, corbet Cc: linux-remoteproc, linux-doc, linux-kernel, anish kumar In preparation for upcoming enhancements, this commit relocates the existing documentation from the staging area to the mainline Documentation directory. Future patches will add further documentation to improve clarity and usability. Signed-off-by: anish kumar <yesanishhere@gmail.com> --- Documentation/driver-api/remoteproc/index.rst | 6 ++++++ .../{staging => driver-api/remoteproc}/remoteproc.rst | 0 Documentation/staging/index.rst | 1 - 3 files changed, 6 insertions(+), 1 deletion(-) rename Documentation/{staging => driver-api/remoteproc}/remoteproc.rst (100%) diff --git a/Documentation/driver-api/remoteproc/index.rst b/Documentation/driver-api/remoteproc/index.rst index 8b8c8f512e2b..6ac5a082bf4f 100644 --- a/Documentation/driver-api/remoteproc/index.rst +++ b/Documentation/driver-api/remoteproc/index.rst @@ -14,6 +14,12 @@ Authors: the hardware differences, so the entire driver doesn't need to be duplicated. + +.. toctree:: + :maxdepth: 1 + + remoteproc + Mailing List ------------ To post a message, send an email to diff --git a/Documentation/staging/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst similarity index 100% rename from Documentation/staging/remoteproc.rst rename to Documentation/driver-api/remoteproc/remoteproc.rst diff --git a/Documentation/staging/index.rst b/Documentation/staging/index.rst index abd0d18254d2..8884af4556b4 100644 --- a/Documentation/staging/index.rst +++ b/Documentation/staging/index.rst @@ -8,7 +8,6 @@ Unsorted Documentation crc32 lzo - remoteproc rpmsg speculation static-keys -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH V4 3/7] Documentation: remoteproc: add more information 2024-10-23 5:33 [PATCH 0/1] remoteproc documentation changes anish kumar 2024-10-23 5:33 ` [PATCH V4 1/7] Remoteproc: Documentation: add the main index anish kumar 2024-10-23 5:33 ` [PATCH V4 2/7] Documentation: remoteproc: move from staging anish kumar @ 2024-10-23 5:33 ` anish kumar 2024-10-24 15:53 ` Mathieu Poirier 2024-10-23 5:33 ` [PATCH V4 4/7] Remoteproc: Documentation: add driver api anish kumar ` (4 subsequent siblings) 7 siblings, 1 reply; 18+ messages in thread From: anish kumar @ 2024-10-23 5:33 UTC (permalink / raw) To: ohad, bjorn.andersson, mathieu.poirier, corbet Cc: linux-remoteproc, linux-doc, linux-kernel, anish kumar Added following changes: 1. Components provided by remoteproc framework. 2. Remoteproc driver responsibilities. 3. Remoteproc framework responsibilities. 4. Better explanation of how to ask for resources from the framework by the remote processor. Signed-off-by: anish kumar <yesanishhere@gmail.com> --- .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- 1 file changed, 243 insertions(+), 353 deletions(-) diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst index 9cccd3dd6a4b..7ca545eea153 100644 --- a/Documentation/driver-api/remoteproc/remoteproc.rst +++ b/Documentation/driver-api/remoteproc/remoteproc.rst @@ -1,359 +1,249 @@ +.. SPDX-License-Identifier: GPL-2.0 + ========================== Remote Processor Framework ========================== -Introduction -============ - -Modern SoCs typically have heterogeneous remote processor devices in asymmetric -multiprocessing (AMP) configurations, which may be running different instances -of operating system, whether it's Linux or any other flavor of real-time OS. - -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. -In a typical configuration, the dual cortex-A9 is running Linux in a SMP -configuration, and each of the other three cores (two M3 cores and a DSP) -is running its own instance of RTOS in an AMP configuration. - -The remoteproc framework allows different platforms/architectures to -control (power on, load firmware, power off) those remote processors while -abstracting the hardware differences, so the entire driver doesn't need to be -duplicated. In addition, this framework also adds rpmsg virtio devices -for remote processors that supports this kind of communication. This way, -platform-specific remoteproc drivers only need to provide a few low-level -handlers, and then all rpmsg drivers will then just work -(for more information about the virtio-based rpmsg bus and its drivers, -please read Documentation/staging/rpmsg.rst). -Registration of other types of virtio devices is now also possible. Firmwares -just need to publish what kind of virtio devices do they support, and then -remoteproc will add those devices. This makes it possible to reuse the -existing virtio drivers with remote processor backends at a minimal development -cost. - -User API -======== - -:: - - int rproc_boot(struct rproc *rproc) - -Boot a remote processor (i.e. load its firmware, power it on, ...). - -If the remote processor is already powered on, this function immediately -returns (successfully). - -Returns 0 on success, and an appropriate error value otherwise. -Note: to use this function you should already have a valid rproc -handle. There are several ways to achieve that cleanly (devres, pdata, -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we -might also consider using dev_archdata for this). - -:: - - void rproc_shutdown(struct rproc *rproc) - -Power off a remote processor (previously booted with rproc_boot()). -In case @rproc is still being used by an additional user(s), then -this function will just decrement the power refcount and exit, -without really powering off the device. - -Every call to rproc_boot() must (eventually) be accompanied by a call -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. - -.. note:: - - we're not decrementing the rproc's refcount, only the power refcount. - which means that the @rproc handle stays valid even after - rproc_shutdown() returns, and users can still use it with a subsequent - rproc_boot(), if needed. - -:: - - struct rproc *rproc_get_by_phandle(phandle phandle) - -Find an rproc handle using a device tree phandle. Returns the rproc -handle on success, and NULL on failure. This function increments -the remote processor's refcount, so always use rproc_put() to -decrement it back once rproc isn't needed anymore. - -Typical usage -============= - -:: - - #include <linux/remoteproc.h> - - /* in case we were given a valid 'rproc' handle */ - int dummy_rproc_example(struct rproc *my_rproc) - { - int ret; - - /* let's power on and boot our remote processor */ - ret = rproc_boot(my_rproc); - if (ret) { - /* - * something went wrong. handle it and leave. - */ - } - - /* - * our remote processor is now powered on... give it some work - */ - - /* let's shut it down now */ - rproc_shutdown(my_rproc); - } - -API for implementors -==================== - -:: - - struct rproc *rproc_alloc(struct device *dev, const char *name, - const struct rproc_ops *ops, - const char *firmware, int len) - -Allocate a new remote processor handle, but don't register -it yet. Required parameters are the underlying device, the -name of this remote processor, platform-specific ops handlers, -the name of the firmware to boot this rproc with, and the -length of private data needed by the allocating rproc driver (in bytes). - -This function should be used by rproc implementations during -initialization of the remote processor. - -After creating an rproc handle using this function, and when ready, -implementations should then call rproc_add() to complete -the registration of the remote processor. - -On success, the new rproc is returned, and on failure, NULL. - -.. note:: - - **never** directly deallocate @rproc, even if it was not registered - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). - -:: - - void rproc_free(struct rproc *rproc) - -Free an rproc handle that was allocated by rproc_alloc. - -This function essentially unrolls rproc_alloc(), by decrementing the -rproc's refcount. It doesn't directly free rproc; that would happen -only if there are no other references to rproc and its refcount now -dropped to zero. - -:: - - int rproc_add(struct rproc *rproc) - -Register @rproc with the remoteproc framework, after it has been -allocated with rproc_alloc(). - -This is called by the platform-specific rproc implementation, whenever -a new remote processor device is probed. - -Returns 0 on success and an appropriate error code otherwise. -Note: this function initiates an asynchronous firmware loading -context, which will look for virtio devices supported by the rproc's -firmware. - -If found, those virtio devices will be created and added, so as a result -of registering this remote processor, additional virtio drivers might get -probed. - -:: - - int rproc_del(struct rproc *rproc) - -Unroll rproc_add(). - -This function should be called when the platform specific rproc -implementation decides to remove the rproc device. it should -_only_ be called if a previous invocation of rproc_add() -has completed successfully. - -After rproc_del() returns, @rproc is still valid, and its -last refcount should be decremented by calling rproc_free(). - -Returns 0 on success and -EINVAL if @rproc isn't valid. - -:: - - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) - -Report a crash in a remoteproc - -This function must be called every time a crash is detected by the -platform specific rproc implementation. This should not be called from a -non-remoteproc driver. This function can be called from atomic/interrupt -context. - -Implementation callbacks +.. Contents: + + 1. Introduction + 2. Remoteproc framework responsibilities + 3. Remoteproc driver responsibilities + 4. Virtio and rpmsg + +1. Introduction +=============== + +Modern System on Chips (SoCs) typically integrate heterogeneous remote +processor devices in asymmetric multiprocessing (AMP) configurations. +These processors may run different operating systems, such as Linux and +various real-time operating systems (RTOS). + +For example, the OMAP4 platform features dual Cortex-A9 cores, dual +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 +cores execute Linux in a symmetric multiprocessing (SMP) configuration, +while the M3 cores and DSP run independent instances of an RTOS. + +The remoteproc framework allows various platforms and architectures to +manage remote processors, including operations such as powering on, +loading firmware, and powering off. This framework abstracts hardware +differences, promoting code reuse and minimizing duplication. It also +supports rpmsg virtio devices for remote processors that utilize this +communication method. Consequently, platform-specific remoteproc drivers +need only implement a few low-level handlers, enabling seamless operation +of all rpmsg drivers. (For more details about the virtio-based rpmsg +bus and its drivers, refer to rpmsg documentation.) + +Additionally, the framework allows for the registration of various +virtio devices. Firmware can publish the types of virtio devices it +supports, facilitating their addition to the remoteproc framework. This +flexibility enables the reuse of existing virtio drivers with remote +processor backends at minimal development cost. + +The primary purpose of the remoteproc framework is to download firmware +for remote processors and manage their lifecycle. The framework consists +of several key components: + +- **Character Driver**: Provides userspace access to control the remote + processor. +- **ELF Utility**: Offers functions for handling ELF files and managing + resources requested by the remote processor. +- **Remoteproc Core**: Manages firmware downloads and recovery actions + in case of a remote processor crash. +- **Coredump**: Provides facilities for coredumping and tracing from + the remote processor in the event of a crash. +- **Userspace Interaction**: Uses sysfs and debugfs to manage the + lifecycle and status of the remote processor. +- **Virtio Support**: Facilitates interaction with the virtio and + rpmsg bus. + +From here on, references to "framework" denote the remoteproc +framework, and "driver" refers to the remoteproc driver that utilizes +the framework for managing remote processors. + +2. Remoteproc framework Responsibilities +======================================== + +The framework begins by gathering information about the firmware file +to be downloaded through the request_firmware function. It supports +the ELF format and parses the firmware image to identify the physical +addresses that need to be populated from the corresponding ELF sections. +The framework also requires knowledge of the logical or I/O-mapped +addresses in the application processor. Once this information is +obtained from the driver, the framework transfers the data to the +specified addresses and starts the remote, along with +any devices physically or logically connected to it. + +Dependent devices, referred to as `subdevices` within the framework, +are also managed post-registration by their respective drivers. +Subdevices can register themselves using `rproc_(add/remove)_subdev`. +Non-remoteproc drivers can use subdevices as a way to logically connect +to remote and get lifecycle notifications of the remote. + +The framework oversees the lifecycle of the remote and +provides the `rproc_report_crash` function, which the driver invokes +upon receiving a crash notification from the remote. The +notification method can differ based on the design of the remote +processor and its communication with the application processor. For +instance, if the remote is a DSP equipped with a watchdog, +unresponsive behavior triggers the watchdog, generating an interrupt +that routes to the application processor, allowing it to call +`rproc_report_crash` in the driver's interrupt context. + +During crash handling, the framework performs the following actions: + +a. Sends a request to stop the remote and any connected or + dependent subdevices. +b. Generates a coredump, dumping all `resources` requested by the + remote alongside relevant debugging information. Resources are + explained below. +c. Reloads the firmware and restarts the remote. + +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and +attach callbacks of the driver are invoked without reloading the +firmware. This is useful when the remote requires no +assistance for recovery, or when the application processor can restart +independently. After recovery, the application processor can reattach +to the remote. + +The remote can request resources from the framework, which +allocates a ".resource_table" section. During the ELF parsing phase, +the framework identifies this section and calls the appropriate +handler to allocate the requested resources. + +Resource management within the framework can accommodate any type of +`fw_resource_type`. + +.. code-block:: c + + enum fw_resource_type { + RSC_CARVEOUT = 0, + RSC_DEVMEM = 1, + RSC_TRACE = 2, + RSC_VDEV = 3, + RSC_LAST = 4, + RSC_VENDOR_START = 128, + RSC_VENDOR_END = 512, + }; + + struct resource_table { + u32 ver; + u32 num; + u32 reserved[2]; + u32 offset[]; + } __packed; + + struct fw_rsc_hdr { + u32 type; + u8 data[]; + } __packed; + +For example, if the remote requests both `RSC_TRACE` and +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured +as follows: + +.. code-block:: c + + #define MAX_SHARED_RESOURCE 2 + #define LOG_BUF_SIZE 1000 + #define CARVEOUT_DUMP_PA 0x12345678 + #define CARVEOUT_DUMP_SIZE 2000 + + struct shared_resource_table { + u32 ver; + u32 num; + u32 reserved[2]; + u32 offset[MAX_SHARED_RESOURCE]; + struct fw_rsc_trace log_trace; + struct fw_rsc_carveout dump_carveout; + }; + + volatile struct shared_resource_table table = { + .ver = 1, + .num = 2, + .reserved = {0, 0}, + .offset = { + offsetof(struct resource_table, log_trace), + offsetof(struct resource_table, dump_carveout), + }, + .log_trace = { + RSC_TRACE, + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", + }, + .dump_carveout = { + RSC_CARVEOUT, + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", + }, + }; + +The framework creates a sysfs file when it encounters the `RSC_TRACE` +type to expose log information to userspace. Other resource types are +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. + + +3. Remoteproc driver responsibilities +===================================== + +The driver must provide the following information to the core: + +a. Translate device addresses (physical addresses) found in the ELF + firmware to virtual addresses in Linux using the `da_to_va` + callback. This allows the framework to copy ELF firmware from the + filesystem to the addresses expected by the remote since + the framework cannot directly access those physical addresses. +b. Prepare/unprepare the remote prior to firmware loading, + which may involve allocating carveout and reserved memory regions. +c. Implement methods for starting and stopping the remote, + whether by setting registers or sending explicit interrupts, + depending on the hardware design. +d. Provide attach and detach callbacks to start the remote + without loading the firmware. This is beneficial when the remote + processor is already loaded and running. +e. Implement a load callback for firmware loading, typically using + the ELF loader provided by the framework; currently, only ELF + format is supported. +f. Invoke the framework's crash handler API upon detecting a remote + crash. + +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` +to register themselves with the framework. + +.. code-block:: c + + struct rproc_ops { + int (*prepare)(struct rproc *rproc); + int (*unprepare)(struct rproc *rproc); + int (*start)(struct rproc *rproc); + int (*stop)(struct rproc *rproc); + int (*attach)(struct rproc *rproc); + int (*detach)(struct rproc *rproc); + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, + bool *is_iomem); + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, + void *rsc, int offset, int avail); + int (*load)(struct rproc *rproc, const struct firmware *fw); + //snip + }; + + +4. Virtio and Remoteproc ======================== -These callbacks should be provided by platform-specific remoteproc -drivers:: - - /** - * struct rproc_ops - platform-specific device handlers - * @start: power on the device and boot it - * @stop: power off the device - * @kick: kick a virtqueue (virtqueue id given as a parameter) - */ - struct rproc_ops { - int (*start)(struct rproc *rproc); - int (*stop)(struct rproc *rproc); - void (*kick)(struct rproc *rproc, int vqid); - }; - -Every remoteproc implementation should at least provide the ->start and ->stop -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler -should be provided as well. - -The ->start() handler takes an rproc handle and should then power on the -device and boot it (use rproc->priv to access platform-specific private data). -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc -core puts there the ELF entry point). -On success, 0 should be returned, and on failure, an appropriate error code. - -The ->stop() handler takes an rproc handle and powers the device down. -On success, 0 is returned, and on failure, an appropriate error code. - -The ->kick() handler takes an rproc handle, and an index of a virtqueue -where new message was placed in. Implementations should interrupt the remote -processor and let it know it has pending messages. Notifying remote processors -the exact virtqueue index to look in is optional: it is easy (and not -too expensive) to go through the existing virtqueues and look for new buffers -in the used rings. - -Binary Firmware Structure -========================= - -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, -it is quite expected that other platforms/devices which we'd want to -support with this framework will be based on different binary formats. - -When those use cases show up, we will have to decouple the binary format -from the framework core, so we can support several binary formats without -duplicating common code. - -When the firmware is parsed, its various segments are loaded to memory -according to the specified device address (might be a physical address -if the remote processor is accessing memory directly). - -In addition to the standard ELF segments, most remote processors would -also include a special section which we call "the resource table". - -The resource table contains system resources that the remote processor -requires before it should be powered on, such as allocation of physically -contiguous memory, or iommu mapping of certain on-chip peripherals. -Remotecore will only power up the device after all the resource table's -requirement are met. - -In addition to system resources, the resource table may also contain -resource entries that publish the existence of supported features -or configurations by the remote processor, such as trace buffers and -supported virtio devices (and their configurations). - -The resource table begins with this header:: - - /** - * struct resource_table - firmware resource table header - * @ver: version number - * @num: number of resource entries - * @reserved: reserved (must be zero) - * @offset: array of offsets pointing at the various resource entries - * - * The header of the resource table, as expressed by this structure, - * contains a version number (should we need to change this format in the - * future), the number of available resource entries, and their offsets - * in the table. - */ - struct resource_table { - u32 ver; - u32 num; - u32 reserved[2]; - u32 offset[0]; - } __packed; - -Immediately following this header are the resource entries themselves, -each of which begins with the following resource entry header:: - - /** - * struct fw_rsc_hdr - firmware resource entry header - * @type: resource type - * @data: resource data - * - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing - * its @type. The content of the entry itself will immediately follow - * this header, and it should be parsed according to the resource type. - */ - struct fw_rsc_hdr { - u32 type; - u8 data[0]; - } __packed; - -Some resources entries are mere announcements, where the host is informed -of specific remoteproc configuration. Other entries require the host to -do something (e.g. allocate a system resource). Sometimes a negotiation -is expected, where the firmware requests a resource, and once allocated, -the host should provide back its details (e.g. address of an allocated -memory region). - -Here are the various resource types that are currently supported:: - - /** - * enum fw_resource_type - types of resource entries - * - * @RSC_CARVEOUT: request for allocation of a physically contiguous - * memory region. - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. - * @RSC_TRACE: announces the availability of a trace buffer into which - * the remote processor will be writing logs. - * @RSC_VDEV: declare support for a virtio device, and serve as its - * virtio header. - * @RSC_LAST: just keep this one at the end - * @RSC_VENDOR_START: start of the vendor specific resource types range - * @RSC_VENDOR_END: end of the vendor specific resource types range - * - * Please note that these values are used as indices to the rproc_handle_rsc - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to - * check the validity of an index before the lookup table is accessed, so - * please update it as needed. - */ - enum fw_resource_type { - RSC_CARVEOUT = 0, - RSC_DEVMEM = 1, - RSC_TRACE = 2, - RSC_VDEV = 3, - RSC_LAST = 4, - RSC_VENDOR_START = 128, - RSC_VENDOR_END = 512, - }; - -For more details regarding a specific resource type, please see its -dedicated structure in include/linux/remoteproc.h. - -We also expect that platform-specific resource entries will show up -at some point. When that happens, we could easily add a new RSC_PLATFORM -type, and hand those resources to the platform-specific rproc driver to handle. - -Virtio and remoteproc -===================== - -The firmware should provide remoteproc information about virtio devices -that it supports, and their configurations: a RSC_VDEV resource entry -should specify the virtio device id (as in virtio_ids.h), virtio features, -virtio config space, vrings information, etc. - -When a new remote processor is registered, the remoteproc framework -will look for its resource table and will register the virtio devices -it supports. A firmware may support any number of virtio devices, and -of any type (a single remote processor can also easily support several -rpmsg virtio devices this way, if desired). - -Of course, RSC_VDEV resource entries are only good enough for static -allocation of virtio devices. Dynamic allocations will also be made possible -using the rpmsg bus (similar to how we already do dynamic allocations of -rpmsg channels; read more about it in rpmsg.txt). +The firmware must provide remoteproc with information regarding the +virtio devices it supports and their configurations: an `RSC_VDEV` +resource entry should detail the virtio device ID (as defined in +`virtio_ids.h`), virtio features, virtio config space, vrings +information, etc. + +Upon registration of a new remote, the remoteproc framework +searches for its resource table and registers the supported virtio +devices. A firmware may support multiple virtio devices, of various +types (a single remote can support multiple rpmsg virtio +devices if required). + +Moreover, `RSC_VDEV` resource entries suffice for static allocation +of virtio devices. Dynamic allocations will also be supported using +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg +channels. For more information, refer to rpmsg documentation. -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH V4 3/7] Documentation: remoteproc: add more information 2024-10-23 5:33 ` [PATCH V4 3/7] Documentation: remoteproc: add more information anish kumar @ 2024-10-24 15:53 ` Mathieu Poirier 2024-10-24 18:17 ` anish kumar 0 siblings, 1 reply; 18+ messages in thread From: Mathieu Poirier @ 2024-10-24 15:53 UTC (permalink / raw) To: anish kumar Cc: ohad, bjorn.andersson, corbet, linux-remoteproc, linux-doc, linux-kernel On Tue, Oct 22, 2024 at 10:33:53PM -0700, anish kumar wrote: > Added following changes: > 1. Components provided by remoteproc framework. > 2. Remoteproc driver responsibilities. > 3. Remoteproc framework responsibilities. > 4. Better explanation of how to ask for resources > from the framework by the remote processor. > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > --- > .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- > 1 file changed, 243 insertions(+), 353 deletions(-) > > diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst > index 9cccd3dd6a4b..7ca545eea153 100644 > --- a/Documentation/driver-api/remoteproc/remoteproc.rst > +++ b/Documentation/driver-api/remoteproc/remoteproc.rst > @@ -1,359 +1,249 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > ========================== > Remote Processor Framework > ========================== > > -Introduction > -============ > - > -Modern SoCs typically have heterogeneous remote processor devices in asymmetric > -multiprocessing (AMP) configurations, which may be running different instances > -of operating system, whether it's Linux or any other flavor of real-time OS. > - > -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. > -In a typical configuration, the dual cortex-A9 is running Linux in a SMP > -configuration, and each of the other three cores (two M3 cores and a DSP) > -is running its own instance of RTOS in an AMP configuration. > - > -The remoteproc framework allows different platforms/architectures to > -control (power on, load firmware, power off) those remote processors while > -abstracting the hardware differences, so the entire driver doesn't need to be > -duplicated. In addition, this framework also adds rpmsg virtio devices > -for remote processors that supports this kind of communication. This way, > -platform-specific remoteproc drivers only need to provide a few low-level > -handlers, and then all rpmsg drivers will then just work > -(for more information about the virtio-based rpmsg bus and its drivers, > -please read Documentation/staging/rpmsg.rst). > -Registration of other types of virtio devices is now also possible. Firmwares > -just need to publish what kind of virtio devices do they support, and then > -remoteproc will add those devices. This makes it possible to reuse the > -existing virtio drivers with remote processor backends at a minimal development > -cost. > - > -User API > -======== > - > -:: > - > - int rproc_boot(struct rproc *rproc) > - > -Boot a remote processor (i.e. load its firmware, power it on, ...). > - > -If the remote processor is already powered on, this function immediately > -returns (successfully). > - > -Returns 0 on success, and an appropriate error value otherwise. > -Note: to use this function you should already have a valid rproc > -handle. There are several ways to achieve that cleanly (devres, pdata, > -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we > -might also consider using dev_archdata for this). > - > -:: > - > - void rproc_shutdown(struct rproc *rproc) > - > -Power off a remote processor (previously booted with rproc_boot()). > -In case @rproc is still being used by an additional user(s), then > -this function will just decrement the power refcount and exit, > -without really powering off the device. > - > -Every call to rproc_boot() must (eventually) be accompanied by a call > -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. > - > -.. note:: > - > - we're not decrementing the rproc's refcount, only the power refcount. > - which means that the @rproc handle stays valid even after > - rproc_shutdown() returns, and users can still use it with a subsequent > - rproc_boot(), if needed. > - > -:: > - > - struct rproc *rproc_get_by_phandle(phandle phandle) > - > -Find an rproc handle using a device tree phandle. Returns the rproc > -handle on success, and NULL on failure. This function increments > -the remote processor's refcount, so always use rproc_put() to > -decrement it back once rproc isn't needed anymore. > - > -Typical usage > -============= > - > -:: > - > - #include <linux/remoteproc.h> > - > - /* in case we were given a valid 'rproc' handle */ > - int dummy_rproc_example(struct rproc *my_rproc) > - { > - int ret; > - > - /* let's power on and boot our remote processor */ > - ret = rproc_boot(my_rproc); > - if (ret) { > - /* > - * something went wrong. handle it and leave. > - */ > - } > - > - /* > - * our remote processor is now powered on... give it some work > - */ > - > - /* let's shut it down now */ > - rproc_shutdown(my_rproc); > - } > - > -API for implementors > -==================== > - > -:: > - > - struct rproc *rproc_alloc(struct device *dev, const char *name, > - const struct rproc_ops *ops, > - const char *firmware, int len) > - > -Allocate a new remote processor handle, but don't register > -it yet. Required parameters are the underlying device, the > -name of this remote processor, platform-specific ops handlers, > -the name of the firmware to boot this rproc with, and the > -length of private data needed by the allocating rproc driver (in bytes). > - > -This function should be used by rproc implementations during > -initialization of the remote processor. > - > -After creating an rproc handle using this function, and when ready, > -implementations should then call rproc_add() to complete > -the registration of the remote processor. > - > -On success, the new rproc is returned, and on failure, NULL. > - > -.. note:: > - > - **never** directly deallocate @rproc, even if it was not registered > - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). > - > -:: > - > - void rproc_free(struct rproc *rproc) > - > -Free an rproc handle that was allocated by rproc_alloc. > - > -This function essentially unrolls rproc_alloc(), by decrementing the > -rproc's refcount. It doesn't directly free rproc; that would happen > -only if there are no other references to rproc and its refcount now > -dropped to zero. > - > -:: > - > - int rproc_add(struct rproc *rproc) > - > -Register @rproc with the remoteproc framework, after it has been > -allocated with rproc_alloc(). > - > -This is called by the platform-specific rproc implementation, whenever > -a new remote processor device is probed. > - > -Returns 0 on success and an appropriate error code otherwise. > -Note: this function initiates an asynchronous firmware loading > -context, which will look for virtio devices supported by the rproc's > -firmware. > - > -If found, those virtio devices will be created and added, so as a result > -of registering this remote processor, additional virtio drivers might get > -probed. > - > -:: > - > - int rproc_del(struct rproc *rproc) > - > -Unroll rproc_add(). > - > -This function should be called when the platform specific rproc > -implementation decides to remove the rproc device. it should > -_only_ be called if a previous invocation of rproc_add() > -has completed successfully. > - > -After rproc_del() returns, @rproc is still valid, and its > -last refcount should be decremented by calling rproc_free(). > - > -Returns 0 on success and -EINVAL if @rproc isn't valid. > - > -:: > - > - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) > - > -Report a crash in a remoteproc > - > -This function must be called every time a crash is detected by the > -platform specific rproc implementation. This should not be called from a > -non-remoteproc driver. This function can be called from atomic/interrupt > -context. > - > -Implementation callbacks > +.. Contents: > + > + 1. Introduction > + 2. Remoteproc framework responsibilities > + 3. Remoteproc driver responsibilities > + 4. Virtio and rpmsg > + > +1. Introduction > +=============== > + > +Modern System on Chips (SoCs) typically integrate heterogeneous remote > +processor devices in asymmetric multiprocessing (AMP) configurations. > +These processors may run different operating systems, such as Linux and > +various real-time operating systems (RTOS). You are moving things around _and_ making modifications to the text in the same patch, something I specifically asked not to do. Moreover, the above conveys exactly the same information as found in [1] but using different words. I am in favour of enhancing documentation but not creating unneeded churn. I found several instances of the same rewording pattern in the sections below. As such I will not look at the other patches nor move forward with this set. Thanks, Mathieu [1]. https://elixir.bootlin.com/linux/v6.12-rc4/source/Documentation/staging/remoteproc.rst > + > +For example, the OMAP4 platform features dual Cortex-A9 cores, dual > +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 > +cores execute Linux in a symmetric multiprocessing (SMP) configuration, > +while the M3 cores and DSP run independent instances of an RTOS. > + > +The remoteproc framework allows various platforms and architectures to > +manage remote processors, including operations such as powering on, > +loading firmware, and powering off. This framework abstracts hardware > +differences, promoting code reuse and minimizing duplication. It also > +supports rpmsg virtio devices for remote processors that utilize this > +communication method. Consequently, platform-specific remoteproc drivers > +need only implement a few low-level handlers, enabling seamless operation > +of all rpmsg drivers. (For more details about the virtio-based rpmsg > +bus and its drivers, refer to rpmsg documentation.) > + > +Additionally, the framework allows for the registration of various > +virtio devices. Firmware can publish the types of virtio devices it > +supports, facilitating their addition to the remoteproc framework. This > +flexibility enables the reuse of existing virtio drivers with remote > +processor backends at minimal development cost. > + > +The primary purpose of the remoteproc framework is to download firmware > +for remote processors and manage their lifecycle. The framework consists > +of several key components: > + > +- **Character Driver**: Provides userspace access to control the remote > + processor. > +- **ELF Utility**: Offers functions for handling ELF files and managing > + resources requested by the remote processor. > +- **Remoteproc Core**: Manages firmware downloads and recovery actions > + in case of a remote processor crash. > +- **Coredump**: Provides facilities for coredumping and tracing from > + the remote processor in the event of a crash. > +- **Userspace Interaction**: Uses sysfs and debugfs to manage the > + lifecycle and status of the remote processor. > +- **Virtio Support**: Facilitates interaction with the virtio and > + rpmsg bus. > + > +From here on, references to "framework" denote the remoteproc > +framework, and "driver" refers to the remoteproc driver that utilizes > +the framework for managing remote processors. > + > +2. Remoteproc framework Responsibilities > +======================================== > + > +The framework begins by gathering information about the firmware file > +to be downloaded through the request_firmware function. It supports > +the ELF format and parses the firmware image to identify the physical > +addresses that need to be populated from the corresponding ELF sections. > +The framework also requires knowledge of the logical or I/O-mapped > +addresses in the application processor. Once this information is > +obtained from the driver, the framework transfers the data to the > +specified addresses and starts the remote, along with > +any devices physically or logically connected to it. > + > +Dependent devices, referred to as `subdevices` within the framework, > +are also managed post-registration by their respective drivers. > +Subdevices can register themselves using `rproc_(add/remove)_subdev`. > +Non-remoteproc drivers can use subdevices as a way to logically connect > +to remote and get lifecycle notifications of the remote. > + > +The framework oversees the lifecycle of the remote and > +provides the `rproc_report_crash` function, which the driver invokes > +upon receiving a crash notification from the remote. The > +notification method can differ based on the design of the remote > +processor and its communication with the application processor. For > +instance, if the remote is a DSP equipped with a watchdog, > +unresponsive behavior triggers the watchdog, generating an interrupt > +that routes to the application processor, allowing it to call > +`rproc_report_crash` in the driver's interrupt context. > + > +During crash handling, the framework performs the following actions: > + > +a. Sends a request to stop the remote and any connected or > + dependent subdevices. > +b. Generates a coredump, dumping all `resources` requested by the > + remote alongside relevant debugging information. Resources are > + explained below. > +c. Reloads the firmware and restarts the remote. > + > +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and > +attach callbacks of the driver are invoked without reloading the > +firmware. This is useful when the remote requires no > +assistance for recovery, or when the application processor can restart > +independently. After recovery, the application processor can reattach > +to the remote. > + > +The remote can request resources from the framework, which > +allocates a ".resource_table" section. During the ELF parsing phase, > +the framework identifies this section and calls the appropriate > +handler to allocate the requested resources. > + > +Resource management within the framework can accommodate any type of > +`fw_resource_type`. > + > +.. code-block:: c > + > + enum fw_resource_type { > + RSC_CARVEOUT = 0, > + RSC_DEVMEM = 1, > + RSC_TRACE = 2, > + RSC_VDEV = 3, > + RSC_LAST = 4, > + RSC_VENDOR_START = 128, > + RSC_VENDOR_END = 512, > + }; > + > + struct resource_table { > + u32 ver; > + u32 num; > + u32 reserved[2]; > + u32 offset[]; > + } __packed; > + > + struct fw_rsc_hdr { > + u32 type; > + u8 data[]; > + } __packed; > + > +For example, if the remote requests both `RSC_TRACE` and > +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured > +as follows: > + > +.. code-block:: c > + > + #define MAX_SHARED_RESOURCE 2 > + #define LOG_BUF_SIZE 1000 > + #define CARVEOUT_DUMP_PA 0x12345678 > + #define CARVEOUT_DUMP_SIZE 2000 > + > + struct shared_resource_table { > + u32 ver; > + u32 num; > + u32 reserved[2]; > + u32 offset[MAX_SHARED_RESOURCE]; > + struct fw_rsc_trace log_trace; > + struct fw_rsc_carveout dump_carveout; > + }; > + > + volatile struct shared_resource_table table = { > + .ver = 1, > + .num = 2, > + .reserved = {0, 0}, > + .offset = { > + offsetof(struct resource_table, log_trace), > + offsetof(struct resource_table, dump_carveout), > + }, > + .log_trace = { > + RSC_TRACE, > + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", > + }, > + .dump_carveout = { > + RSC_CARVEOUT, > + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", > + }, > + }; > + > +The framework creates a sysfs file when it encounters the `RSC_TRACE` > +type to expose log information to userspace. Other resource types are > +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes > +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. > + > + > +3. Remoteproc driver responsibilities > +===================================== > + > +The driver must provide the following information to the core: > + > +a. Translate device addresses (physical addresses) found in the ELF > + firmware to virtual addresses in Linux using the `da_to_va` > + callback. This allows the framework to copy ELF firmware from the > + filesystem to the addresses expected by the remote since > + the framework cannot directly access those physical addresses. > +b. Prepare/unprepare the remote prior to firmware loading, > + which may involve allocating carveout and reserved memory regions. > +c. Implement methods for starting and stopping the remote, > + whether by setting registers or sending explicit interrupts, > + depending on the hardware design. > +d. Provide attach and detach callbacks to start the remote > + without loading the firmware. This is beneficial when the remote > + processor is already loaded and running. > +e. Implement a load callback for firmware loading, typically using > + the ELF loader provided by the framework; currently, only ELF > + format is supported. > +f. Invoke the framework's crash handler API upon detecting a remote > + crash. > + > +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` > +to register themselves with the framework. > + > +.. code-block:: c > + > + struct rproc_ops { > + int (*prepare)(struct rproc *rproc); > + int (*unprepare)(struct rproc *rproc); > + int (*start)(struct rproc *rproc); > + int (*stop)(struct rproc *rproc); > + int (*attach)(struct rproc *rproc); > + int (*detach)(struct rproc *rproc); > + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, > + bool *is_iomem); > + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); > + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, > + void *rsc, int offset, int avail); > + int (*load)(struct rproc *rproc, const struct firmware *fw); > + //snip > + }; > + > + > +4. Virtio and Remoteproc > ======================== > > -These callbacks should be provided by platform-specific remoteproc > -drivers:: > - > - /** > - * struct rproc_ops - platform-specific device handlers > - * @start: power on the device and boot it > - * @stop: power off the device > - * @kick: kick a virtqueue (virtqueue id given as a parameter) > - */ > - struct rproc_ops { > - int (*start)(struct rproc *rproc); > - int (*stop)(struct rproc *rproc); > - void (*kick)(struct rproc *rproc, int vqid); > - }; > - > -Every remoteproc implementation should at least provide the ->start and ->stop > -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler > -should be provided as well. > - > -The ->start() handler takes an rproc handle and should then power on the > -device and boot it (use rproc->priv to access platform-specific private data). > -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc > -core puts there the ELF entry point). > -On success, 0 should be returned, and on failure, an appropriate error code. > - > -The ->stop() handler takes an rproc handle and powers the device down. > -On success, 0 is returned, and on failure, an appropriate error code. > - > -The ->kick() handler takes an rproc handle, and an index of a virtqueue > -where new message was placed in. Implementations should interrupt the remote > -processor and let it know it has pending messages. Notifying remote processors > -the exact virtqueue index to look in is optional: it is easy (and not > -too expensive) to go through the existing virtqueues and look for new buffers > -in the used rings. > - > -Binary Firmware Structure > -========================= > - > -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, > -it is quite expected that other platforms/devices which we'd want to > -support with this framework will be based on different binary formats. > - > -When those use cases show up, we will have to decouple the binary format > -from the framework core, so we can support several binary formats without > -duplicating common code. > - > -When the firmware is parsed, its various segments are loaded to memory > -according to the specified device address (might be a physical address > -if the remote processor is accessing memory directly). > - > -In addition to the standard ELF segments, most remote processors would > -also include a special section which we call "the resource table". > - > -The resource table contains system resources that the remote processor > -requires before it should be powered on, such as allocation of physically > -contiguous memory, or iommu mapping of certain on-chip peripherals. > -Remotecore will only power up the device after all the resource table's > -requirement are met. > - > -In addition to system resources, the resource table may also contain > -resource entries that publish the existence of supported features > -or configurations by the remote processor, such as trace buffers and > -supported virtio devices (and their configurations). > - > -The resource table begins with this header:: > - > - /** > - * struct resource_table - firmware resource table header > - * @ver: version number > - * @num: number of resource entries > - * @reserved: reserved (must be zero) > - * @offset: array of offsets pointing at the various resource entries > - * > - * The header of the resource table, as expressed by this structure, > - * contains a version number (should we need to change this format in the > - * future), the number of available resource entries, and their offsets > - * in the table. > - */ > - struct resource_table { > - u32 ver; > - u32 num; > - u32 reserved[2]; > - u32 offset[0]; > - } __packed; > - > -Immediately following this header are the resource entries themselves, > -each of which begins with the following resource entry header:: > - > - /** > - * struct fw_rsc_hdr - firmware resource entry header > - * @type: resource type > - * @data: resource data > - * > - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing > - * its @type. The content of the entry itself will immediately follow > - * this header, and it should be parsed according to the resource type. > - */ > - struct fw_rsc_hdr { > - u32 type; > - u8 data[0]; > - } __packed; > - > -Some resources entries are mere announcements, where the host is informed > -of specific remoteproc configuration. Other entries require the host to > -do something (e.g. allocate a system resource). Sometimes a negotiation > -is expected, where the firmware requests a resource, and once allocated, > -the host should provide back its details (e.g. address of an allocated > -memory region). > - > -Here are the various resource types that are currently supported:: > - > - /** > - * enum fw_resource_type - types of resource entries > - * > - * @RSC_CARVEOUT: request for allocation of a physically contiguous > - * memory region. > - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. > - * @RSC_TRACE: announces the availability of a trace buffer into which > - * the remote processor will be writing logs. > - * @RSC_VDEV: declare support for a virtio device, and serve as its > - * virtio header. > - * @RSC_LAST: just keep this one at the end > - * @RSC_VENDOR_START: start of the vendor specific resource types range > - * @RSC_VENDOR_END: end of the vendor specific resource types range > - * > - * Please note that these values are used as indices to the rproc_handle_rsc > - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to > - * check the validity of an index before the lookup table is accessed, so > - * please update it as needed. > - */ > - enum fw_resource_type { > - RSC_CARVEOUT = 0, > - RSC_DEVMEM = 1, > - RSC_TRACE = 2, > - RSC_VDEV = 3, > - RSC_LAST = 4, > - RSC_VENDOR_START = 128, > - RSC_VENDOR_END = 512, > - }; > - > -For more details regarding a specific resource type, please see its > -dedicated structure in include/linux/remoteproc.h. > - > -We also expect that platform-specific resource entries will show up > -at some point. When that happens, we could easily add a new RSC_PLATFORM > -type, and hand those resources to the platform-specific rproc driver to handle. > - > -Virtio and remoteproc > -===================== > - > -The firmware should provide remoteproc information about virtio devices > -that it supports, and their configurations: a RSC_VDEV resource entry > -should specify the virtio device id (as in virtio_ids.h), virtio features, > -virtio config space, vrings information, etc. > - > -When a new remote processor is registered, the remoteproc framework > -will look for its resource table and will register the virtio devices > -it supports. A firmware may support any number of virtio devices, and > -of any type (a single remote processor can also easily support several > -rpmsg virtio devices this way, if desired). > - > -Of course, RSC_VDEV resource entries are only good enough for static > -allocation of virtio devices. Dynamic allocations will also be made possible > -using the rpmsg bus (similar to how we already do dynamic allocations of > -rpmsg channels; read more about it in rpmsg.txt). > +The firmware must provide remoteproc with information regarding the > +virtio devices it supports and their configurations: an `RSC_VDEV` > +resource entry should detail the virtio device ID (as defined in > +`virtio_ids.h`), virtio features, virtio config space, vrings > +information, etc. > + > +Upon registration of a new remote, the remoteproc framework > +searches for its resource table and registers the supported virtio > +devices. A firmware may support multiple virtio devices, of various > +types (a single remote can support multiple rpmsg virtio > +devices if required). > + > +Moreover, `RSC_VDEV` resource entries suffice for static allocation > +of virtio devices. Dynamic allocations will also be supported using > +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg > +channels. For more information, refer to rpmsg documentation. > -- > 2.39.3 (Apple Git-146) > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V4 3/7] Documentation: remoteproc: add more information 2024-10-24 15:53 ` Mathieu Poirier @ 2024-10-24 18:17 ` anish kumar 2024-10-25 14:57 ` Mathieu Poirier 0 siblings, 1 reply; 18+ messages in thread From: anish kumar @ 2024-10-24 18:17 UTC (permalink / raw) To: Mathieu Poirier Cc: ohad, bjorn.andersson, corbet, linux-remoteproc, linux-doc, linux-kernel On Thu, Oct 24, 2024 at 8:53 AM Mathieu Poirier <mathieu.poirier@linaro.org> wrote: > > On Tue, Oct 22, 2024 at 10:33:53PM -0700, anish kumar wrote: > > Added following changes: > > 1. Components provided by remoteproc framework. > > 2. Remoteproc driver responsibilities. > > 3. Remoteproc framework responsibilities. > > 4. Better explanation of how to ask for resources > > from the framework by the remote processor. > > > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > > --- > > .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- > > 1 file changed, 243 insertions(+), 353 deletions(-) > > > > diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst > > index 9cccd3dd6a4b..7ca545eea153 100644 > > --- a/Documentation/driver-api/remoteproc/remoteproc.rst > > +++ b/Documentation/driver-api/remoteproc/remoteproc.rst > > @@ -1,359 +1,249 @@ > > +.. SPDX-License-Identifier: GPL-2.0 > > + > > ========================== > > Remote Processor Framework > > ========================== > > > > -Introduction > > -============ > > - > > -Modern SoCs typically have heterogeneous remote processor devices in asymmetric > > -multiprocessing (AMP) configurations, which may be running different instances > > -of operating system, whether it's Linux or any other flavor of real-time OS. > > - > > -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. > > -In a typical configuration, the dual cortex-A9 is running Linux in a SMP > > -configuration, and each of the other three cores (two M3 cores and a DSP) > > -is running its own instance of RTOS in an AMP configuration. > > - > > -The remoteproc framework allows different platforms/architectures to > > -control (power on, load firmware, power off) those remote processors while > > -abstracting the hardware differences, so the entire driver doesn't need to be > > -duplicated. In addition, this framework also adds rpmsg virtio devices > > -for remote processors that supports this kind of communication. This way, > > -platform-specific remoteproc drivers only need to provide a few low-level > > -handlers, and then all rpmsg drivers will then just work > > -(for more information about the virtio-based rpmsg bus and its drivers, > > -please read Documentation/staging/rpmsg.rst). > > -Registration of other types of virtio devices is now also possible. Firmwares > > -just need to publish what kind of virtio devices do they support, and then > > -remoteproc will add those devices. This makes it possible to reuse the > > -existing virtio drivers with remote processor backends at a minimal development > > -cost. > > - > > -User API > > -======== > > - > > -:: > > - > > - int rproc_boot(struct rproc *rproc) > > - > > -Boot a remote processor (i.e. load its firmware, power it on, ...). > > - > > -If the remote processor is already powered on, this function immediately > > -returns (successfully). > > - > > -Returns 0 on success, and an appropriate error value otherwise. > > -Note: to use this function you should already have a valid rproc > > -handle. There are several ways to achieve that cleanly (devres, pdata, > > -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we > > -might also consider using dev_archdata for this). > > - > > -:: > > - > > - void rproc_shutdown(struct rproc *rproc) > > - > > -Power off a remote processor (previously booted with rproc_boot()). > > -In case @rproc is still being used by an additional user(s), then > > -this function will just decrement the power refcount and exit, > > -without really powering off the device. > > - > > -Every call to rproc_boot() must (eventually) be accompanied by a call > > -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. > > - > > -.. note:: > > - > > - we're not decrementing the rproc's refcount, only the power refcount. > > - which means that the @rproc handle stays valid even after > > - rproc_shutdown() returns, and users can still use it with a subsequent > > - rproc_boot(), if needed. > > - > > -:: > > - > > - struct rproc *rproc_get_by_phandle(phandle phandle) > > - > > -Find an rproc handle using a device tree phandle. Returns the rproc > > -handle on success, and NULL on failure. This function increments > > -the remote processor's refcount, so always use rproc_put() to > > -decrement it back once rproc isn't needed anymore. > > - > > -Typical usage > > -============= > > - > > -:: > > - > > - #include <linux/remoteproc.h> > > - > > - /* in case we were given a valid 'rproc' handle */ > > - int dummy_rproc_example(struct rproc *my_rproc) > > - { > > - int ret; > > - > > - /* let's power on and boot our remote processor */ > > - ret = rproc_boot(my_rproc); > > - if (ret) { > > - /* > > - * something went wrong. handle it and leave. > > - */ > > - } > > - > > - /* > > - * our remote processor is now powered on... give it some work > > - */ > > - > > - /* let's shut it down now */ > > - rproc_shutdown(my_rproc); > > - } > > - > > -API for implementors > > -==================== > > - > > -:: > > - > > - struct rproc *rproc_alloc(struct device *dev, const char *name, > > - const struct rproc_ops *ops, > > - const char *firmware, int len) > > - > > -Allocate a new remote processor handle, but don't register > > -it yet. Required parameters are the underlying device, the > > -name of this remote processor, platform-specific ops handlers, > > -the name of the firmware to boot this rproc with, and the > > -length of private data needed by the allocating rproc driver (in bytes). > > - > > -This function should be used by rproc implementations during > > -initialization of the remote processor. > > - > > -After creating an rproc handle using this function, and when ready, > > -implementations should then call rproc_add() to complete > > -the registration of the remote processor. > > - > > -On success, the new rproc is returned, and on failure, NULL. > > - > > -.. note:: > > - > > - **never** directly deallocate @rproc, even if it was not registered > > - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). > > - > > -:: > > - > > - void rproc_free(struct rproc *rproc) > > - > > -Free an rproc handle that was allocated by rproc_alloc. > > - > > -This function essentially unrolls rproc_alloc(), by decrementing the > > -rproc's refcount. It doesn't directly free rproc; that would happen > > -only if there are no other references to rproc and its refcount now > > -dropped to zero. > > - > > -:: > > - > > - int rproc_add(struct rproc *rproc) > > - > > -Register @rproc with the remoteproc framework, after it has been > > -allocated with rproc_alloc(). > > - > > -This is called by the platform-specific rproc implementation, whenever > > -a new remote processor device is probed. > > - > > -Returns 0 on success and an appropriate error code otherwise. > > -Note: this function initiates an asynchronous firmware loading > > -context, which will look for virtio devices supported by the rproc's > > -firmware. > > - > > -If found, those virtio devices will be created and added, so as a result > > -of registering this remote processor, additional virtio drivers might get > > -probed. > > - > > -:: > > - > > - int rproc_del(struct rproc *rproc) > > - > > -Unroll rproc_add(). > > - > > -This function should be called when the platform specific rproc > > -implementation decides to remove the rproc device. it should > > -_only_ be called if a previous invocation of rproc_add() > > -has completed successfully. > > - > > -After rproc_del() returns, @rproc is still valid, and its > > -last refcount should be decremented by calling rproc_free(). > > - > > -Returns 0 on success and -EINVAL if @rproc isn't valid. > > - > > -:: > > - > > - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) > > - > > -Report a crash in a remoteproc > > - > > -This function must be called every time a crash is detected by the > > -platform specific rproc implementation. This should not be called from a > > -non-remoteproc driver. This function can be called from atomic/interrupt > > -context. > > - > > -Implementation callbacks > > +.. Contents: > > + > > + 1. Introduction > > + 2. Remoteproc framework responsibilities > > + 3. Remoteproc driver responsibilities > > + 4. Virtio and rpmsg > > + > > +1. Introduction > > +=============== > > + > > +Modern System on Chips (SoCs) typically integrate heterogeneous remote > > +processor devices in asymmetric multiprocessing (AMP) configurations. > > +These processors may run different operating systems, such as Linux and > > +various real-time operating systems (RTOS). > > You are moving things around _and_ making modifications to the text in the same > patch, something I specifically asked not to do. Moreover, the above conveys Sorry for doing that, I thought I moved the file first as you instructed but I didn't understand that I should also not reword the sentences that convey the same meaning. Will keep the text of below content in the same file: 1. Introduction 2. Binary Firmware Structure: will just add example as shown in this patch. 3. Virtio and remoteproc: didn't change anything in the patch for this. For below: 1. User API: 2. Typical usage: 3. API for implementors : Should I just move all of the above text as it is to remoteproc-kernel-api.rst or you prefer here as I feel it belongs there? > exactly the same information as found in [1] but using different words. I > am in favour of enhancing documentation but not creating unneeded churn. > > I found several instances of the same rewording pattern in the sections below. > As such I will not look at the other patches nor move forward with this set. > > Thanks, > Mathieu Thanks Mathieu, for reviewing and providing feedback as always. > > [1]. https://elixir.bootlin.com/linux/v6.12-rc4/source/Documentation/staging/remoteproc.rst > > > + > > +For example, the OMAP4 platform features dual Cortex-A9 cores, dual > > +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 > > +cores execute Linux in a symmetric multiprocessing (SMP) configuration, > > +while the M3 cores and DSP run independent instances of an RTOS. > > + > > +The remoteproc framework allows various platforms and architectures to > > +manage remote processors, including operations such as powering on, > > +loading firmware, and powering off. This framework abstracts hardware > > +differences, promoting code reuse and minimizing duplication. It also > > +supports rpmsg virtio devices for remote processors that utilize this > > +communication method. Consequently, platform-specific remoteproc drivers > > +need only implement a few low-level handlers, enabling seamless operation > > +of all rpmsg drivers. (For more details about the virtio-based rpmsg > > +bus and its drivers, refer to rpmsg documentation.) > > + > > +Additionally, the framework allows for the registration of various > > +virtio devices. Firmware can publish the types of virtio devices it > > +supports, facilitating their addition to the remoteproc framework. This > > +flexibility enables the reuse of existing virtio drivers with remote > > +processor backends at minimal development cost. > > + > > +The primary purpose of the remoteproc framework is to download firmware > > +for remote processors and manage their lifecycle. The framework consists > > +of several key components: > > + > > +- **Character Driver**: Provides userspace access to control the remote > > + processor. > > +- **ELF Utility**: Offers functions for handling ELF files and managing > > + resources requested by the remote processor. > > +- **Remoteproc Core**: Manages firmware downloads and recovery actions > > + in case of a remote processor crash. > > +- **Coredump**: Provides facilities for coredumping and tracing from > > + the remote processor in the event of a crash. > > +- **Userspace Interaction**: Uses sysfs and debugfs to manage the > > + lifecycle and status of the remote processor. > > +- **Virtio Support**: Facilitates interaction with the virtio and > > + rpmsg bus. > > + > > +From here on, references to "framework" denote the remoteproc > > +framework, and "driver" refers to the remoteproc driver that utilizes > > +the framework for managing remote processors. > > + > > +2. Remoteproc framework Responsibilities > > +======================================== > > + > > +The framework begins by gathering information about the firmware file > > +to be downloaded through the request_firmware function. It supports > > +the ELF format and parses the firmware image to identify the physical > > +addresses that need to be populated from the corresponding ELF sections. > > +The framework also requires knowledge of the logical or I/O-mapped > > +addresses in the application processor. Once this information is > > +obtained from the driver, the framework transfers the data to the > > +specified addresses and starts the remote, along with > > +any devices physically or logically connected to it. > > + > > +Dependent devices, referred to as `subdevices` within the framework, > > +are also managed post-registration by their respective drivers. > > +Subdevices can register themselves using `rproc_(add/remove)_subdev`. > > +Non-remoteproc drivers can use subdevices as a way to logically connect > > +to remote and get lifecycle notifications of the remote. > > + > > +The framework oversees the lifecycle of the remote and > > +provides the `rproc_report_crash` function, which the driver invokes > > +upon receiving a crash notification from the remote. The > > +notification method can differ based on the design of the remote > > +processor and its communication with the application processor. For > > +instance, if the remote is a DSP equipped with a watchdog, > > +unresponsive behavior triggers the watchdog, generating an interrupt > > +that routes to the application processor, allowing it to call > > +`rproc_report_crash` in the driver's interrupt context. > > + > > +During crash handling, the framework performs the following actions: > > + > > +a. Sends a request to stop the remote and any connected or > > + dependent subdevices. > > +b. Generates a coredump, dumping all `resources` requested by the > > + remote alongside relevant debugging information. Resources are > > + explained below. > > +c. Reloads the firmware and restarts the remote. > > + > > +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and > > +attach callbacks of the driver are invoked without reloading the > > +firmware. This is useful when the remote requires no > > +assistance for recovery, or when the application processor can restart > > +independently. After recovery, the application processor can reattach > > +to the remote. > > + > > +The remote can request resources from the framework, which > > +allocates a ".resource_table" section. During the ELF parsing phase, > > +the framework identifies this section and calls the appropriate > > +handler to allocate the requested resources. > > + > > +Resource management within the framework can accommodate any type of > > +`fw_resource_type`. > > + > > +.. code-block:: c > > + > > + enum fw_resource_type { > > + RSC_CARVEOUT = 0, > > + RSC_DEVMEM = 1, > > + RSC_TRACE = 2, > > + RSC_VDEV = 3, > > + RSC_LAST = 4, > > + RSC_VENDOR_START = 128, > > + RSC_VENDOR_END = 512, > > + }; > > + > > + struct resource_table { > > + u32 ver; > > + u32 num; > > + u32 reserved[2]; > > + u32 offset[]; > > + } __packed; > > + > > + struct fw_rsc_hdr { > > + u32 type; > > + u8 data[]; > > + } __packed; > > + > > +For example, if the remote requests both `RSC_TRACE` and > > +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured > > +as follows: > > + > > +.. code-block:: c > > + > > + #define MAX_SHARED_RESOURCE 2 > > + #define LOG_BUF_SIZE 1000 > > + #define CARVEOUT_DUMP_PA 0x12345678 > > + #define CARVEOUT_DUMP_SIZE 2000 > > + > > + struct shared_resource_table { > > + u32 ver; > > + u32 num; > > + u32 reserved[2]; > > + u32 offset[MAX_SHARED_RESOURCE]; > > + struct fw_rsc_trace log_trace; > > + struct fw_rsc_carveout dump_carveout; > > + }; > > + > > + volatile struct shared_resource_table table = { > > + .ver = 1, > > + .num = 2, > > + .reserved = {0, 0}, > > + .offset = { > > + offsetof(struct resource_table, log_trace), > > + offsetof(struct resource_table, dump_carveout), > > + }, > > + .log_trace = { > > + RSC_TRACE, > > + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", > > + }, > > + .dump_carveout = { > > + RSC_CARVEOUT, > > + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", > > + }, > > + }; > > + > > +The framework creates a sysfs file when it encounters the `RSC_TRACE` > > +type to expose log information to userspace. Other resource types are > > +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes > > +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. > > + > > + > > +3. Remoteproc driver responsibilities > > +===================================== > > + > > +The driver must provide the following information to the core: > > + > > +a. Translate device addresses (physical addresses) found in the ELF > > + firmware to virtual addresses in Linux using the `da_to_va` > > + callback. This allows the framework to copy ELF firmware from the > > + filesystem to the addresses expected by the remote since > > + the framework cannot directly access those physical addresses. > > +b. Prepare/unprepare the remote prior to firmware loading, > > + which may involve allocating carveout and reserved memory regions. > > +c. Implement methods for starting and stopping the remote, > > + whether by setting registers or sending explicit interrupts, > > + depending on the hardware design. > > +d. Provide attach and detach callbacks to start the remote > > + without loading the firmware. This is beneficial when the remote > > + processor is already loaded and running. > > +e. Implement a load callback for firmware loading, typically using > > + the ELF loader provided by the framework; currently, only ELF > > + format is supported. > > +f. Invoke the framework's crash handler API upon detecting a remote > > + crash. > > + > > +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` > > +to register themselves with the framework. > > + > > +.. code-block:: c > > + > > + struct rproc_ops { > > + int (*prepare)(struct rproc *rproc); > > + int (*unprepare)(struct rproc *rproc); > > + int (*start)(struct rproc *rproc); > > + int (*stop)(struct rproc *rproc); > > + int (*attach)(struct rproc *rproc); > > + int (*detach)(struct rproc *rproc); > > + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, > > + bool *is_iomem); > > + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); > > + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, > > + void *rsc, int offset, int avail); > > + int (*load)(struct rproc *rproc, const struct firmware *fw); > > + //snip > > + }; > > + > > + > > +4. Virtio and Remoteproc > > ======================== > > > > -These callbacks should be provided by platform-specific remoteproc > > -drivers:: > > - > > - /** > > - * struct rproc_ops - platform-specific device handlers > > - * @start: power on the device and boot it > > - * @stop: power off the device > > - * @kick: kick a virtqueue (virtqueue id given as a parameter) > > - */ > > - struct rproc_ops { > > - int (*start)(struct rproc *rproc); > > - int (*stop)(struct rproc *rproc); > > - void (*kick)(struct rproc *rproc, int vqid); > > - }; > > - > > -Every remoteproc implementation should at least provide the ->start and ->stop > > -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler > > -should be provided as well. > > - > > -The ->start() handler takes an rproc handle and should then power on the > > -device and boot it (use rproc->priv to access platform-specific private data). > > -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc > > -core puts there the ELF entry point). > > -On success, 0 should be returned, and on failure, an appropriate error code. > > - > > -The ->stop() handler takes an rproc handle and powers the device down. > > -On success, 0 is returned, and on failure, an appropriate error code. > > - > > -The ->kick() handler takes an rproc handle, and an index of a virtqueue > > -where new message was placed in. Implementations should interrupt the remote > > -processor and let it know it has pending messages. Notifying remote processors > > -the exact virtqueue index to look in is optional: it is easy (and not > > -too expensive) to go through the existing virtqueues and look for new buffers > > -in the used rings. > > - > > -Binary Firmware Structure > > -========================= > > - > > -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, > > -it is quite expected that other platforms/devices which we'd want to > > -support with this framework will be based on different binary formats. > > - > > -When those use cases show up, we will have to decouple the binary format > > -from the framework core, so we can support several binary formats without > > -duplicating common code. > > - > > -When the firmware is parsed, its various segments are loaded to memory > > -according to the specified device address (might be a physical address > > -if the remote processor is accessing memory directly). > > - > > -In addition to the standard ELF segments, most remote processors would > > -also include a special section which we call "the resource table". > > - > > -The resource table contains system resources that the remote processor > > -requires before it should be powered on, such as allocation of physically > > -contiguous memory, or iommu mapping of certain on-chip peripherals. > > -Remotecore will only power up the device after all the resource table's > > -requirement are met. > > - > > -In addition to system resources, the resource table may also contain > > -resource entries that publish the existence of supported features > > -or configurations by the remote processor, such as trace buffers and > > -supported virtio devices (and their configurations). > > - > > -The resource table begins with this header:: > > - > > - /** > > - * struct resource_table - firmware resource table header > > - * @ver: version number > > - * @num: number of resource entries > > - * @reserved: reserved (must be zero) > > - * @offset: array of offsets pointing at the various resource entries > > - * > > - * The header of the resource table, as expressed by this structure, > > - * contains a version number (should we need to change this format in the > > - * future), the number of available resource entries, and their offsets > > - * in the table. > > - */ > > - struct resource_table { > > - u32 ver; > > - u32 num; > > - u32 reserved[2]; > > - u32 offset[0]; > > - } __packed; > > - > > -Immediately following this header are the resource entries themselves, > > -each of which begins with the following resource entry header:: > > - > > - /** > > - * struct fw_rsc_hdr - firmware resource entry header > > - * @type: resource type > > - * @data: resource data > > - * > > - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing > > - * its @type. The content of the entry itself will immediately follow > > - * this header, and it should be parsed according to the resource type. > > - */ > > - struct fw_rsc_hdr { > > - u32 type; > > - u8 data[0]; > > - } __packed; > > - > > -Some resources entries are mere announcements, where the host is informed > > -of specific remoteproc configuration. Other entries require the host to > > -do something (e.g. allocate a system resource). Sometimes a negotiation > > -is expected, where the firmware requests a resource, and once allocated, > > -the host should provide back its details (e.g. address of an allocated > > -memory region). > > - > > -Here are the various resource types that are currently supported:: > > - > > - /** > > - * enum fw_resource_type - types of resource entries > > - * > > - * @RSC_CARVEOUT: request for allocation of a physically contiguous > > - * memory region. > > - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. > > - * @RSC_TRACE: announces the availability of a trace buffer into which > > - * the remote processor will be writing logs. > > - * @RSC_VDEV: declare support for a virtio device, and serve as its > > - * virtio header. > > - * @RSC_LAST: just keep this one at the end > > - * @RSC_VENDOR_START: start of the vendor specific resource types range > > - * @RSC_VENDOR_END: end of the vendor specific resource types range > > - * > > - * Please note that these values are used as indices to the rproc_handle_rsc > > - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to > > - * check the validity of an index before the lookup table is accessed, so > > - * please update it as needed. > > - */ > > - enum fw_resource_type { > > - RSC_CARVEOUT = 0, > > - RSC_DEVMEM = 1, > > - RSC_TRACE = 2, > > - RSC_VDEV = 3, > > - RSC_LAST = 4, > > - RSC_VENDOR_START = 128, > > - RSC_VENDOR_END = 512, > > - }; > > - > > -For more details regarding a specific resource type, please see its > > -dedicated structure in include/linux/remoteproc.h. > > - > > -We also expect that platform-specific resource entries will show up > > -at some point. When that happens, we could easily add a new RSC_PLATFORM > > -type, and hand those resources to the platform-specific rproc driver to handle. > > - > > -Virtio and remoteproc > > -===================== > > - > > -The firmware should provide remoteproc information about virtio devices > > -that it supports, and their configurations: a RSC_VDEV resource entry > > -should specify the virtio device id (as in virtio_ids.h), virtio features, > > -virtio config space, vrings information, etc. > > - > > -When a new remote processor is registered, the remoteproc framework > > -will look for its resource table and will register the virtio devices > > -it supports. A firmware may support any number of virtio devices, and > > -of any type (a single remote processor can also easily support several > > -rpmsg virtio devices this way, if desired). > > - > > -Of course, RSC_VDEV resource entries are only good enough for static > > -allocation of virtio devices. Dynamic allocations will also be made possible > > -using the rpmsg bus (similar to how we already do dynamic allocations of > > -rpmsg channels; read more about it in rpmsg.txt). > > +The firmware must provide remoteproc with information regarding the > > +virtio devices it supports and their configurations: an `RSC_VDEV` > > +resource entry should detail the virtio device ID (as defined in > > +`virtio_ids.h`), virtio features, virtio config space, vrings > > +information, etc. > > + > > +Upon registration of a new remote, the remoteproc framework > > +searches for its resource table and registers the supported virtio > > +devices. A firmware may support multiple virtio devices, of various > > +types (a single remote can support multiple rpmsg virtio > > +devices if required). > > + > > +Moreover, `RSC_VDEV` resource entries suffice for static allocation > > +of virtio devices. Dynamic allocations will also be supported using > > +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg > > +channels. For more information, refer to rpmsg documentation. > > -- > > 2.39.3 (Apple Git-146) > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V4 3/7] Documentation: remoteproc: add more information 2024-10-24 18:17 ` anish kumar @ 2024-10-25 14:57 ` Mathieu Poirier 2024-10-25 15:13 ` anish kumar 0 siblings, 1 reply; 18+ messages in thread From: Mathieu Poirier @ 2024-10-25 14:57 UTC (permalink / raw) To: anish kumar Cc: ohad, bjorn.andersson, corbet, linux-remoteproc, linux-doc, linux-kernel On Thu, Oct 24, 2024 at 11:17:40AM -0700, anish kumar wrote: > On Thu, Oct 24, 2024 at 8:53 AM Mathieu Poirier > <mathieu.poirier@linaro.org> wrote: > > > > On Tue, Oct 22, 2024 at 10:33:53PM -0700, anish kumar wrote: > > > Added following changes: > > > 1. Components provided by remoteproc framework. > > > 2. Remoteproc driver responsibilities. > > > 3. Remoteproc framework responsibilities. > > > 4. Better explanation of how to ask for resources > > > from the framework by the remote processor. > > > > > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > > > --- > > > .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- > > > 1 file changed, 243 insertions(+), 353 deletions(-) > > > > > > diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst > > > index 9cccd3dd6a4b..7ca545eea153 100644 > > > --- a/Documentation/driver-api/remoteproc/remoteproc.rst > > > +++ b/Documentation/driver-api/remoteproc/remoteproc.rst > > > @@ -1,359 +1,249 @@ > > > +.. SPDX-License-Identifier: GPL-2.0 > > > + > > > ========================== > > > Remote Processor Framework > > > ========================== > > > > > > -Introduction > > > -============ > > > - > > > -Modern SoCs typically have heterogeneous remote processor devices in asymmetric > > > -multiprocessing (AMP) configurations, which may be running different instances > > > -of operating system, whether it's Linux or any other flavor of real-time OS. > > > - > > > -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. > > > -In a typical configuration, the dual cortex-A9 is running Linux in a SMP > > > -configuration, and each of the other three cores (two M3 cores and a DSP) > > > -is running its own instance of RTOS in an AMP configuration. > > > - > > > -The remoteproc framework allows different platforms/architectures to > > > -control (power on, load firmware, power off) those remote processors while > > > -abstracting the hardware differences, so the entire driver doesn't need to be > > > -duplicated. In addition, this framework also adds rpmsg virtio devices > > > -for remote processors that supports this kind of communication. This way, > > > -platform-specific remoteproc drivers only need to provide a few low-level > > > -handlers, and then all rpmsg drivers will then just work > > > -(for more information about the virtio-based rpmsg bus and its drivers, > > > -please read Documentation/staging/rpmsg.rst). > > > -Registration of other types of virtio devices is now also possible. Firmwares > > > -just need to publish what kind of virtio devices do they support, and then > > > -remoteproc will add those devices. This makes it possible to reuse the > > > -existing virtio drivers with remote processor backends at a minimal development > > > -cost. > > > - > > > -User API > > > -======== > > > - > > > -:: > > > - > > > - int rproc_boot(struct rproc *rproc) > > > - > > > -Boot a remote processor (i.e. load its firmware, power it on, ...). > > > - > > > -If the remote processor is already powered on, this function immediately > > > -returns (successfully). > > > - > > > -Returns 0 on success, and an appropriate error value otherwise. > > > -Note: to use this function you should already have a valid rproc > > > -handle. There are several ways to achieve that cleanly (devres, pdata, > > > -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we > > > -might also consider using dev_archdata for this). > > > - > > > -:: > > > - > > > - void rproc_shutdown(struct rproc *rproc) > > > - > > > -Power off a remote processor (previously booted with rproc_boot()). > > > -In case @rproc is still being used by an additional user(s), then > > > -this function will just decrement the power refcount and exit, > > > -without really powering off the device. > > > - > > > -Every call to rproc_boot() must (eventually) be accompanied by a call > > > -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. > > > - > > > -.. note:: > > > - > > > - we're not decrementing the rproc's refcount, only the power refcount. > > > - which means that the @rproc handle stays valid even after > > > - rproc_shutdown() returns, and users can still use it with a subsequent > > > - rproc_boot(), if needed. > > > - > > > -:: > > > - > > > - struct rproc *rproc_get_by_phandle(phandle phandle) > > > - > > > -Find an rproc handle using a device tree phandle. Returns the rproc > > > -handle on success, and NULL on failure. This function increments > > > -the remote processor's refcount, so always use rproc_put() to > > > -decrement it back once rproc isn't needed anymore. > > > - > > > -Typical usage > > > -============= > > > - > > > -:: > > > - > > > - #include <linux/remoteproc.h> > > > - > > > - /* in case we were given a valid 'rproc' handle */ > > > - int dummy_rproc_example(struct rproc *my_rproc) > > > - { > > > - int ret; > > > - > > > - /* let's power on and boot our remote processor */ > > > - ret = rproc_boot(my_rproc); > > > - if (ret) { > > > - /* > > > - * something went wrong. handle it and leave. > > > - */ > > > - } > > > - > > > - /* > > > - * our remote processor is now powered on... give it some work > > > - */ > > > - > > > - /* let's shut it down now */ > > > - rproc_shutdown(my_rproc); > > > - } > > > - > > > -API for implementors > > > -==================== > > > - > > > -:: > > > - > > > - struct rproc *rproc_alloc(struct device *dev, const char *name, > > > - const struct rproc_ops *ops, > > > - const char *firmware, int len) > > > - > > > -Allocate a new remote processor handle, but don't register > > > -it yet. Required parameters are the underlying device, the > > > -name of this remote processor, platform-specific ops handlers, > > > -the name of the firmware to boot this rproc with, and the > > > -length of private data needed by the allocating rproc driver (in bytes). > > > - > > > -This function should be used by rproc implementations during > > > -initialization of the remote processor. > > > - > > > -After creating an rproc handle using this function, and when ready, > > > -implementations should then call rproc_add() to complete > > > -the registration of the remote processor. > > > - > > > -On success, the new rproc is returned, and on failure, NULL. > > > - > > > -.. note:: > > > - > > > - **never** directly deallocate @rproc, even if it was not registered > > > - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). > > > - > > > -:: > > > - > > > - void rproc_free(struct rproc *rproc) > > > - > > > -Free an rproc handle that was allocated by rproc_alloc. > > > - > > > -This function essentially unrolls rproc_alloc(), by decrementing the > > > -rproc's refcount. It doesn't directly free rproc; that would happen > > > -only if there are no other references to rproc and its refcount now > > > -dropped to zero. > > > - > > > -:: > > > - > > > - int rproc_add(struct rproc *rproc) > > > - > > > -Register @rproc with the remoteproc framework, after it has been > > > -allocated with rproc_alloc(). > > > - > > > -This is called by the platform-specific rproc implementation, whenever > > > -a new remote processor device is probed. > > > - > > > -Returns 0 on success and an appropriate error code otherwise. > > > -Note: this function initiates an asynchronous firmware loading > > > -context, which will look for virtio devices supported by the rproc's > > > -firmware. > > > - > > > -If found, those virtio devices will be created and added, so as a result > > > -of registering this remote processor, additional virtio drivers might get > > > -probed. > > > - > > > -:: > > > - > > > - int rproc_del(struct rproc *rproc) > > > - > > > -Unroll rproc_add(). > > > - > > > -This function should be called when the platform specific rproc > > > -implementation decides to remove the rproc device. it should > > > -_only_ be called if a previous invocation of rproc_add() > > > -has completed successfully. > > > - > > > -After rproc_del() returns, @rproc is still valid, and its > > > -last refcount should be decremented by calling rproc_free(). > > > - > > > -Returns 0 on success and -EINVAL if @rproc isn't valid. > > > - > > > -:: > > > - > > > - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) > > > - > > > -Report a crash in a remoteproc > > > - > > > -This function must be called every time a crash is detected by the > > > -platform specific rproc implementation. This should not be called from a > > > -non-remoteproc driver. This function can be called from atomic/interrupt > > > -context. > > > - > > > -Implementation callbacks > > > +.. Contents: > > > + > > > + 1. Introduction > > > + 2. Remoteproc framework responsibilities > > > + 3. Remoteproc driver responsibilities > > > + 4. Virtio and rpmsg > > > + > > > +1. Introduction > > > +=============== > > > + > > > +Modern System on Chips (SoCs) typically integrate heterogeneous remote > > > +processor devices in asymmetric multiprocessing (AMP) configurations. > > > +These processors may run different operating systems, such as Linux and > > > +various real-time operating systems (RTOS). > > > > You are moving things around _and_ making modifications to the text in the same > > patch, something I specifically asked not to do. Moreover, the above conveys > > Sorry for doing that, I thought I moved the file first as you instructed but > I didn't understand that I should also not reword the sentences that convey > the same meaning. Will keep the text of below content in the same file: > 1. Introduction > 2. Binary Firmware Structure: will just add example as shown in this patch. > 3. Virtio and remoteproc: didn't change anything in the patch for this. We don't seem to understand each other - the "Virtio and remoteproc" section is the first thing I red and what made me aware of the rewording. > > For below: > 1. User API: > 2. Typical usage: > 3. API for implementors : > > Should I just move all of the above text as it is to > remoteproc-kernel-api.rst or > you prefer here as I feel it belongs there? I have looked at other subsystems documented under Documentation/driver-api and I can't find instances of the type [subsystem]-kernel-api.rst. As such I don't see what is so special about remoteproc that it would mandate adding a new file. Why can't it just be in a single file like most subsystems do? > > > exactly the same information as found in [1] but using different words. I > > am in favour of enhancing documentation but not creating unneeded churn. > > > > I found several instances of the same rewording pattern in the sections below. > > As such I will not look at the other patches nor move forward with this set. > > > > Thanks, > > Mathieu > > Thanks Mathieu, for reviewing and providing feedback as always. > > > > > [1]. https://elixir.bootlin.com/linux/v6.12-rc4/source/Documentation/staging/remoteproc.rst > > > > > + > > > +For example, the OMAP4 platform features dual Cortex-A9 cores, dual > > > +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 > > > +cores execute Linux in a symmetric multiprocessing (SMP) configuration, > > > +while the M3 cores and DSP run independent instances of an RTOS. > > > + > > > +The remoteproc framework allows various platforms and architectures to > > > +manage remote processors, including operations such as powering on, > > > +loading firmware, and powering off. This framework abstracts hardware > > > +differences, promoting code reuse and minimizing duplication. It also > > > +supports rpmsg virtio devices for remote processors that utilize this > > > +communication method. Consequently, platform-specific remoteproc drivers > > > +need only implement a few low-level handlers, enabling seamless operation > > > +of all rpmsg drivers. (For more details about the virtio-based rpmsg > > > +bus and its drivers, refer to rpmsg documentation.) > > > + > > > +Additionally, the framework allows for the registration of various > > > +virtio devices. Firmware can publish the types of virtio devices it > > > +supports, facilitating their addition to the remoteproc framework. This > > > +flexibility enables the reuse of existing virtio drivers with remote > > > +processor backends at minimal development cost. > > > + > > > +The primary purpose of the remoteproc framework is to download firmware > > > +for remote processors and manage their lifecycle. The framework consists > > > +of several key components: > > > + > > > +- **Character Driver**: Provides userspace access to control the remote > > > + processor. > > > +- **ELF Utility**: Offers functions for handling ELF files and managing > > > + resources requested by the remote processor. > > > +- **Remoteproc Core**: Manages firmware downloads and recovery actions > > > + in case of a remote processor crash. > > > +- **Coredump**: Provides facilities for coredumping and tracing from > > > + the remote processor in the event of a crash. > > > +- **Userspace Interaction**: Uses sysfs and debugfs to manage the > > > + lifecycle and status of the remote processor. > > > +- **Virtio Support**: Facilitates interaction with the virtio and > > > + rpmsg bus. > > > + > > > +From here on, references to "framework" denote the remoteproc > > > +framework, and "driver" refers to the remoteproc driver that utilizes > > > +the framework for managing remote processors. > > > + > > > +2. Remoteproc framework Responsibilities > > > +======================================== > > > + > > > +The framework begins by gathering information about the firmware file > > > +to be downloaded through the request_firmware function. It supports > > > +the ELF format and parses the firmware image to identify the physical > > > +addresses that need to be populated from the corresponding ELF sections. > > > +The framework also requires knowledge of the logical or I/O-mapped > > > +addresses in the application processor. Once this information is > > > +obtained from the driver, the framework transfers the data to the > > > +specified addresses and starts the remote, along with > > > +any devices physically or logically connected to it. > > > + > > > +Dependent devices, referred to as `subdevices` within the framework, > > > +are also managed post-registration by their respective drivers. > > > +Subdevices can register themselves using `rproc_(add/remove)_subdev`. > > > +Non-remoteproc drivers can use subdevices as a way to logically connect > > > +to remote and get lifecycle notifications of the remote. > > > + > > > +The framework oversees the lifecycle of the remote and > > > +provides the `rproc_report_crash` function, which the driver invokes > > > +upon receiving a crash notification from the remote. The > > > +notification method can differ based on the design of the remote > > > +processor and its communication with the application processor. For > > > +instance, if the remote is a DSP equipped with a watchdog, > > > +unresponsive behavior triggers the watchdog, generating an interrupt > > > +that routes to the application processor, allowing it to call > > > +`rproc_report_crash` in the driver's interrupt context. > > > + > > > +During crash handling, the framework performs the following actions: > > > + > > > +a. Sends a request to stop the remote and any connected or > > > + dependent subdevices. > > > +b. Generates a coredump, dumping all `resources` requested by the > > > + remote alongside relevant debugging information. Resources are > > > + explained below. > > > +c. Reloads the firmware and restarts the remote. > > > + > > > +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and > > > +attach callbacks of the driver are invoked without reloading the > > > +firmware. This is useful when the remote requires no > > > +assistance for recovery, or when the application processor can restart > > > +independently. After recovery, the application processor can reattach > > > +to the remote. > > > + > > > +The remote can request resources from the framework, which > > > +allocates a ".resource_table" section. During the ELF parsing phase, > > > +the framework identifies this section and calls the appropriate > > > +handler to allocate the requested resources. > > > + > > > +Resource management within the framework can accommodate any type of > > > +`fw_resource_type`. > > > + > > > +.. code-block:: c > > > + > > > + enum fw_resource_type { > > > + RSC_CARVEOUT = 0, > > > + RSC_DEVMEM = 1, > > > + RSC_TRACE = 2, > > > + RSC_VDEV = 3, > > > + RSC_LAST = 4, > > > + RSC_VENDOR_START = 128, > > > + RSC_VENDOR_END = 512, > > > + }; > > > + > > > + struct resource_table { > > > + u32 ver; > > > + u32 num; > > > + u32 reserved[2]; > > > + u32 offset[]; > > > + } __packed; > > > + > > > + struct fw_rsc_hdr { > > > + u32 type; > > > + u8 data[]; > > > + } __packed; > > > + > > > +For example, if the remote requests both `RSC_TRACE` and > > > +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured > > > +as follows: > > > + > > > +.. code-block:: c > > > + > > > + #define MAX_SHARED_RESOURCE 2 > > > + #define LOG_BUF_SIZE 1000 > > > + #define CARVEOUT_DUMP_PA 0x12345678 > > > + #define CARVEOUT_DUMP_SIZE 2000 > > > + > > > + struct shared_resource_table { > > > + u32 ver; > > > + u32 num; > > > + u32 reserved[2]; > > > + u32 offset[MAX_SHARED_RESOURCE]; > > > + struct fw_rsc_trace log_trace; > > > + struct fw_rsc_carveout dump_carveout; > > > + }; > > > + > > > + volatile struct shared_resource_table table = { > > > + .ver = 1, > > > + .num = 2, > > > + .reserved = {0, 0}, > > > + .offset = { > > > + offsetof(struct resource_table, log_trace), > > > + offsetof(struct resource_table, dump_carveout), > > > + }, > > > + .log_trace = { > > > + RSC_TRACE, > > > + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", > > > + }, > > > + .dump_carveout = { > > > + RSC_CARVEOUT, > > > + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", > > > + }, > > > + }; > > > + > > > +The framework creates a sysfs file when it encounters the `RSC_TRACE` > > > +type to expose log information to userspace. Other resource types are > > > +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes > > > +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. > > > + > > > + > > > +3. Remoteproc driver responsibilities > > > +===================================== > > > + > > > +The driver must provide the following information to the core: > > > + > > > +a. Translate device addresses (physical addresses) found in the ELF > > > + firmware to virtual addresses in Linux using the `da_to_va` > > > + callback. This allows the framework to copy ELF firmware from the > > > + filesystem to the addresses expected by the remote since > > > + the framework cannot directly access those physical addresses. > > > +b. Prepare/unprepare the remote prior to firmware loading, > > > + which may involve allocating carveout and reserved memory regions. > > > +c. Implement methods for starting and stopping the remote, > > > + whether by setting registers or sending explicit interrupts, > > > + depending on the hardware design. > > > +d. Provide attach and detach callbacks to start the remote > > > + without loading the firmware. This is beneficial when the remote > > > + processor is already loaded and running. > > > +e. Implement a load callback for firmware loading, typically using > > > + the ELF loader provided by the framework; currently, only ELF > > > + format is supported. > > > +f. Invoke the framework's crash handler API upon detecting a remote > > > + crash. > > > + > > > +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` > > > +to register themselves with the framework. > > > + > > > +.. code-block:: c > > > + > > > + struct rproc_ops { > > > + int (*prepare)(struct rproc *rproc); > > > + int (*unprepare)(struct rproc *rproc); > > > + int (*start)(struct rproc *rproc); > > > + int (*stop)(struct rproc *rproc); > > > + int (*attach)(struct rproc *rproc); > > > + int (*detach)(struct rproc *rproc); > > > + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, > > > + bool *is_iomem); > > > + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); > > > + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, > > > + void *rsc, int offset, int avail); > > > + int (*load)(struct rproc *rproc, const struct firmware *fw); > > > + //snip > > > + }; > > > + > > > + > > > +4. Virtio and Remoteproc > > > ======================== > > > > > > -These callbacks should be provided by platform-specific remoteproc > > > -drivers:: > > > - > > > - /** > > > - * struct rproc_ops - platform-specific device handlers > > > - * @start: power on the device and boot it > > > - * @stop: power off the device > > > - * @kick: kick a virtqueue (virtqueue id given as a parameter) > > > - */ > > > - struct rproc_ops { > > > - int (*start)(struct rproc *rproc); > > > - int (*stop)(struct rproc *rproc); > > > - void (*kick)(struct rproc *rproc, int vqid); > > > - }; > > > - > > > -Every remoteproc implementation should at least provide the ->start and ->stop > > > -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler > > > -should be provided as well. > > > - > > > -The ->start() handler takes an rproc handle and should then power on the > > > -device and boot it (use rproc->priv to access platform-specific private data). > > > -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc > > > -core puts there the ELF entry point). > > > -On success, 0 should be returned, and on failure, an appropriate error code. > > > - > > > -The ->stop() handler takes an rproc handle and powers the device down. > > > -On success, 0 is returned, and on failure, an appropriate error code. > > > - > > > -The ->kick() handler takes an rproc handle, and an index of a virtqueue > > > -where new message was placed in. Implementations should interrupt the remote > > > -processor and let it know it has pending messages. Notifying remote processors > > > -the exact virtqueue index to look in is optional: it is easy (and not > > > -too expensive) to go through the existing virtqueues and look for new buffers > > > -in the used rings. > > > - > > > -Binary Firmware Structure > > > -========================= > > > - > > > -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, > > > -it is quite expected that other platforms/devices which we'd want to > > > -support with this framework will be based on different binary formats. > > > - > > > -When those use cases show up, we will have to decouple the binary format > > > -from the framework core, so we can support several binary formats without > > > -duplicating common code. > > > - > > > -When the firmware is parsed, its various segments are loaded to memory > > > -according to the specified device address (might be a physical address > > > -if the remote processor is accessing memory directly). > > > - > > > -In addition to the standard ELF segments, most remote processors would > > > -also include a special section which we call "the resource table". > > > - > > > -The resource table contains system resources that the remote processor > > > -requires before it should be powered on, such as allocation of physically > > > -contiguous memory, or iommu mapping of certain on-chip peripherals. > > > -Remotecore will only power up the device after all the resource table's > > > -requirement are met. > > > - > > > -In addition to system resources, the resource table may also contain > > > -resource entries that publish the existence of supported features > > > -or configurations by the remote processor, such as trace buffers and > > > -supported virtio devices (and their configurations). > > > - > > > -The resource table begins with this header:: > > > - > > > - /** > > > - * struct resource_table - firmware resource table header > > > - * @ver: version number > > > - * @num: number of resource entries > > > - * @reserved: reserved (must be zero) > > > - * @offset: array of offsets pointing at the various resource entries > > > - * > > > - * The header of the resource table, as expressed by this structure, > > > - * contains a version number (should we need to change this format in the > > > - * future), the number of available resource entries, and their offsets > > > - * in the table. > > > - */ > > > - struct resource_table { > > > - u32 ver; > > > - u32 num; > > > - u32 reserved[2]; > > > - u32 offset[0]; > > > - } __packed; > > > - > > > -Immediately following this header are the resource entries themselves, > > > -each of which begins with the following resource entry header:: > > > - > > > - /** > > > - * struct fw_rsc_hdr - firmware resource entry header > > > - * @type: resource type > > > - * @data: resource data > > > - * > > > - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing > > > - * its @type. The content of the entry itself will immediately follow > > > - * this header, and it should be parsed according to the resource type. > > > - */ > > > - struct fw_rsc_hdr { > > > - u32 type; > > > - u8 data[0]; > > > - } __packed; > > > - > > > -Some resources entries are mere announcements, where the host is informed > > > -of specific remoteproc configuration. Other entries require the host to > > > -do something (e.g. allocate a system resource). Sometimes a negotiation > > > -is expected, where the firmware requests a resource, and once allocated, > > > -the host should provide back its details (e.g. address of an allocated > > > -memory region). > > > - > > > -Here are the various resource types that are currently supported:: > > > - > > > - /** > > > - * enum fw_resource_type - types of resource entries > > > - * > > > - * @RSC_CARVEOUT: request for allocation of a physically contiguous > > > - * memory region. > > > - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. > > > - * @RSC_TRACE: announces the availability of a trace buffer into which > > > - * the remote processor will be writing logs. > > > - * @RSC_VDEV: declare support for a virtio device, and serve as its > > > - * virtio header. > > > - * @RSC_LAST: just keep this one at the end > > > - * @RSC_VENDOR_START: start of the vendor specific resource types range > > > - * @RSC_VENDOR_END: end of the vendor specific resource types range > > > - * > > > - * Please note that these values are used as indices to the rproc_handle_rsc > > > - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to > > > - * check the validity of an index before the lookup table is accessed, so > > > - * please update it as needed. > > > - */ > > > - enum fw_resource_type { > > > - RSC_CARVEOUT = 0, > > > - RSC_DEVMEM = 1, > > > - RSC_TRACE = 2, > > > - RSC_VDEV = 3, > > > - RSC_LAST = 4, > > > - RSC_VENDOR_START = 128, > > > - RSC_VENDOR_END = 512, > > > - }; > > > - > > > -For more details regarding a specific resource type, please see its > > > -dedicated structure in include/linux/remoteproc.h. > > > - > > > -We also expect that platform-specific resource entries will show up > > > -at some point. When that happens, we could easily add a new RSC_PLATFORM > > > -type, and hand those resources to the platform-specific rproc driver to handle. > > > - > > > -Virtio and remoteproc > > > -===================== > > > - > > > -The firmware should provide remoteproc information about virtio devices > > > -that it supports, and their configurations: a RSC_VDEV resource entry > > > -should specify the virtio device id (as in virtio_ids.h), virtio features, > > > -virtio config space, vrings information, etc. > > > - > > > -When a new remote processor is registered, the remoteproc framework > > > -will look for its resource table and will register the virtio devices > > > -it supports. A firmware may support any number of virtio devices, and > > > -of any type (a single remote processor can also easily support several > > > -rpmsg virtio devices this way, if desired). > > > - > > > -Of course, RSC_VDEV resource entries are only good enough for static > > > -allocation of virtio devices. Dynamic allocations will also be made possible > > > -using the rpmsg bus (similar to how we already do dynamic allocations of > > > -rpmsg channels; read more about it in rpmsg.txt). > > > +The firmware must provide remoteproc with information regarding the > > > +virtio devices it supports and their configurations: an `RSC_VDEV` > > > +resource entry should detail the virtio device ID (as defined in > > > +`virtio_ids.h`), virtio features, virtio config space, vrings > > > +information, etc. > > > + > > > +Upon registration of a new remote, the remoteproc framework > > > +searches for its resource table and registers the supported virtio > > > +devices. A firmware may support multiple virtio devices, of various > > > +types (a single remote can support multiple rpmsg virtio > > > +devices if required). > > > + > > > +Moreover, `RSC_VDEV` resource entries suffice for static allocation > > > +of virtio devices. Dynamic allocations will also be supported using > > > +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg > > > +channels. For more information, refer to rpmsg documentation. > > > -- > > > 2.39.3 (Apple Git-146) > > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V4 3/7] Documentation: remoteproc: add more information 2024-10-25 14:57 ` Mathieu Poirier @ 2024-10-25 15:13 ` anish kumar 2024-10-25 15:18 ` Mathieu Poirier 0 siblings, 1 reply; 18+ messages in thread From: anish kumar @ 2024-10-25 15:13 UTC (permalink / raw) To: Mathieu Poirier Cc: ohad, bjorn.andersson, corbet, linux-remoteproc, linux-doc, linux-kernel On Fri, Oct 25, 2024 at 7:57 AM Mathieu Poirier <mathieu.poirier@linaro.org> wrote: > > On Thu, Oct 24, 2024 at 11:17:40AM -0700, anish kumar wrote: > > On Thu, Oct 24, 2024 at 8:53 AM Mathieu Poirier > > <mathieu.poirier@linaro.org> wrote: > > > > > > On Tue, Oct 22, 2024 at 10:33:53PM -0700, anish kumar wrote: > > > > Added following changes: > > > > 1. Components provided by remoteproc framework. > > > > 2. Remoteproc driver responsibilities. > > > > 3. Remoteproc framework responsibilities. > > > > 4. Better explanation of how to ask for resources > > > > from the framework by the remote processor. > > > > > > > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > > > > --- > > > > .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- > > > > 1 file changed, 243 insertions(+), 353 deletions(-) > > > > > > > > diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst > > > > index 9cccd3dd6a4b..7ca545eea153 100644 > > > > --- a/Documentation/driver-api/remoteproc/remoteproc.rst > > > > +++ b/Documentation/driver-api/remoteproc/remoteproc.rst > > > > @@ -1,359 +1,249 @@ > > > > +.. SPDX-License-Identifier: GPL-2.0 > > > > + > > > > ========================== > > > > Remote Processor Framework > > > > ========================== > > > > > > > > -Introduction > > > > -============ > > > > - > > > > -Modern SoCs typically have heterogeneous remote processor devices in asymmetric > > > > -multiprocessing (AMP) configurations, which may be running different instances > > > > -of operating system, whether it's Linux or any other flavor of real-time OS. > > > > - > > > > -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. > > > > -In a typical configuration, the dual cortex-A9 is running Linux in a SMP > > > > -configuration, and each of the other three cores (two M3 cores and a DSP) > > > > -is running its own instance of RTOS in an AMP configuration. > > > > - > > > > -The remoteproc framework allows different platforms/architectures to > > > > -control (power on, load firmware, power off) those remote processors while > > > > -abstracting the hardware differences, so the entire driver doesn't need to be > > > > -duplicated. In addition, this framework also adds rpmsg virtio devices > > > > -for remote processors that supports this kind of communication. This way, > > > > -platform-specific remoteproc drivers only need to provide a few low-level > > > > -handlers, and then all rpmsg drivers will then just work > > > > -(for more information about the virtio-based rpmsg bus and its drivers, > > > > -please read Documentation/staging/rpmsg.rst). > > > > -Registration of other types of virtio devices is now also possible. Firmwares > > > > -just need to publish what kind of virtio devices do they support, and then > > > > -remoteproc will add those devices. This makes it possible to reuse the > > > > -existing virtio drivers with remote processor backends at a minimal development > > > > -cost. > > > > - > > > > -User API > > > > -======== > > > > - > > > > -:: > > > > - > > > > - int rproc_boot(struct rproc *rproc) > > > > - > > > > -Boot a remote processor (i.e. load its firmware, power it on, ...). > > > > - > > > > -If the remote processor is already powered on, this function immediately > > > > -returns (successfully). > > > > - > > > > -Returns 0 on success, and an appropriate error value otherwise. > > > > -Note: to use this function you should already have a valid rproc > > > > -handle. There are several ways to achieve that cleanly (devres, pdata, > > > > -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we > > > > -might also consider using dev_archdata for this). > > > > - > > > > -:: > > > > - > > > > - void rproc_shutdown(struct rproc *rproc) > > > > - > > > > -Power off a remote processor (previously booted with rproc_boot()). > > > > -In case @rproc is still being used by an additional user(s), then > > > > -this function will just decrement the power refcount and exit, > > > > -without really powering off the device. > > > > - > > > > -Every call to rproc_boot() must (eventually) be accompanied by a call > > > > -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. > > > > - > > > > -.. note:: > > > > - > > > > - we're not decrementing the rproc's refcount, only the power refcount. > > > > - which means that the @rproc handle stays valid even after > > > > - rproc_shutdown() returns, and users can still use it with a subsequent > > > > - rproc_boot(), if needed. > > > > - > > > > -:: > > > > - > > > > - struct rproc *rproc_get_by_phandle(phandle phandle) > > > > - > > > > -Find an rproc handle using a device tree phandle. Returns the rproc > > > > -handle on success, and NULL on failure. This function increments > > > > -the remote processor's refcount, so always use rproc_put() to > > > > -decrement it back once rproc isn't needed anymore. > > > > - > > > > -Typical usage > > > > -============= > > > > - > > > > -:: > > > > - > > > > - #include <linux/remoteproc.h> > > > > - > > > > - /* in case we were given a valid 'rproc' handle */ > > > > - int dummy_rproc_example(struct rproc *my_rproc) > > > > - { > > > > - int ret; > > > > - > > > > - /* let's power on and boot our remote processor */ > > > > - ret = rproc_boot(my_rproc); > > > > - if (ret) { > > > > - /* > > > > - * something went wrong. handle it and leave. > > > > - */ > > > > - } > > > > - > > > > - /* > > > > - * our remote processor is now powered on... give it some work > > > > - */ > > > > - > > > > - /* let's shut it down now */ > > > > - rproc_shutdown(my_rproc); > > > > - } > > > > - > > > > -API for implementors > > > > -==================== > > > > - > > > > -:: > > > > - > > > > - struct rproc *rproc_alloc(struct device *dev, const char *name, > > > > - const struct rproc_ops *ops, > > > > - const char *firmware, int len) > > > > - > > > > -Allocate a new remote processor handle, but don't register > > > > -it yet. Required parameters are the underlying device, the > > > > -name of this remote processor, platform-specific ops handlers, > > > > -the name of the firmware to boot this rproc with, and the > > > > -length of private data needed by the allocating rproc driver (in bytes). > > > > - > > > > -This function should be used by rproc implementations during > > > > -initialization of the remote processor. > > > > - > > > > -After creating an rproc handle using this function, and when ready, > > > > -implementations should then call rproc_add() to complete > > > > -the registration of the remote processor. > > > > - > > > > -On success, the new rproc is returned, and on failure, NULL. > > > > - > > > > -.. note:: > > > > - > > > > - **never** directly deallocate @rproc, even if it was not registered > > > > - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). > > > > - > > > > -:: > > > > - > > > > - void rproc_free(struct rproc *rproc) > > > > - > > > > -Free an rproc handle that was allocated by rproc_alloc. > > > > - > > > > -This function essentially unrolls rproc_alloc(), by decrementing the > > > > -rproc's refcount. It doesn't directly free rproc; that would happen > > > > -only if there are no other references to rproc and its refcount now > > > > -dropped to zero. > > > > - > > > > -:: > > > > - > > > > - int rproc_add(struct rproc *rproc) > > > > - > > > > -Register @rproc with the remoteproc framework, after it has been > > > > -allocated with rproc_alloc(). > > > > - > > > > -This is called by the platform-specific rproc implementation, whenever > > > > -a new remote processor device is probed. > > > > - > > > > -Returns 0 on success and an appropriate error code otherwise. > > > > -Note: this function initiates an asynchronous firmware loading > > > > -context, which will look for virtio devices supported by the rproc's > > > > -firmware. > > > > - > > > > -If found, those virtio devices will be created and added, so as a result > > > > -of registering this remote processor, additional virtio drivers might get > > > > -probed. > > > > - > > > > -:: > > > > - > > > > - int rproc_del(struct rproc *rproc) > > > > - > > > > -Unroll rproc_add(). > > > > - > > > > -This function should be called when the platform specific rproc > > > > -implementation decides to remove the rproc device. it should > > > > -_only_ be called if a previous invocation of rproc_add() > > > > -has completed successfully. > > > > - > > > > -After rproc_del() returns, @rproc is still valid, and its > > > > -last refcount should be decremented by calling rproc_free(). > > > > - > > > > -Returns 0 on success and -EINVAL if @rproc isn't valid. > > > > - > > > > -:: > > > > - > > > > - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) > > > > - > > > > -Report a crash in a remoteproc > > > > - > > > > -This function must be called every time a crash is detected by the > > > > -platform specific rproc implementation. This should not be called from a > > > > -non-remoteproc driver. This function can be called from atomic/interrupt > > > > -context. > > > > - > > > > -Implementation callbacks > > > > +.. Contents: > > > > + > > > > + 1. Introduction > > > > + 2. Remoteproc framework responsibilities > > > > + 3. Remoteproc driver responsibilities > > > > + 4. Virtio and rpmsg > > > > + > > > > +1. Introduction > > > > +=============== > > > > + > > > > +Modern System on Chips (SoCs) typically integrate heterogeneous remote > > > > +processor devices in asymmetric multiprocessing (AMP) configurations. > > > > +These processors may run different operating systems, such as Linux and > > > > +various real-time operating systems (RTOS). > > > > > > You are moving things around _and_ making modifications to the text in the same > > > patch, something I specifically asked not to do. Moreover, the above conveys > > > > Sorry for doing that, I thought I moved the file first as you instructed but > > I didn't understand that I should also not reword the sentences that convey > > the same meaning. Will keep the text of below content in the same file: > > 1. Introduction > > 2. Binary Firmware Structure: will just add example as shown in this patch. > > 3. Virtio and remoteproc: didn't change anything in the patch for this. > > We don't seem to understand each other - the "Virtio and remoteproc" section is > the first thing I red and what made me aware of the rewording. I double checked and found you are right, I initially had changed it but had no intention of changing in following patches but somehow it creeped in. Will keep everything as before. > > > > > For below: > > 1. User API: > > 2. Typical usage: > > 3. API for implementors : > > > > Should I just move all of the above text as it is to > > remoteproc-kernel-api.rst or > > you prefer here as I feel it belongs there? > > I have looked at other subsystems documented under Documentation/driver-api and > I can't find instances of the type [subsystem]-kernel-api.rst. As such I don't > see what is so special about remoteproc that it would mandate adding a new file. > Why can't it just be in a single file like most subsystems do? Certainly, I can do that. Let me keep only the main file which was the case, just add the relevant updates to the sections that needs updates. Will that be agreeable to you? > > > > > > exactly the same information as found in [1] but using different words. I > > > am in favour of enhancing documentation but not creating unneeded churn. > > > > > > I found several instances of the same rewording pattern in the sections below. > > > As such I will not look at the other patches nor move forward with this set. > > > > > > Thanks, > > > Mathieu > > > > Thanks Mathieu, for reviewing and providing feedback as always. > > > > > > > > [1]. https://elixir.bootlin.com/linux/v6.12-rc4/source/Documentation/staging/remoteproc.rst > > > > > > > + > > > > +For example, the OMAP4 platform features dual Cortex-A9 cores, dual > > > > +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 > > > > +cores execute Linux in a symmetric multiprocessing (SMP) configuration, > > > > +while the M3 cores and DSP run independent instances of an RTOS. > > > > + > > > > +The remoteproc framework allows various platforms and architectures to > > > > +manage remote processors, including operations such as powering on, > > > > +loading firmware, and powering off. This framework abstracts hardware > > > > +differences, promoting code reuse and minimizing duplication. It also > > > > +supports rpmsg virtio devices for remote processors that utilize this > > > > +communication method. Consequently, platform-specific remoteproc drivers > > > > +need only implement a few low-level handlers, enabling seamless operation > > > > +of all rpmsg drivers. (For more details about the virtio-based rpmsg > > > > +bus and its drivers, refer to rpmsg documentation.) > > > > + > > > > +Additionally, the framework allows for the registration of various > > > > +virtio devices. Firmware can publish the types of virtio devices it > > > > +supports, facilitating their addition to the remoteproc framework. This > > > > +flexibility enables the reuse of existing virtio drivers with remote > > > > +processor backends at minimal development cost. > > > > + > > > > +The primary purpose of the remoteproc framework is to download firmware > > > > +for remote processors and manage their lifecycle. The framework consists > > > > +of several key components: > > > > + > > > > +- **Character Driver**: Provides userspace access to control the remote > > > > + processor. > > > > +- **ELF Utility**: Offers functions for handling ELF files and managing > > > > + resources requested by the remote processor. > > > > +- **Remoteproc Core**: Manages firmware downloads and recovery actions > > > > + in case of a remote processor crash. > > > > +- **Coredump**: Provides facilities for coredumping and tracing from > > > > + the remote processor in the event of a crash. > > > > +- **Userspace Interaction**: Uses sysfs and debugfs to manage the > > > > + lifecycle and status of the remote processor. > > > > +- **Virtio Support**: Facilitates interaction with the virtio and > > > > + rpmsg bus. > > > > + > > > > +From here on, references to "framework" denote the remoteproc > > > > +framework, and "driver" refers to the remoteproc driver that utilizes > > > > +the framework for managing remote processors. > > > > + > > > > +2. Remoteproc framework Responsibilities > > > > +======================================== > > > > + > > > > +The framework begins by gathering information about the firmware file > > > > +to be downloaded through the request_firmware function. It supports > > > > +the ELF format and parses the firmware image to identify the physical > > > > +addresses that need to be populated from the corresponding ELF sections. > > > > +The framework also requires knowledge of the logical or I/O-mapped > > > > +addresses in the application processor. Once this information is > > > > +obtained from the driver, the framework transfers the data to the > > > > +specified addresses and starts the remote, along with > > > > +any devices physically or logically connected to it. > > > > + > > > > +Dependent devices, referred to as `subdevices` within the framework, > > > > +are also managed post-registration by their respective drivers. > > > > +Subdevices can register themselves using `rproc_(add/remove)_subdev`. > > > > +Non-remoteproc drivers can use subdevices as a way to logically connect > > > > +to remote and get lifecycle notifications of the remote. > > > > + > > > > +The framework oversees the lifecycle of the remote and > > > > +provides the `rproc_report_crash` function, which the driver invokes > > > > +upon receiving a crash notification from the remote. The > > > > +notification method can differ based on the design of the remote > > > > +processor and its communication with the application processor. For > > > > +instance, if the remote is a DSP equipped with a watchdog, > > > > +unresponsive behavior triggers the watchdog, generating an interrupt > > > > +that routes to the application processor, allowing it to call > > > > +`rproc_report_crash` in the driver's interrupt context. > > > > + > > > > +During crash handling, the framework performs the following actions: > > > > + > > > > +a. Sends a request to stop the remote and any connected or > > > > + dependent subdevices. > > > > +b. Generates a coredump, dumping all `resources` requested by the > > > > + remote alongside relevant debugging information. Resources are > > > > + explained below. > > > > +c. Reloads the firmware and restarts the remote. > > > > + > > > > +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and > > > > +attach callbacks of the driver are invoked without reloading the > > > > +firmware. This is useful when the remote requires no > > > > +assistance for recovery, or when the application processor can restart > > > > +independently. After recovery, the application processor can reattach > > > > +to the remote. > > > > + > > > > +The remote can request resources from the framework, which > > > > +allocates a ".resource_table" section. During the ELF parsing phase, > > > > +the framework identifies this section and calls the appropriate > > > > +handler to allocate the requested resources. > > > > + > > > > +Resource management within the framework can accommodate any type of > > > > +`fw_resource_type`. > > > > + > > > > +.. code-block:: c > > > > + > > > > + enum fw_resource_type { > > > > + RSC_CARVEOUT = 0, > > > > + RSC_DEVMEM = 1, > > > > + RSC_TRACE = 2, > > > > + RSC_VDEV = 3, > > > > + RSC_LAST = 4, > > > > + RSC_VENDOR_START = 128, > > > > + RSC_VENDOR_END = 512, > > > > + }; > > > > + > > > > + struct resource_table { > > > > + u32 ver; > > > > + u32 num; > > > > + u32 reserved[2]; > > > > + u32 offset[]; > > > > + } __packed; > > > > + > > > > + struct fw_rsc_hdr { > > > > + u32 type; > > > > + u8 data[]; > > > > + } __packed; > > > > + > > > > +For example, if the remote requests both `RSC_TRACE` and > > > > +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured > > > > +as follows: > > > > + > > > > +.. code-block:: c > > > > + > > > > + #define MAX_SHARED_RESOURCE 2 > > > > + #define LOG_BUF_SIZE 1000 > > > > + #define CARVEOUT_DUMP_PA 0x12345678 > > > > + #define CARVEOUT_DUMP_SIZE 2000 > > > > + > > > > + struct shared_resource_table { > > > > + u32 ver; > > > > + u32 num; > > > > + u32 reserved[2]; > > > > + u32 offset[MAX_SHARED_RESOURCE]; > > > > + struct fw_rsc_trace log_trace; > > > > + struct fw_rsc_carveout dump_carveout; > > > > + }; > > > > + > > > > + volatile struct shared_resource_table table = { > > > > + .ver = 1, > > > > + .num = 2, > > > > + .reserved = {0, 0}, > > > > + .offset = { > > > > + offsetof(struct resource_table, log_trace), > > > > + offsetof(struct resource_table, dump_carveout), > > > > + }, > > > > + .log_trace = { > > > > + RSC_TRACE, > > > > + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", > > > > + }, > > > > + .dump_carveout = { > > > > + RSC_CARVEOUT, > > > > + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", > > > > + }, > > > > + }; > > > > + > > > > +The framework creates a sysfs file when it encounters the `RSC_TRACE` > > > > +type to expose log information to userspace. Other resource types are > > > > +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes > > > > +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. > > > > + > > > > + > > > > +3. Remoteproc driver responsibilities > > > > +===================================== > > > > + > > > > +The driver must provide the following information to the core: > > > > + > > > > +a. Translate device addresses (physical addresses) found in the ELF > > > > + firmware to virtual addresses in Linux using the `da_to_va` > > > > + callback. This allows the framework to copy ELF firmware from the > > > > + filesystem to the addresses expected by the remote since > > > > + the framework cannot directly access those physical addresses. > > > > +b. Prepare/unprepare the remote prior to firmware loading, > > > > + which may involve allocating carveout and reserved memory regions. > > > > +c. Implement methods for starting and stopping the remote, > > > > + whether by setting registers or sending explicit interrupts, > > > > + depending on the hardware design. > > > > +d. Provide attach and detach callbacks to start the remote > > > > + without loading the firmware. This is beneficial when the remote > > > > + processor is already loaded and running. > > > > +e. Implement a load callback for firmware loading, typically using > > > > + the ELF loader provided by the framework; currently, only ELF > > > > + format is supported. > > > > +f. Invoke the framework's crash handler API upon detecting a remote > > > > + crash. > > > > + > > > > +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` > > > > +to register themselves with the framework. > > > > + > > > > +.. code-block:: c > > > > + > > > > + struct rproc_ops { > > > > + int (*prepare)(struct rproc *rproc); > > > > + int (*unprepare)(struct rproc *rproc); > > > > + int (*start)(struct rproc *rproc); > > > > + int (*stop)(struct rproc *rproc); > > > > + int (*attach)(struct rproc *rproc); > > > > + int (*detach)(struct rproc *rproc); > > > > + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, > > > > + bool *is_iomem); > > > > + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); > > > > + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, > > > > + void *rsc, int offset, int avail); > > > > + int (*load)(struct rproc *rproc, const struct firmware *fw); > > > > + //snip > > > > + }; > > > > + > > > > + > > > > +4. Virtio and Remoteproc > > > > ======================== > > > > > > > > -These callbacks should be provided by platform-specific remoteproc > > > > -drivers:: > > > > - > > > > - /** > > > > - * struct rproc_ops - platform-specific device handlers > > > > - * @start: power on the device and boot it > > > > - * @stop: power off the device > > > > - * @kick: kick a virtqueue (virtqueue id given as a parameter) > > > > - */ > > > > - struct rproc_ops { > > > > - int (*start)(struct rproc *rproc); > > > > - int (*stop)(struct rproc *rproc); > > > > - void (*kick)(struct rproc *rproc, int vqid); > > > > - }; > > > > - > > > > -Every remoteproc implementation should at least provide the ->start and ->stop > > > > -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler > > > > -should be provided as well. > > > > - > > > > -The ->start() handler takes an rproc handle and should then power on the > > > > -device and boot it (use rproc->priv to access platform-specific private data). > > > > -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc > > > > -core puts there the ELF entry point). > > > > -On success, 0 should be returned, and on failure, an appropriate error code. > > > > - > > > > -The ->stop() handler takes an rproc handle and powers the device down. > > > > -On success, 0 is returned, and on failure, an appropriate error code. > > > > - > > > > -The ->kick() handler takes an rproc handle, and an index of a virtqueue > > > > -where new message was placed in. Implementations should interrupt the remote > > > > -processor and let it know it has pending messages. Notifying remote processors > > > > -the exact virtqueue index to look in is optional: it is easy (and not > > > > -too expensive) to go through the existing virtqueues and look for new buffers > > > > -in the used rings. > > > > - > > > > -Binary Firmware Structure > > > > -========================= > > > > - > > > > -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, > > > > -it is quite expected that other platforms/devices which we'd want to > > > > -support with this framework will be based on different binary formats. > > > > - > > > > -When those use cases show up, we will have to decouple the binary format > > > > -from the framework core, so we can support several binary formats without > > > > -duplicating common code. > > > > - > > > > -When the firmware is parsed, its various segments are loaded to memory > > > > -according to the specified device address (might be a physical address > > > > -if the remote processor is accessing memory directly). > > > > - > > > > -In addition to the standard ELF segments, most remote processors would > > > > -also include a special section which we call "the resource table". > > > > - > > > > -The resource table contains system resources that the remote processor > > > > -requires before it should be powered on, such as allocation of physically > > > > -contiguous memory, or iommu mapping of certain on-chip peripherals. > > > > -Remotecore will only power up the device after all the resource table's > > > > -requirement are met. > > > > - > > > > -In addition to system resources, the resource table may also contain > > > > -resource entries that publish the existence of supported features > > > > -or configurations by the remote processor, such as trace buffers and > > > > -supported virtio devices (and their configurations). > > > > - > > > > -The resource table begins with this header:: > > > > - > > > > - /** > > > > - * struct resource_table - firmware resource table header > > > > - * @ver: version number > > > > - * @num: number of resource entries > > > > - * @reserved: reserved (must be zero) > > > > - * @offset: array of offsets pointing at the various resource entries > > > > - * > > > > - * The header of the resource table, as expressed by this structure, > > > > - * contains a version number (should we need to change this format in the > > > > - * future), the number of available resource entries, and their offsets > > > > - * in the table. > > > > - */ > > > > - struct resource_table { > > > > - u32 ver; > > > > - u32 num; > > > > - u32 reserved[2]; > > > > - u32 offset[0]; > > > > - } __packed; > > > > - > > > > -Immediately following this header are the resource entries themselves, > > > > -each of which begins with the following resource entry header:: > > > > - > > > > - /** > > > > - * struct fw_rsc_hdr - firmware resource entry header > > > > - * @type: resource type > > > > - * @data: resource data > > > > - * > > > > - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing > > > > - * its @type. The content of the entry itself will immediately follow > > > > - * this header, and it should be parsed according to the resource type. > > > > - */ > > > > - struct fw_rsc_hdr { > > > > - u32 type; > > > > - u8 data[0]; > > > > - } __packed; > > > > - > > > > -Some resources entries are mere announcements, where the host is informed > > > > -of specific remoteproc configuration. Other entries require the host to > > > > -do something (e.g. allocate a system resource). Sometimes a negotiation > > > > -is expected, where the firmware requests a resource, and once allocated, > > > > -the host should provide back its details (e.g. address of an allocated > > > > -memory region). > > > > - > > > > -Here are the various resource types that are currently supported:: > > > > - > > > > - /** > > > > - * enum fw_resource_type - types of resource entries > > > > - * > > > > - * @RSC_CARVEOUT: request for allocation of a physically contiguous > > > > - * memory region. > > > > - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. > > > > - * @RSC_TRACE: announces the availability of a trace buffer into which > > > > - * the remote processor will be writing logs. > > > > - * @RSC_VDEV: declare support for a virtio device, and serve as its > > > > - * virtio header. > > > > - * @RSC_LAST: just keep this one at the end > > > > - * @RSC_VENDOR_START: start of the vendor specific resource types range > > > > - * @RSC_VENDOR_END: end of the vendor specific resource types range > > > > - * > > > > - * Please note that these values are used as indices to the rproc_handle_rsc > > > > - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to > > > > - * check the validity of an index before the lookup table is accessed, so > > > > - * please update it as needed. > > > > - */ > > > > - enum fw_resource_type { > > > > - RSC_CARVEOUT = 0, > > > > - RSC_DEVMEM = 1, > > > > - RSC_TRACE = 2, > > > > - RSC_VDEV = 3, > > > > - RSC_LAST = 4, > > > > - RSC_VENDOR_START = 128, > > > > - RSC_VENDOR_END = 512, > > > > - }; > > > > - > > > > -For more details regarding a specific resource type, please see its > > > > -dedicated structure in include/linux/remoteproc.h. > > > > - > > > > -We also expect that platform-specific resource entries will show up > > > > -at some point. When that happens, we could easily add a new RSC_PLATFORM > > > > -type, and hand those resources to the platform-specific rproc driver to handle. > > > > - > > > > -Virtio and remoteproc > > > > -===================== > > > > - > > > > -The firmware should provide remoteproc information about virtio devices > > > > -that it supports, and their configurations: a RSC_VDEV resource entry > > > > -should specify the virtio device id (as in virtio_ids.h), virtio features, > > > > -virtio config space, vrings information, etc. > > > > - > > > > -When a new remote processor is registered, the remoteproc framework > > > > -will look for its resource table and will register the virtio devices > > > > -it supports. A firmware may support any number of virtio devices, and > > > > -of any type (a single remote processor can also easily support several > > > > -rpmsg virtio devices this way, if desired). > > > > - > > > > -Of course, RSC_VDEV resource entries are only good enough for static > > > > -allocation of virtio devices. Dynamic allocations will also be made possible > > > > -using the rpmsg bus (similar to how we already do dynamic allocations of > > > > -rpmsg channels; read more about it in rpmsg.txt). > > > > +The firmware must provide remoteproc with information regarding the > > > > +virtio devices it supports and their configurations: an `RSC_VDEV` > > > > +resource entry should detail the virtio device ID (as defined in > > > > +`virtio_ids.h`), virtio features, virtio config space, vrings > > > > +information, etc. > > > > + > > > > +Upon registration of a new remote, the remoteproc framework > > > > +searches for its resource table and registers the supported virtio > > > > +devices. A firmware may support multiple virtio devices, of various > > > > +types (a single remote can support multiple rpmsg virtio > > > > +devices if required). > > > > + > > > > +Moreover, `RSC_VDEV` resource entries suffice for static allocation > > > > +of virtio devices. Dynamic allocations will also be supported using > > > > +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg > > > > +channels. For more information, refer to rpmsg documentation. > > > > -- > > > > 2.39.3 (Apple Git-146) > > > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V4 3/7] Documentation: remoteproc: add more information 2024-10-25 15:13 ` anish kumar @ 2024-10-25 15:18 ` Mathieu Poirier 0 siblings, 0 replies; 18+ messages in thread From: Mathieu Poirier @ 2024-10-25 15:18 UTC (permalink / raw) To: anish kumar Cc: ohad, bjorn.andersson, corbet, linux-remoteproc, linux-doc, linux-kernel On Fri, 25 Oct 2024 at 09:13, anish kumar <yesanishhere@gmail.com> wrote: > > On Fri, Oct 25, 2024 at 7:57 AM Mathieu Poirier > <mathieu.poirier@linaro.org> wrote: > > > > On Thu, Oct 24, 2024 at 11:17:40AM -0700, anish kumar wrote: > > > On Thu, Oct 24, 2024 at 8:53 AM Mathieu Poirier > > > <mathieu.poirier@linaro.org> wrote: > > > > > > > > On Tue, Oct 22, 2024 at 10:33:53PM -0700, anish kumar wrote: > > > > > Added following changes: > > > > > 1. Components provided by remoteproc framework. > > > > > 2. Remoteproc driver responsibilities. > > > > > 3. Remoteproc framework responsibilities. > > > > > 4. Better explanation of how to ask for resources > > > > > from the framework by the remote processor. > > > > > > > > > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > > > > > --- > > > > > .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- > > > > > 1 file changed, 243 insertions(+), 353 deletions(-) > > > > > > > > > > diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst > > > > > index 9cccd3dd6a4b..7ca545eea153 100644 > > > > > --- a/Documentation/driver-api/remoteproc/remoteproc.rst > > > > > +++ b/Documentation/driver-api/remoteproc/remoteproc.rst > > > > > @@ -1,359 +1,249 @@ > > > > > +.. SPDX-License-Identifier: GPL-2.0 > > > > > + > > > > > ========================== > > > > > Remote Processor Framework > > > > > ========================== > > > > > > > > > > -Introduction > > > > > -============ > > > > > - > > > > > -Modern SoCs typically have heterogeneous remote processor devices in asymmetric > > > > > -multiprocessing (AMP) configurations, which may be running different instances > > > > > -of operating system, whether it's Linux or any other flavor of real-time OS. > > > > > - > > > > > -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. > > > > > -In a typical configuration, the dual cortex-A9 is running Linux in a SMP > > > > > -configuration, and each of the other three cores (two M3 cores and a DSP) > > > > > -is running its own instance of RTOS in an AMP configuration. > > > > > - > > > > > -The remoteproc framework allows different platforms/architectures to > > > > > -control (power on, load firmware, power off) those remote processors while > > > > > -abstracting the hardware differences, so the entire driver doesn't need to be > > > > > -duplicated. In addition, this framework also adds rpmsg virtio devices > > > > > -for remote processors that supports this kind of communication. This way, > > > > > -platform-specific remoteproc drivers only need to provide a few low-level > > > > > -handlers, and then all rpmsg drivers will then just work > > > > > -(for more information about the virtio-based rpmsg bus and its drivers, > > > > > -please read Documentation/staging/rpmsg.rst). > > > > > -Registration of other types of virtio devices is now also possible. Firmwares > > > > > -just need to publish what kind of virtio devices do they support, and then > > > > > -remoteproc will add those devices. This makes it possible to reuse the > > > > > -existing virtio drivers with remote processor backends at a minimal development > > > > > -cost. > > > > > - > > > > > -User API > > > > > -======== > > > > > - > > > > > -:: > > > > > - > > > > > - int rproc_boot(struct rproc *rproc) > > > > > - > > > > > -Boot a remote processor (i.e. load its firmware, power it on, ...). > > > > > - > > > > > -If the remote processor is already powered on, this function immediately > > > > > -returns (successfully). > > > > > - > > > > > -Returns 0 on success, and an appropriate error value otherwise. > > > > > -Note: to use this function you should already have a valid rproc > > > > > -handle. There are several ways to achieve that cleanly (devres, pdata, > > > > > -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we > > > > > -might also consider using dev_archdata for this). > > > > > - > > > > > -:: > > > > > - > > > > > - void rproc_shutdown(struct rproc *rproc) > > > > > - > > > > > -Power off a remote processor (previously booted with rproc_boot()). > > > > > -In case @rproc is still being used by an additional user(s), then > > > > > -this function will just decrement the power refcount and exit, > > > > > -without really powering off the device. > > > > > - > > > > > -Every call to rproc_boot() must (eventually) be accompanied by a call > > > > > -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. > > > > > - > > > > > -.. note:: > > > > > - > > > > > - we're not decrementing the rproc's refcount, only the power refcount. > > > > > - which means that the @rproc handle stays valid even after > > > > > - rproc_shutdown() returns, and users can still use it with a subsequent > > > > > - rproc_boot(), if needed. > > > > > - > > > > > -:: > > > > > - > > > > > - struct rproc *rproc_get_by_phandle(phandle phandle) > > > > > - > > > > > -Find an rproc handle using a device tree phandle. Returns the rproc > > > > > -handle on success, and NULL on failure. This function increments > > > > > -the remote processor's refcount, so always use rproc_put() to > > > > > -decrement it back once rproc isn't needed anymore. > > > > > - > > > > > -Typical usage > > > > > -============= > > > > > - > > > > > -:: > > > > > - > > > > > - #include <linux/remoteproc.h> > > > > > - > > > > > - /* in case we were given a valid 'rproc' handle */ > > > > > - int dummy_rproc_example(struct rproc *my_rproc) > > > > > - { > > > > > - int ret; > > > > > - > > > > > - /* let's power on and boot our remote processor */ > > > > > - ret = rproc_boot(my_rproc); > > > > > - if (ret) { > > > > > - /* > > > > > - * something went wrong. handle it and leave. > > > > > - */ > > > > > - } > > > > > - > > > > > - /* > > > > > - * our remote processor is now powered on... give it some work > > > > > - */ > > > > > - > > > > > - /* let's shut it down now */ > > > > > - rproc_shutdown(my_rproc); > > > > > - } > > > > > - > > > > > -API for implementors > > > > > -==================== > > > > > - > > > > > -:: > > > > > - > > > > > - struct rproc *rproc_alloc(struct device *dev, const char *name, > > > > > - const struct rproc_ops *ops, > > > > > - const char *firmware, int len) > > > > > - > > > > > -Allocate a new remote processor handle, but don't register > > > > > -it yet. Required parameters are the underlying device, the > > > > > -name of this remote processor, platform-specific ops handlers, > > > > > -the name of the firmware to boot this rproc with, and the > > > > > -length of private data needed by the allocating rproc driver (in bytes). > > > > > - > > > > > -This function should be used by rproc implementations during > > > > > -initialization of the remote processor. > > > > > - > > > > > -After creating an rproc handle using this function, and when ready, > > > > > -implementations should then call rproc_add() to complete > > > > > -the registration of the remote processor. > > > > > - > > > > > -On success, the new rproc is returned, and on failure, NULL. > > > > > - > > > > > -.. note:: > > > > > - > > > > > - **never** directly deallocate @rproc, even if it was not registered > > > > > - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). > > > > > - > > > > > -:: > > > > > - > > > > > - void rproc_free(struct rproc *rproc) > > > > > - > > > > > -Free an rproc handle that was allocated by rproc_alloc. > > > > > - > > > > > -This function essentially unrolls rproc_alloc(), by decrementing the > > > > > -rproc's refcount. It doesn't directly free rproc; that would happen > > > > > -only if there are no other references to rproc and its refcount now > > > > > -dropped to zero. > > > > > - > > > > > -:: > > > > > - > > > > > - int rproc_add(struct rproc *rproc) > > > > > - > > > > > -Register @rproc with the remoteproc framework, after it has been > > > > > -allocated with rproc_alloc(). > > > > > - > > > > > -This is called by the platform-specific rproc implementation, whenever > > > > > -a new remote processor device is probed. > > > > > - > > > > > -Returns 0 on success and an appropriate error code otherwise. > > > > > -Note: this function initiates an asynchronous firmware loading > > > > > -context, which will look for virtio devices supported by the rproc's > > > > > -firmware. > > > > > - > > > > > -If found, those virtio devices will be created and added, so as a result > > > > > -of registering this remote processor, additional virtio drivers might get > > > > > -probed. > > > > > - > > > > > -:: > > > > > - > > > > > - int rproc_del(struct rproc *rproc) > > > > > - > > > > > -Unroll rproc_add(). > > > > > - > > > > > -This function should be called when the platform specific rproc > > > > > -implementation decides to remove the rproc device. it should > > > > > -_only_ be called if a previous invocation of rproc_add() > > > > > -has completed successfully. > > > > > - > > > > > -After rproc_del() returns, @rproc is still valid, and its > > > > > -last refcount should be decremented by calling rproc_free(). > > > > > - > > > > > -Returns 0 on success and -EINVAL if @rproc isn't valid. > > > > > - > > > > > -:: > > > > > - > > > > > - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) > > > > > - > > > > > -Report a crash in a remoteproc > > > > > - > > > > > -This function must be called every time a crash is detected by the > > > > > -platform specific rproc implementation. This should not be called from a > > > > > -non-remoteproc driver. This function can be called from atomic/interrupt > > > > > -context. > > > > > - > > > > > -Implementation callbacks > > > > > +.. Contents: > > > > > + > > > > > + 1. Introduction > > > > > + 2. Remoteproc framework responsibilities > > > > > + 3. Remoteproc driver responsibilities > > > > > + 4. Virtio and rpmsg > > > > > + > > > > > +1. Introduction > > > > > +=============== > > > > > + > > > > > +Modern System on Chips (SoCs) typically integrate heterogeneous remote > > > > > +processor devices in asymmetric multiprocessing (AMP) configurations. > > > > > +These processors may run different operating systems, such as Linux and > > > > > +various real-time operating systems (RTOS). > > > > > > > > You are moving things around _and_ making modifications to the text in the same > > > > patch, something I specifically asked not to do. Moreover, the above conveys > > > > > > Sorry for doing that, I thought I moved the file first as you instructed but > > > I didn't understand that I should also not reword the sentences that convey > > > the same meaning. Will keep the text of below content in the same file: > > > 1. Introduction > > > 2. Binary Firmware Structure: will just add example as shown in this patch. > > > 3. Virtio and remoteproc: didn't change anything in the patch for this. > > > > We don't seem to understand each other - the "Virtio and remoteproc" section is > > the first thing I red and what made me aware of the rewording. > > I double checked and found you are right, I initially had changed > it but had no intention of changing in following > patches but somehow it creeped in. Will keep everything as before. > > > > > > > > > For below: > > > 1. User API: > > > 2. Typical usage: > > > 3. API for implementors : > > > > > > Should I just move all of the above text as it is to > > > remoteproc-kernel-api.rst or > > > you prefer here as I feel it belongs there? > > > > I have looked at other subsystems documented under Documentation/driver-api and > > I can't find instances of the type [subsystem]-kernel-api.rst. As such I don't > > see what is so special about remoteproc that it would mandate adding a new file. > > Why can't it just be in a single file like most subsystems do? > > Certainly, I can do that. Let me keep only the main file which > was the case, just add the relevant updates to the sections that > needs updates. Will that be agreeable to you? > That's all I'm asking for. > > > > > > > > > exactly the same information as found in [1] but using different words. I > > > > am in favour of enhancing documentation but not creating unneeded churn. > > > > > > > > I found several instances of the same rewording pattern in the sections below. > > > > As such I will not look at the other patches nor move forward with this set. > > > > > > > > Thanks, > > > > Mathieu > > > > > > Thanks Mathieu, for reviewing and providing feedback as always. > > > > > > > > > > > [1]. https://elixir.bootlin.com/linux/v6.12-rc4/source/Documentation/staging/remoteproc.rst > > > > > > > > > + > > > > > +For example, the OMAP4 platform features dual Cortex-A9 cores, dual > > > > > +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 > > > > > +cores execute Linux in a symmetric multiprocessing (SMP) configuration, > > > > > +while the M3 cores and DSP run independent instances of an RTOS. > > > > > + > > > > > +The remoteproc framework allows various platforms and architectures to > > > > > +manage remote processors, including operations such as powering on, > > > > > +loading firmware, and powering off. This framework abstracts hardware > > > > > +differences, promoting code reuse and minimizing duplication. It also > > > > > +supports rpmsg virtio devices for remote processors that utilize this > > > > > +communication method. Consequently, platform-specific remoteproc drivers > > > > > +need only implement a few low-level handlers, enabling seamless operation > > > > > +of all rpmsg drivers. (For more details about the virtio-based rpmsg > > > > > +bus and its drivers, refer to rpmsg documentation.) > > > > > + > > > > > +Additionally, the framework allows for the registration of various > > > > > +virtio devices. Firmware can publish the types of virtio devices it > > > > > +supports, facilitating their addition to the remoteproc framework. This > > > > > +flexibility enables the reuse of existing virtio drivers with remote > > > > > +processor backends at minimal development cost. > > > > > + > > > > > +The primary purpose of the remoteproc framework is to download firmware > > > > > +for remote processors and manage their lifecycle. The framework consists > > > > > +of several key components: > > > > > + > > > > > +- **Character Driver**: Provides userspace access to control the remote > > > > > + processor. > > > > > +- **ELF Utility**: Offers functions for handling ELF files and managing > > > > > + resources requested by the remote processor. > > > > > +- **Remoteproc Core**: Manages firmware downloads and recovery actions > > > > > + in case of a remote processor crash. > > > > > +- **Coredump**: Provides facilities for coredumping and tracing from > > > > > + the remote processor in the event of a crash. > > > > > +- **Userspace Interaction**: Uses sysfs and debugfs to manage the > > > > > + lifecycle and status of the remote processor. > > > > > +- **Virtio Support**: Facilitates interaction with the virtio and > > > > > + rpmsg bus. > > > > > + > > > > > +From here on, references to "framework" denote the remoteproc > > > > > +framework, and "driver" refers to the remoteproc driver that utilizes > > > > > +the framework for managing remote processors. > > > > > + > > > > > +2. Remoteproc framework Responsibilities > > > > > +======================================== > > > > > + > > > > > +The framework begins by gathering information about the firmware file > > > > > +to be downloaded through the request_firmware function. It supports > > > > > +the ELF format and parses the firmware image to identify the physical > > > > > +addresses that need to be populated from the corresponding ELF sections. > > > > > +The framework also requires knowledge of the logical or I/O-mapped > > > > > +addresses in the application processor. Once this information is > > > > > +obtained from the driver, the framework transfers the data to the > > > > > +specified addresses and starts the remote, along with > > > > > +any devices physically or logically connected to it. > > > > > + > > > > > +Dependent devices, referred to as `subdevices` within the framework, > > > > > +are also managed post-registration by their respective drivers. > > > > > +Subdevices can register themselves using `rproc_(add/remove)_subdev`. > > > > > +Non-remoteproc drivers can use subdevices as a way to logically connect > > > > > +to remote and get lifecycle notifications of the remote. > > > > > + > > > > > +The framework oversees the lifecycle of the remote and > > > > > +provides the `rproc_report_crash` function, which the driver invokes > > > > > +upon receiving a crash notification from the remote. The > > > > > +notification method can differ based on the design of the remote > > > > > +processor and its communication with the application processor. For > > > > > +instance, if the remote is a DSP equipped with a watchdog, > > > > > +unresponsive behavior triggers the watchdog, generating an interrupt > > > > > +that routes to the application processor, allowing it to call > > > > > +`rproc_report_crash` in the driver's interrupt context. > > > > > + > > > > > +During crash handling, the framework performs the following actions: > > > > > + > > > > > +a. Sends a request to stop the remote and any connected or > > > > > + dependent subdevices. > > > > > +b. Generates a coredump, dumping all `resources` requested by the > > > > > + remote alongside relevant debugging information. Resources are > > > > > + explained below. > > > > > +c. Reloads the firmware and restarts the remote. > > > > > + > > > > > +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and > > > > > +attach callbacks of the driver are invoked without reloading the > > > > > +firmware. This is useful when the remote requires no > > > > > +assistance for recovery, or when the application processor can restart > > > > > +independently. After recovery, the application processor can reattach > > > > > +to the remote. > > > > > + > > > > > +The remote can request resources from the framework, which > > > > > +allocates a ".resource_table" section. During the ELF parsing phase, > > > > > +the framework identifies this section and calls the appropriate > > > > > +handler to allocate the requested resources. > > > > > + > > > > > +Resource management within the framework can accommodate any type of > > > > > +`fw_resource_type`. > > > > > + > > > > > +.. code-block:: c > > > > > + > > > > > + enum fw_resource_type { > > > > > + RSC_CARVEOUT = 0, > > > > > + RSC_DEVMEM = 1, > > > > > + RSC_TRACE = 2, > > > > > + RSC_VDEV = 3, > > > > > + RSC_LAST = 4, > > > > > + RSC_VENDOR_START = 128, > > > > > + RSC_VENDOR_END = 512, > > > > > + }; > > > > > + > > > > > + struct resource_table { > > > > > + u32 ver; > > > > > + u32 num; > > > > > + u32 reserved[2]; > > > > > + u32 offset[]; > > > > > + } __packed; > > > > > + > > > > > + struct fw_rsc_hdr { > > > > > + u32 type; > > > > > + u8 data[]; > > > > > + } __packed; > > > > > + > > > > > +For example, if the remote requests both `RSC_TRACE` and > > > > > +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured > > > > > +as follows: > > > > > + > > > > > +.. code-block:: c > > > > > + > > > > > + #define MAX_SHARED_RESOURCE 2 > > > > > + #define LOG_BUF_SIZE 1000 > > > > > + #define CARVEOUT_DUMP_PA 0x12345678 > > > > > + #define CARVEOUT_DUMP_SIZE 2000 > > > > > + > > > > > + struct shared_resource_table { > > > > > + u32 ver; > > > > > + u32 num; > > > > > + u32 reserved[2]; > > > > > + u32 offset[MAX_SHARED_RESOURCE]; > > > > > + struct fw_rsc_trace log_trace; > > > > > + struct fw_rsc_carveout dump_carveout; > > > > > + }; > > > > > + > > > > > + volatile struct shared_resource_table table = { > > > > > + .ver = 1, > > > > > + .num = 2, > > > > > + .reserved = {0, 0}, > > > > > + .offset = { > > > > > + offsetof(struct resource_table, log_trace), > > > > > + offsetof(struct resource_table, dump_carveout), > > > > > + }, > > > > > + .log_trace = { > > > > > + RSC_TRACE, > > > > > + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", > > > > > + }, > > > > > + .dump_carveout = { > > > > > + RSC_CARVEOUT, > > > > > + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", > > > > > + }, > > > > > + }; > > > > > + > > > > > +The framework creates a sysfs file when it encounters the `RSC_TRACE` > > > > > +type to expose log information to userspace. Other resource types are > > > > > +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes > > > > > +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. > > > > > + > > > > > + > > > > > +3. Remoteproc driver responsibilities > > > > > +===================================== > > > > > + > > > > > +The driver must provide the following information to the core: > > > > > + > > > > > +a. Translate device addresses (physical addresses) found in the ELF > > > > > + firmware to virtual addresses in Linux using the `da_to_va` > > > > > + callback. This allows the framework to copy ELF firmware from the > > > > > + filesystem to the addresses expected by the remote since > > > > > + the framework cannot directly access those physical addresses. > > > > > +b. Prepare/unprepare the remote prior to firmware loading, > > > > > + which may involve allocating carveout and reserved memory regions. > > > > > +c. Implement methods for starting and stopping the remote, > > > > > + whether by setting registers or sending explicit interrupts, > > > > > + depending on the hardware design. > > > > > +d. Provide attach and detach callbacks to start the remote > > > > > + without loading the firmware. This is beneficial when the remote > > > > > + processor is already loaded and running. > > > > > +e. Implement a load callback for firmware loading, typically using > > > > > + the ELF loader provided by the framework; currently, only ELF > > > > > + format is supported. > > > > > +f. Invoke the framework's crash handler API upon detecting a remote > > > > > + crash. > > > > > + > > > > > +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` > > > > > +to register themselves with the framework. > > > > > + > > > > > +.. code-block:: c > > > > > + > > > > > + struct rproc_ops { > > > > > + int (*prepare)(struct rproc *rproc); > > > > > + int (*unprepare)(struct rproc *rproc); > > > > > + int (*start)(struct rproc *rproc); > > > > > + int (*stop)(struct rproc *rproc); > > > > > + int (*attach)(struct rproc *rproc); > > > > > + int (*detach)(struct rproc *rproc); > > > > > + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, > > > > > + bool *is_iomem); > > > > > + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); > > > > > + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, > > > > > + void *rsc, int offset, int avail); > > > > > + int (*load)(struct rproc *rproc, const struct firmware *fw); > > > > > + //snip > > > > > + }; > > > > > + > > > > > + > > > > > +4. Virtio and Remoteproc > > > > > ======================== > > > > > > > > > > -These callbacks should be provided by platform-specific remoteproc > > > > > -drivers:: > > > > > - > > > > > - /** > > > > > - * struct rproc_ops - platform-specific device handlers > > > > > - * @start: power on the device and boot it > > > > > - * @stop: power off the device > > > > > - * @kick: kick a virtqueue (virtqueue id given as a parameter) > > > > > - */ > > > > > - struct rproc_ops { > > > > > - int (*start)(struct rproc *rproc); > > > > > - int (*stop)(struct rproc *rproc); > > > > > - void (*kick)(struct rproc *rproc, int vqid); > > > > > - }; > > > > > - > > > > > -Every remoteproc implementation should at least provide the ->start and ->stop > > > > > -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler > > > > > -should be provided as well. > > > > > - > > > > > -The ->start() handler takes an rproc handle and should then power on the > > > > > -device and boot it (use rproc->priv to access platform-specific private data). > > > > > -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc > > > > > -core puts there the ELF entry point). > > > > > -On success, 0 should be returned, and on failure, an appropriate error code. > > > > > - > > > > > -The ->stop() handler takes an rproc handle and powers the device down. > > > > > -On success, 0 is returned, and on failure, an appropriate error code. > > > > > - > > > > > -The ->kick() handler takes an rproc handle, and an index of a virtqueue > > > > > -where new message was placed in. Implementations should interrupt the remote > > > > > -processor and let it know it has pending messages. Notifying remote processors > > > > > -the exact virtqueue index to look in is optional: it is easy (and not > > > > > -too expensive) to go through the existing virtqueues and look for new buffers > > > > > -in the used rings. > > > > > - > > > > > -Binary Firmware Structure > > > > > -========================= > > > > > - > > > > > -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, > > > > > -it is quite expected that other platforms/devices which we'd want to > > > > > -support with this framework will be based on different binary formats. > > > > > - > > > > > -When those use cases show up, we will have to decouple the binary format > > > > > -from the framework core, so we can support several binary formats without > > > > > -duplicating common code. > > > > > - > > > > > -When the firmware is parsed, its various segments are loaded to memory > > > > > -according to the specified device address (might be a physical address > > > > > -if the remote processor is accessing memory directly). > > > > > - > > > > > -In addition to the standard ELF segments, most remote processors would > > > > > -also include a special section which we call "the resource table". > > > > > - > > > > > -The resource table contains system resources that the remote processor > > > > > -requires before it should be powered on, such as allocation of physically > > > > > -contiguous memory, or iommu mapping of certain on-chip peripherals. > > > > > -Remotecore will only power up the device after all the resource table's > > > > > -requirement are met. > > > > > - > > > > > -In addition to system resources, the resource table may also contain > > > > > -resource entries that publish the existence of supported features > > > > > -or configurations by the remote processor, such as trace buffers and > > > > > -supported virtio devices (and their configurations). > > > > > - > > > > > -The resource table begins with this header:: > > > > > - > > > > > - /** > > > > > - * struct resource_table - firmware resource table header > > > > > - * @ver: version number > > > > > - * @num: number of resource entries > > > > > - * @reserved: reserved (must be zero) > > > > > - * @offset: array of offsets pointing at the various resource entries > > > > > - * > > > > > - * The header of the resource table, as expressed by this structure, > > > > > - * contains a version number (should we need to change this format in the > > > > > - * future), the number of available resource entries, and their offsets > > > > > - * in the table. > > > > > - */ > > > > > - struct resource_table { > > > > > - u32 ver; > > > > > - u32 num; > > > > > - u32 reserved[2]; > > > > > - u32 offset[0]; > > > > > - } __packed; > > > > > - > > > > > -Immediately following this header are the resource entries themselves, > > > > > -each of which begins with the following resource entry header:: > > > > > - > > > > > - /** > > > > > - * struct fw_rsc_hdr - firmware resource entry header > > > > > - * @type: resource type > > > > > - * @data: resource data > > > > > - * > > > > > - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing > > > > > - * its @type. The content of the entry itself will immediately follow > > > > > - * this header, and it should be parsed according to the resource type. > > > > > - */ > > > > > - struct fw_rsc_hdr { > > > > > - u32 type; > > > > > - u8 data[0]; > > > > > - } __packed; > > > > > - > > > > > -Some resources entries are mere announcements, where the host is informed > > > > > -of specific remoteproc configuration. Other entries require the host to > > > > > -do something (e.g. allocate a system resource). Sometimes a negotiation > > > > > -is expected, where the firmware requests a resource, and once allocated, > > > > > -the host should provide back its details (e.g. address of an allocated > > > > > -memory region). > > > > > - > > > > > -Here are the various resource types that are currently supported:: > > > > > - > > > > > - /** > > > > > - * enum fw_resource_type - types of resource entries > > > > > - * > > > > > - * @RSC_CARVEOUT: request for allocation of a physically contiguous > > > > > - * memory region. > > > > > - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. > > > > > - * @RSC_TRACE: announces the availability of a trace buffer into which > > > > > - * the remote processor will be writing logs. > > > > > - * @RSC_VDEV: declare support for a virtio device, and serve as its > > > > > - * virtio header. > > > > > - * @RSC_LAST: just keep this one at the end > > > > > - * @RSC_VENDOR_START: start of the vendor specific resource types range > > > > > - * @RSC_VENDOR_END: end of the vendor specific resource types range > > > > > - * > > > > > - * Please note that these values are used as indices to the rproc_handle_rsc > > > > > - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to > > > > > - * check the validity of an index before the lookup table is accessed, so > > > > > - * please update it as needed. > > > > > - */ > > > > > - enum fw_resource_type { > > > > > - RSC_CARVEOUT = 0, > > > > > - RSC_DEVMEM = 1, > > > > > - RSC_TRACE = 2, > > > > > - RSC_VDEV = 3, > > > > > - RSC_LAST = 4, > > > > > - RSC_VENDOR_START = 128, > > > > > - RSC_VENDOR_END = 512, > > > > > - }; > > > > > - > > > > > -For more details regarding a specific resource type, please see its > > > > > -dedicated structure in include/linux/remoteproc.h. > > > > > - > > > > > -We also expect that platform-specific resource entries will show up > > > > > -at some point. When that happens, we could easily add a new RSC_PLATFORM > > > > > -type, and hand those resources to the platform-specific rproc driver to handle. > > > > > - > > > > > -Virtio and remoteproc > > > > > -===================== > > > > > - > > > > > -The firmware should provide remoteproc information about virtio devices > > > > > -that it supports, and their configurations: a RSC_VDEV resource entry > > > > > -should specify the virtio device id (as in virtio_ids.h), virtio features, > > > > > -virtio config space, vrings information, etc. > > > > > - > > > > > -When a new remote processor is registered, the remoteproc framework > > > > > -will look for its resource table and will register the virtio devices > > > > > -it supports. A firmware may support any number of virtio devices, and > > > > > -of any type (a single remote processor can also easily support several > > > > > -rpmsg virtio devices this way, if desired). > > > > > - > > > > > -Of course, RSC_VDEV resource entries are only good enough for static > > > > > -allocation of virtio devices. Dynamic allocations will also be made possible > > > > > -using the rpmsg bus (similar to how we already do dynamic allocations of > > > > > -rpmsg channels; read more about it in rpmsg.txt). > > > > > +The firmware must provide remoteproc with information regarding the > > > > > +virtio devices it supports and their configurations: an `RSC_VDEV` > > > > > +resource entry should detail the virtio device ID (as defined in > > > > > +`virtio_ids.h`), virtio features, virtio config space, vrings > > > > > +information, etc. > > > > > + > > > > > +Upon registration of a new remote, the remoteproc framework > > > > > +searches for its resource table and registers the supported virtio > > > > > +devices. A firmware may support multiple virtio devices, of various > > > > > +types (a single remote can support multiple rpmsg virtio > > > > > +devices if required). > > > > > + > > > > > +Moreover, `RSC_VDEV` resource entries suffice for static allocation > > > > > +of virtio devices. Dynamic allocations will also be supported using > > > > > +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg > > > > > +channels. For more information, refer to rpmsg documentation. > > > > > -- > > > > > 2.39.3 (Apple Git-146) > > > > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH V4 4/7] Remoteproc: Documentation: add driver api 2024-10-23 5:33 [PATCH 0/1] remoteproc documentation changes anish kumar ` (2 preceding siblings ...) 2024-10-23 5:33 ` [PATCH V4 3/7] Documentation: remoteproc: add more information anish kumar @ 2024-10-23 5:33 ` anish kumar 2024-10-23 5:33 ` [PATCH V4 5/7] Remoteproc: Documentation: add userspace api anish kumar ` (3 subsequent siblings) 7 siblings, 0 replies; 18+ messages in thread From: anish kumar @ 2024-10-23 5:33 UTC (permalink / raw) To: ohad, bjorn.andersson, mathieu.poirier, corbet Cc: linux-remoteproc, linux-doc, linux-kernel, anish kumar Add driver api documenation for remoteproc. Signed-off-by: anish kumar <yesanishhere@gmail.com> --- Documentation/driver-api/remoteproc/index.rst | 1 + .../remoteproc/remoteproc-kernel-api.rst | 239 ++++++++++++++++++ 2 files changed, 240 insertions(+) create mode 100644 Documentation/driver-api/remoteproc/remoteproc-kernel-api.rst diff --git a/Documentation/driver-api/remoteproc/index.rst b/Documentation/driver-api/remoteproc/index.rst index 6ac5a082bf4f..7fa1c5755be8 100644 --- a/Documentation/driver-api/remoteproc/index.rst +++ b/Documentation/driver-api/remoteproc/index.rst @@ -19,6 +19,7 @@ Authors: :maxdepth: 1 remoteproc + remoteproc-kernel-api Mailing List ------------ diff --git a/Documentation/driver-api/remoteproc/remoteproc-kernel-api.rst b/Documentation/driver-api/remoteproc/remoteproc-kernel-api.rst new file mode 100644 index 000000000000..018c5bf9b3d7 --- /dev/null +++ b/Documentation/driver-api/remoteproc/remoteproc-kernel-api.rst @@ -0,0 +1,239 @@ +===================================================== +The Linux Remoteproc subsystem Driver Core kernel API +===================================================== + +anish kumar <yesanishhere@gmail.com> + +Introduction +------------ +This document does not describe what a Remote processor subsystem +(RPROC) Driver or Device is. It also does not describe the API +which can be used by user space to communicate with a RPROC driver. +If you want to know this then please read the following +file: Documentation/remoteproc/rproc-api.rst . + +So what does this document describe? It describes the API that can be used by +remote processor Drivers that want to use the remote processor Driver Core +Framework. This framework provides all interfacing towards user space so that +the same code does not have to be reproduced each time. This also means that +a remote processor driver then only needs to provide the different routines +(operations) that control the remote processor. + +The API +------- +Each remote processor driver that wants to use the remote processor Driver Core +must #include <linux/remoteproc.h> (you would have to do this anyway when +writing a rproc device driver). This include file contains following +register routine:: + + int devm_rproc_add(struct device *dev, struct rproc *rproc) + +The devm_rproc_add routine registers a remote processor device. +The parameter of this routine is a pointer to a rproc device structure. +This routine returns zero on success and a negative errno code for failure. + +The rproc device structure looks like this:: + + struct rproc { + struct list_head node; + struct iommu_domain *domain; + const char *name; + const char *firmware; + void *priv; + struct rproc_ops *ops; + struct device dev; + atomic_t power; + unsigned int state; + enum rproc_dump_mechanism dump_conf; + struct mutex lock; + struct dentry *dbg_dir; + struct list_head traces; + int num_traces; + struct list_head carveouts; + struct list_head mappings; + u64 bootaddr; + struct list_head rvdevs; + struct list_head subdevs; + struct idr notifyids; + int index; + struct work_struct crash_handler; + unsigned int crash_cnt; + bool recovery_disabled; + int max_notifyid; + struct resource_table *table_ptr; + struct resource_table *clean_table; + struct resource_table *cached_table; + size_t table_sz; + bool has_iommu; + bool auto_boot; + bool sysfs_read_only; + struct list_head dump_segments; + int nb_vdev; + u8 elf_class; + u16 elf_machine; + struct cdev cdev; + bool cdev_put_on_release; + DECLARE_BITMAP(features, RPROC_MAX_FEATURES); + }; + +It contains following fields: + +* node: list node of this rproc object +* domain: iommu domain +* name: human readable name of the rproc +* firmware: name of firmware file to be loaded +* priv: private data which belongs to the platform-specific rproc module +* ops: platform-specific start/stop rproc handlers +* dev: virtual device for refcounting and common remoteproc behavior +* power: refcount of users who need this rproc powered up +* state: state of the device +* dump_conf: Currently selected coredump configuration +* lock: lock which protects concurrent manipulations of the rproc +* dbg_dir: debugfs directory of this rproc device +* traces: list of trace buffers +* num_traces: number of trace buffers +* carveouts: list of physically contiguous memory allocations +* mappings: list of iommu mappings we initiated, needed on shutdown +* bootaddr: address of first instruction to boot rproc with (optional) +* rvdevs: list of remote virtio devices +* subdevs: list of subdevices, to following the running state +* notifyids: idr for dynamically assigning rproc-wide unique notify ids +* index: index of this rproc device +* crash_handler: workqueue for handling a crash +* crash_cnt: crash counter +* recovery_disabled: flag that state if recovery was disabled +* max_notifyid: largest allocated notify id. +* table_ptr: pointer to the resource table in effect +* clean_table: copy of the resource table without modifications. Used +* when a remote processor is attached or detached from the core +* cached_table: copy of the resource table +* table_sz: size of @cached_table +* has_iommu: flag to indicate if remote processor is behind an MMU +* auto_boot: flag to indicate if remote processor should be auto-started +* sysfs_read_only: flag to make remoteproc sysfs files read only +* dump_segments: list of segments in the firmware +* nb_vdev: number of vdev currently handled by rproc +* elf_class: firmware ELF class +* elf_machine: firmware ELF machine +* cdev: character device of the rproc +* cdev_put_on_release: flag to indicate if remoteproc should be shutdown on @char_dev release +* features: indicate remoteproc features + +The list of rproc operations is defined as:: + + struct rproc_ops { + int (*prepare)(struct rproc *rproc); + int (*unprepare)(struct rproc *rproc); + int (*start)(struct rproc *rproc); + int (*stop)(struct rproc *rproc); + int (*attach)(struct rproc *rproc); + int (*detach)(struct rproc *rproc); + void (*kick)(struct rproc *rproc, int vqid); + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, bool *is_iomem); + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, void *rsc, + int offset, int avail); + struct resource_table *(*find_loaded_rsc_table)( + struct rproc *rproc, const struct firmware *fw); + struct resource_table *(*get_loaded_rsc_table)( + struct rproc *rproc, size_t *size); + int (*load)(struct rproc *rproc, const struct firmware *fw); + int (*sanity_check)(struct rproc *rproc, const struct firmware *fw); + u64 (*get_boot_addr)(struct rproc *rproc, const struct firmware *fw); + unsigned long (*panic)(struct rproc *rproc); + void (*coredump)(struct rproc *rproc); + }; + +Most of the operations are optional. Currently in the implementation +there are no mandatory operations, however from the practical standpoint +minimum ops are: + +* start: this is a pointer to the routine that starts the remote processor + device. + The routine needs a pointer to the remote processor device structure as a + parameter. It returns zero on success or a negative errno code for failure. + +* stop: with this routine the remote processor device is being stopped. + + The routine needs a pointer to the remote processor device structure as a + parameter. It returns zero on success or a negative errno code for failure. + +* da_to_va: this is the routine that needs to translate device address to + application processor virtual address that it can copy code to. + + The routine needs a pointer to the remote processor device structure as a + parameter. It returns zero on success or a negative errno code for failure. + + The routine provides the device address it finds in the ELF firmware and asks + the driver to convert that to virtual address. + +All other callbacks are optional in case of ELF provided firmware. + +* load: this is to load the firmware on to the remote device. + + The routine needs firmware file that it needs to load on to the remote processor. + If the driver overrides this callback then default ELF loader will not get used. + Otherwise default framework provided loader gets used. + + load = rproc_elf_load_segments; + parse_fw = rproc_elf_load_rsc_table; + find_loaded_rsc_table = rproc_elf_find_loaded_rsc_table; + sanity_check = rproc_elf_sanity_check; + get_boot_addr = rproc_elf_get_boot_addr; + +* parse_fw: this routing parses the provided firmware. In case of ELF format, + framework provided rproc_elf_load_rsc_table function can be used. + +* sanity_check: Check the format of the firmware. + +* coredump: If the driver prefers to manage coredumps independently, it can + implement its own coredump handling. However, the framework offers a default + implementation for the ELF format by assigning this callback to + rproc_coredump, unless the driver has overridden it. + +* get_boot_addr: In case the bootaddr defined in ELF firmware is different, driver + can use this callback to set a different boot address for remote processor to + starts its reset vector from. + +* find_loaded_rsc_table: this routine gets the loaded resource table from the firmware. + + resource table should have a section named (.resource_table) for the framework + to understand and interpret its content. Resource table is a way for remote + processor to ask for resources such as memory for dumping and logging. Look + at core documentation to know how to create the ELF section for the same. + +* get_loaded_rsc_table: Driver can customize passing the resource table by overriding + this callback. Framework doesn't provide any default implementation for the same. + + +The rproc_report_crash function allows you to report a crash when crash is +detected by the driver. + +:: + + void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type); + +To add a subdev corresponding driver can call:: + + void rproc_add_subdev(struct rproc *rproc, struct rproc_subdev *subdev); + +To remove a subdev, driver can call. + +:: + + void rproc_remove_subdev(struct rproc *rproc, struct rproc_subdev *subdev); + +To work with ELF coredump below function can be called:: + + void rproc_coredump_cleanup(struct rproc *rproc); + void rproc_coredump(struct rproc *rproc); + void rproc_coredump_using_sections(struct rproc *rproc); + int rproc_coredump_add_segment(struct rproc *rproc, dma_addr_t da, size_t size); + int rproc_coredump_add_custom_segment(struct rproc *rproc, + dma_addr_t da, size_t size, + void (*dumpfn)(struct rproc *rproc, + struct rproc_dump_segment *segment, + void *dest, size_t offset, + size_t size), + +Remember that coredump functions provided by the framework only works with ELF format. -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH V4 5/7] Remoteproc: Documentation: add userspace api 2024-10-23 5:33 [PATCH 0/1] remoteproc documentation changes anish kumar ` (3 preceding siblings ...) 2024-10-23 5:33 ` [PATCH V4 4/7] Remoteproc: Documentation: add driver api anish kumar @ 2024-10-23 5:33 ` anish kumar 2024-10-23 5:33 ` [PATCH V4 6/7] MAINTAINERS: remoteproc documentation path update anish kumar ` (2 subsequent siblings) 7 siblings, 0 replies; 18+ messages in thread From: anish kumar @ 2024-10-23 5:33 UTC (permalink / raw) To: ohad, bjorn.andersson, mathieu.poirier, corbet Cc: linux-remoteproc, linux-doc, linux-kernel, anish kumar Add information about how userspace can control the remoteproc framework and different ways to do that. Signed-off-by: anish kumar <yesanishhere@gmail.com> --- Documentation/driver-api/remoteproc/index.rst | 1 + .../driver-api/remoteproc/remoteproc-api.rst | 77 +++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100644 Documentation/driver-api/remoteproc/remoteproc-api.rst diff --git a/Documentation/driver-api/remoteproc/index.rst b/Documentation/driver-api/remoteproc/index.rst index 7fa1c5755be8..840b4672352b 100644 --- a/Documentation/driver-api/remoteproc/index.rst +++ b/Documentation/driver-api/remoteproc/index.rst @@ -20,6 +20,7 @@ Authors: remoteproc remoteproc-kernel-api + remoteproc-api Mailing List ------------ diff --git a/Documentation/driver-api/remoteproc/remoteproc-api.rst b/Documentation/driver-api/remoteproc/remoteproc-api.rst new file mode 100644 index 000000000000..0306b331a20e --- /dev/null +++ b/Documentation/driver-api/remoteproc/remoteproc-api.rst @@ -0,0 +1,77 @@ +================================== +The Linux Remoteproc userspace API +================================== + +Introduction +============ + +A Remoteproc (rproc) is a subsystem for managing the lifecycle +of a processor that is connected to Linux. + +At times, userspace may need to check the state of the remote processor to +prevent other processes from using it. For instance, if the remote processor +is a DSP used for playback, there may be situations where the DSP is +undergoing recovery and cannot be used. In such cases, attempts to access the +DSP for playback should be blocked. The rproc framework provides sysfs APIs +to inform userspace of the processor's current status which should be utilised +to achieve the same. + +Additionally, there are scenarios where userspace applications need to explicitly +control the rproc. In these cases, rproc also offers the file descriptors. + +The simplest API +================ + +Below set of api's can be used to start and stop the rproc +where 'X' refers to instance of associated remoteproc. There can be systems +where there are more than one rprocs such as multiple DSP's +connected to application processors running Linux. + +.. code-block:: c + + echo start > /sys/class/remoteproc/remoteprocX/state + echo stop > /sys/class/remoteproc/remoteprocX/state + +To know the state of rproc: + +.. code-block:: c + + cat /sys/class/remoteproc/remoteprocX/state + + +To dynamically replace firmware, execute the following commands: + +.. code-block:: c + + echo stop > /sys/class/remoteproc/remoteprocX/state + echo -n <firmware_name> > + /sys/class/remoteproc/remoteprocX/firmware + echo start > /sys/class/remoteproc/remoteprocX/state + +To simulate a remote crash, execute: + +.. code-block:: c + + echo 1 > /sys/kernel/debug/remoteproc/remoteprocX/crash + +To get the trace logs, execute + +.. code-block:: c + + cat /sys/kernel/debug/remoteproc/remoteprocX/crashX + +where X will be 0 or 1 if there are 2 resources. Also, this +file will only exist if resources are defined in ELF firmware +file. + +The coredump feature can be disabled with the following command: + +.. code-block:: c + + echo disabled > /sys/kernel/debug/remoteproc/remoteprocX/coredump + +Userspace can also control start/stop of rproc by using a +remoteproc Character Device, it can open the open a file descriptor +and write `start` to initiate it, and `stop` to terminate it. + +[FIXME -- better explanations] -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH V4 6/7] MAINTAINERS: remoteproc documentation path update 2024-10-23 5:33 [PATCH 0/1] remoteproc documentation changes anish kumar ` (4 preceding siblings ...) 2024-10-23 5:33 ` [PATCH V4 5/7] Remoteproc: Documentation: add userspace api anish kumar @ 2024-10-23 5:33 ` anish kumar 2024-10-23 5:33 ` [PATCH V4 7/7] rpmsg: Documentation: update the path anish kumar 2024-10-23 13:53 ` [PATCH 0/1] remoteproc documentation changes Jonathan Corbet 7 siblings, 0 replies; 18+ messages in thread From: anish kumar @ 2024-10-23 5:33 UTC (permalink / raw) To: ohad, bjorn.andersson, mathieu.poirier, corbet Cc: linux-remoteproc, linux-doc, linux-kernel, anish kumar As remoteproc documentation moved from staging, update the same in this file. Signed-off-by: anish kumar <yesanishhere@gmail.com> --- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MAINTAINERS b/MAINTAINERS index eeb4c70b3d5b..1fa39bee6ab7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -15907,7 +15907,7 @@ S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/andersson/remoteproc.git rproc-next F: Documentation/ABI/testing/sysfs-class-remoteproc F: Documentation/devicetree/bindings/remoteproc/ -F: Documentation/staging/remoteproc.rst +F: Documentation/driver-api/remoteproc/ F: drivers/remoteproc/ F: include/linux/remoteproc.h F: include/linux/remoteproc/ -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH V4 7/7] rpmsg: Documentation: update the path 2024-10-23 5:33 [PATCH 0/1] remoteproc documentation changes anish kumar ` (5 preceding siblings ...) 2024-10-23 5:33 ` [PATCH V4 6/7] MAINTAINERS: remoteproc documentation path update anish kumar @ 2024-10-23 5:33 ` anish kumar 2024-10-23 13:53 ` [PATCH 0/1] remoteproc documentation changes Jonathan Corbet 7 siblings, 0 replies; 18+ messages in thread From: anish kumar @ 2024-10-23 5:33 UTC (permalink / raw) To: ohad, bjorn.andersson, mathieu.poirier, corbet Cc: linux-remoteproc, linux-doc, linux-kernel, anish kumar Remoteproc documentation is moved from text file to reStructuredText file. Signed-off-by: anish kumar <yesanishhere@gmail.com> --- Documentation/staging/rpmsg.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/staging/rpmsg.rst b/Documentation/staging/rpmsg.rst index 1ce353cb232a..937cd2574584 100644 --- a/Documentation/staging/rpmsg.rst +++ b/Documentation/staging/rpmsg.rst @@ -5,7 +5,7 @@ Remote Processor Messaging (rpmsg) Framework .. note:: This document describes the rpmsg bus and how to write rpmsg drivers. - To learn how to add rpmsg support for new platforms, check out remoteproc.txt + To learn how to add rpmsg support for new platforms, check out remoteproc (also a resident of Documentation/). Introduction -- 2.39.3 (Apple Git-146) ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH 0/1] remoteproc documentation changes 2024-10-23 5:33 [PATCH 0/1] remoteproc documentation changes anish kumar ` (6 preceding siblings ...) 2024-10-23 5:33 ` [PATCH V4 7/7] rpmsg: Documentation: update the path anish kumar @ 2024-10-23 13:53 ` Jonathan Corbet 2024-10-23 14:43 ` Jonathan Corbet 2024-10-23 15:35 ` Mathieu Poirier 7 siblings, 2 replies; 18+ messages in thread From: Jonathan Corbet @ 2024-10-23 13:53 UTC (permalink / raw) To: anish kumar, ohad, bjorn.andersson, mathieu.poirier Cc: linux-remoteproc, linux-doc, linux-kernel, kernel test robot anish kumar <yesanishhere@gmail.com> writes: > This patch series transitions the documentation > for remoteproc from the staging directory to the > mainline kernel. It introduces both kernel and > user-space APIs, enhancing the overall documentation > quality. > > V4: > Fixed compilation errors and moved documentation to > driver-api directory. > > V3: > Seperated out the patches further to make the intention > clear for each patch. > > V2: > Reported-by: kernel test robot <lkp@intel.com> > Closes: https://lore.kernel.org/oe-kbuild-all/202410161444.jOKMsoGS-lkp@intel.com/ So I think you could make better use of kerneldoc comments for a number of your APIs and structures - a project for the future. I can't judge the remoteproc aspects of this, but from a documentation mechanics point of view, this looks about ready to me. In the absence of objections I'll apply it in the near future. Thanks, jon ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/1] remoteproc documentation changes 2024-10-23 13:53 ` [PATCH 0/1] remoteproc documentation changes Jonathan Corbet @ 2024-10-23 14:43 ` Jonathan Corbet 2024-10-23 15:35 ` Mathieu Poirier 1 sibling, 0 replies; 18+ messages in thread From: Jonathan Corbet @ 2024-10-23 14:43 UTC (permalink / raw) To: anish kumar, ohad, bjorn.andersson, mathieu.poirier Cc: linux-remoteproc, linux-doc, linux-kernel, kernel test robot Jonathan Corbet <corbet@lwn.net> writes: > anish kumar <yesanishhere@gmail.com> writes: > >> This patch series transitions the documentation >> for remoteproc from the staging directory to the >> mainline kernel. It introduces both kernel and >> user-space APIs, enhancing the overall documentation >> quality. >> >> V4: >> Fixed compilation errors and moved documentation to >> driver-api directory. >> >> V3: >> Seperated out the patches further to make the intention >> clear for each patch. >> >> V2: >> Reported-by: kernel test robot <lkp@intel.com> >> Closes: https://lore.kernel.org/oe-kbuild-all/202410161444.jOKMsoGS-lkp@intel.com/ > > So I think you could make better use of kerneldoc comments for a number > of your APIs and structures - a project for the future. I can't judge > the remoteproc aspects of this, but from a documentation mechanics point > of view, this looks about ready to me. In the absence of objections > I'll apply it in the near future. One other question, actually - what kernel version did you make these patches against? It looks like something rather old...? Thanks, jon ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/1] remoteproc documentation changes 2024-10-23 13:53 ` [PATCH 0/1] remoteproc documentation changes Jonathan Corbet 2024-10-23 14:43 ` Jonathan Corbet @ 2024-10-23 15:35 ` Mathieu Poirier 1 sibling, 0 replies; 18+ messages in thread From: Mathieu Poirier @ 2024-10-23 15:35 UTC (permalink / raw) To: Jonathan Corbet Cc: anish kumar, ohad, bjorn.andersson, linux-remoteproc, linux-doc, linux-kernel, kernel test robot On Wed, 23 Oct 2024 at 07:53, Jonathan Corbet <corbet@lwn.net> wrote: > > anish kumar <yesanishhere@gmail.com> writes: > > > This patch series transitions the documentation > > for remoteproc from the staging directory to the > > mainline kernel. It introduces both kernel and > > user-space APIs, enhancing the overall documentation > > quality. > > > > V4: > > Fixed compilation errors and moved documentation to > > driver-api directory. > > > > V3: > > Seperated out the patches further to make the intention > > clear for each patch. > > > > V2: > > Reported-by: kernel test robot <lkp@intel.com> > > Closes: https://lore.kernel.org/oe-kbuild-all/202410161444.jOKMsoGS-lkp@intel.com/ > > So I think you could make better use of kerneldoc comments for a number > of your APIs and structures - a project for the future. I can't judge > the remoteproc aspects of this, but from a documentation mechanics point > of view, this looks about ready to me. In the absence of objections > I'll apply it in the near future. > Please hold off before applying, I will review the content in the coming days. Thanks, Mathieu ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2024-10-25 15:19 UTC | newest] Thread overview: 18+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-10-23 5:33 [PATCH 0/1] remoteproc documentation changes anish kumar 2024-10-23 5:33 ` [PATCH V4 1/7] Remoteproc: Documentation: add the main index anish kumar 2024-10-24 15:42 ` Mathieu Poirier 2024-10-24 17:51 ` anish kumar 2024-10-23 5:33 ` [PATCH V4 2/7] Documentation: remoteproc: move from staging anish kumar 2024-10-23 5:33 ` [PATCH V4 3/7] Documentation: remoteproc: add more information anish kumar 2024-10-24 15:53 ` Mathieu Poirier 2024-10-24 18:17 ` anish kumar 2024-10-25 14:57 ` Mathieu Poirier 2024-10-25 15:13 ` anish kumar 2024-10-25 15:18 ` Mathieu Poirier 2024-10-23 5:33 ` [PATCH V4 4/7] Remoteproc: Documentation: add driver api anish kumar 2024-10-23 5:33 ` [PATCH V4 5/7] Remoteproc: Documentation: add userspace api anish kumar 2024-10-23 5:33 ` [PATCH V4 6/7] MAINTAINERS: remoteproc documentation path update anish kumar 2024-10-23 5:33 ` [PATCH V4 7/7] rpmsg: Documentation: update the path anish kumar 2024-10-23 13:53 ` [PATCH 0/1] remoteproc documentation changes Jonathan Corbet 2024-10-23 14:43 ` Jonathan Corbet 2024-10-23 15:35 ` Mathieu Poirier
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).