Linux RPMsg framework overview

Applicable for

STM32MP15x lines, STM32MP25x lines

---

This article gives information about the Linux^® Remote Processor Messaging (RPMsg) framework. The RPMsg framework is a virtio-based messaging bus that allows a local processor to communicate with remote processors available on the system.

1. Framework purpose

The Linux^® RPMsg framework is a messaging mechanism implemented on top of the virtio framework to communicate with a remote processor. It is based on virtio vrings to send/receive messages to/from the remote CPU over shared memory.

The vrings are uni-directional, one vring is dedicated to messages sent to the remote processor, and the other vring is used for messages received from the remote processor. In addition, shared buffers are created in memory space visible to both processors.

The Mailbox framework is then used to notify cores when new messages are waiting in the shared buffers.

Relying on these frameworks, The RPMsg framework implements a communication based on channels. The channels are identified by a textual name and have a local (“source”) RPMsg address, and a remote (“destination”) RPMsg address.

On the remote processor side, a RPMsg framework must also be implemented. Several solutions exist, we recommend using OpenAMP

Overviews of the communication mechanisms involved are available at:

• OpenAMP wiki

2. System overview

2.1. Component description

• remoteproc: The remoteproc framework allows different platforms/architectures to control (power on, load firmware, power off) the remote processors. This framework also adds rpmsg virtio devices for remote processors that support the RPMsg protocol. More details on this framework are available in the remote proc framework page.

• virtio: VirtIO framework that supports virtualization. It provides an efficient transport layer based on a shared ring buffer (vring). For more details about this framework please refer to the link below:
  • Virtio: An I/O virtualization framework for Linux
  • virtio introduction - SlideShare

• rpmsg: A virtio-based messaging bus that allows kernel drivers to communicate with remote processors available on the system. It provides the messaging infrastructure, facilitating the writing of wire-protocol messages by client drivers. Client drivers can then, in turn, expose appropriate user space interfaces if needed.

• rpmsg_client_driver is the client driver that implements a service associated to the remote processor. This driver is probed by the RPMsg framework when an associated service is requested by a remote processor using a "new service announcement" RPMsg message.

2.2. RPMsg definitions

To implement an RPMsg client, channel and endpoint concepts need to be understood for a good understanding of the framework.

• RPMsg channel:
  

An RPMsg client is associated to a communication channel between master and remote processors. This RPMsg client is identified by the textual service name, registered in the RPMsg framework. The communication channel is established when a match is found between the local service name registered and the remote service announced.

• RPMsg endpoint:
  

The RPMsg endpoints provide logical connections through an RPMsg channel. An RPMsg endpoint has a unique source address and associated call back function, allowing the user to bind multiple endpoints on the same channel. When a client driver creates an endpoint with the local address, all the inbound messages with a destination address equal to the endpoint local address are routed to that endpoint. Notice that every channel has a default endpoint, which enables applications to communicate without even creating new endpoints.

2.3. API description

The User API usage is described in Linux kernel RPMsg documentation

3. Configuration

3.1. Kernel configuration

The RPMsg framework is automatically enabled when the remote STM32_RPROC configuration is activated.

3.2. Device tree configuration

No device tree configuration is needed to enable the RPmsg protocol. The memory region allocated for RPMsg buffers is declared in the remote proc framework device tree. Nevertheless some device tree configurations can be be needed for specific RPMsg devices. This is the case for the RPmsg I2C and the RPMsg_interrupt_controller devices.

4. How to use the framework

The RPMsg framework is used by linux driver client. For details and an example of a simple client, please refer to associated Linux documentation

4.1. RPMsg TTY

The rpmsg_tty driver simulates a serial link for communication between the host processor running Linux and the coprocessor. It allows the user applications to communicate with the coprocessor thanks to a standard TTY sysfs device interface.

4.1.1. Linux kernel configuration

Activate RPMSG_TTY in kernel configuration with Linux Menuconfig tool: Menuconfig or how to configure kernel.

 Device Drivers  --->
   Character devices  --->
     <*> RPMSG tty driver

4.1.2. RPMSg channel name

The associated RPMsg service name is "rpmsg_tty",

4.1.3. User space interface

"/dev/ttyRPMSG<X>", where <X> is the instance number from 0 to 32.

4.1.4. Application sample

• open:

int openTtyRpmsg(int ttyNb)
{
    char devName[50];
    sprintf(devName, "/dev/ttyRPMSG%d", ttyNb);
    FdRpmsg = open(devName, O_RDWR |  O_NOCTTY | O_NONBLOCK);
    if (FdRpmsg < 0) {
        printf(" Error opening ttyRPMSG%d, err=-%d\n", ttyNb, errno);
        return (errno * -1);
    }
}

• close:

int closeTtyRpmsg(int FdRpmsg)
{
    close(FdRpmsg);
    return 0;
}

• write:

int writeTtyRpmsg(int FdRpmsg, int len, char* pData)
{
    int result = 0;
    if (FdRpmsg < 0) {
        printf("CA7 : Error writing ttyRPMSG, fileDescriptor is not set\n");
        return FdRpmsg;
    }
 
    result = write(FdRpmsg, pData, len);
    return result;
}

• read:

int readTtyRpmsg(int FdRpmsg, int len, char* pData)
{
    int byte_rd, byte_avail;
    int result = 0;
    if (FdRpmsg < 0) {
        printf("CA7 : Error reading ttyRPMSG, fileDescriptor is not set\n");
        return FdRpmsg;
    }
    ioctl(FdRpmsg, FIONREAD, &byte_avail);
    if (byte_avail > 0) {
        if (byte_avail >= len) {
            byte_rd = read(FdRpmsg, pData, len);
        } else {
            byte_rd = read(FdRpmsg, pData, byte_avail);
        }
        result = byte_rd;
    } else {
        result = 0;
    }
    return result;
}

• control:

int ControlTtyRpmsg(int FdRpmsg, int modeRaw)
{
    struct termios tiorpmsg;

    /* get current port settings */
    tcgetattr(FdRpmsg,&tiorpmsg);
    if (modeRaw) {
        /* Terminal controls are deactivated, used to transfer raw data without control */
        memset(&tiorpmsg, 0, sizeof(tiorpmsg));
        tiorpmsg.c_cflag = (CS8 | CLOCAL | CREAD);
        tiorpmsg.c_iflag = IGNPAR;
        tiorpmsg.c_oflag = 0;
        tiorpmsg.c_lflag = 0;
        tiorpmsg.c_cc[VTIME] = 0;
        tiorpmsg.c_cc[VMIN] = 1;
        cfmakeraw(&tiorpmsg);
    } else {
        /* ECHO off, other bits unchanged */
        tiorpmsg.c_lflag &= ~ECHO;
        /*do not convert LF to CR LF */
        tiorpmsg.c_oflag &= ~ONLCR;
    }
    if (tcsetattr(FdRpmsg, TCSANOW, &tiorpmsg) < 0) {
        printf("Error %d in %s  tcsetattr", errno, __func__);
        return (errno * -1);
    }
    return 0;
}

For more examples, please refer to:

• the "OpenAMP_TTY_echo" application example in the list of STM32CubeMP15 available projects and in the list of STM32CubeMP2 available projects,
• the How to exchange data buffers with the coprocessor article that provides source code example for the direct buffer exchange mode.

4.1.5. Related STM32Cube_MPU implementation

Associated STM32CubeMP15 Package or STM32CubeMP2 Package virtual UART driver is available in Middlewares/Third_Party/OpenAMP/virtual_driver/virt_uart.c

4.2. RPMsg char

The rpmsg_char driver allows user applications to create local endpoints with a char device sysfs interface to communicate with a predefined remote processor endpoint (predefined means with a known endpoint address).

4.2.1. Linux kernel configuration

Activate RPMSG_CHAR in kernel configuration with Linux Menuconfig tool: Menuconfig or how to configure kernel.

 Device Drivers  --->
   Rpmsg drivers  --->
     <*> RPMSG device interface

4.2.2. RPMSg channel name

The associated RPMsg service name is "rpmsg-raw". The rpmsg_char can also be used for endpoint to endpoint communication with predefined addresses using the /dev/rpmg_ctrl0 interface

4.2.3. User space interface

• "/dev/rpmg_ctrl0" to instantiate /dev/rpmsg0 sysfs device interface,
• "/dev/rpmg0" sysfs device interface for communication.

4.2.4. Application sample

Following code examples implement routines to create and to use a local endpoint with a specified address.

• create local endpoint

struct rpmsg_endpoint_info {
	char name[32];
	uint32_t src;
	uint32_t dst;
};

#define RPMSG_CREATE_EPT_IOCTL  _IOW(0xb5, 0x1, struct rpmsg_endpoint_info)

/*
 * argv[1] : "/dev/rpmsg_ctrl<X>", where <X> is the instance number of the device
 * argv[2] : a name for the device
 * argv[3] : local endpoint address 
 * argv[4] : remote endpoint address to communicate with
 */
int createEndPoint(int argc, char **argv)
{
	struct rpmsg_endpoint_info ept;
	int ret;
	int fd;
	char *endptr;

	if (argc != 5)
		exit(1);

	fd = open(argv[1], O_RDWR);
	if (fd < 0)
		err(1, "failed to open %s\n", argv[1]);

	strncpy(ept.name, argv[2], sizeof(ept.name));
	ept.name[sizeof(ept.name)-1] = '\0';

	ept.src = strtoul(argv[3], &endptr, 10);

	if (*endptr || ept.src <= 0)
		exit(1);

	ept.dst = strtoul(argv[4], &endptr, 10);

	if (*endptr  || ept.src <= 0)
		exit(1);

	ret = ioctl(fd, RPMSG_CREATE_EPT_IOCTL, &ept);
	if (ret < 0) {
		fprintf(stderr, "failed to create endpoint");
		exit(1);
	}

	close(fd);
	return 0;
}

• write and read on the local endpoint

/*
 * argv[1] : "/dev/rpmsg<X>", where <X> is the instance number of the device
 */
int ping(int argc, char **argv)
{
	char buffer[256];
	int ret;
	int fd;

	if (argc != 2 )
		exit(1);

	fd = open(argv[1], O_RDWR);
	if (fd < 0)
		err(1, "failed to open %s\n", argv[1]);

	sprintf(buffer, "ping %s",  argv[1]);
	ret = write(fd, buffer,strlen(buffer));
	if (ret < 0) {
		fprintf(stderr, "failed to write endpoint  %s", argv[1]);
		exit(1);
	}
	ret = read(fd, buffer,256);
	if (ret < 0) {
		fprintf(stderr, "failed to read endpoint %s", argv[1]);
		exit(1);
	}
	fprintf(stderr, "message received: \"%s\"\n", buffer);

	close(fd);

	return 0;
}

• remove local endpoint

/*
 * argv[1] : "/dev/rpmsg<X>", where <X> is the instance number of the device
 */
int destroyEndPoint(int argc, char **argv)
{
	int ret;
	int fd;

	if (argc != 2)
		exit(1);

	fd = open(argv[1], O_RDWR);
	if (fd < 0)
		err(1, "failed to open %s\n", argv[1]);

	ret = ioctl(fd, RPMSG_DESTROY_EPT_IOCTL, NULL);
	if (ret < 0) {
		fprintf(stderr, "failed to destroy endpoint");
		exit(1);
	}

	close(fd);
	return 0;
}

Refer to |rpmsg-utils for more exemples.

4.3. RPMsg I2C

The rpmsg_i2c driver simulates an I2C bus for communication between the host processor running Linux and the coprocessor. It allows the linux I2C drivers or user applications to communicate with the coprocessor thanks to the standard I2C interface.

4.3.1. Linux kernel configuration

Activate RPMSG_I2C in kernel configuration with Linux Menuconfig tool: Menuconfig or how to configure kernel.

 Device Drivers  --->
    I2C support  --->
        I2C Hardware Bus support   --->
             <*> RPMSG I2C driver

4.3.2. RPMSg channel name

The associated RPMsg service name is "rpmsg_i2c".

4.3.3. Device tree

The i2c_rpmsg bus is declared as a standard hardware I2C bus in the device tree and described in Documentation/devicetree/bindings/i2c/i2c-rpmsg.yaml .

An exemple of the rpmsg_I2C use is available in arch/arm64/boot/dts/st/stm32mp257f-ev1.dts .

Information

The "i2c_rpmsg" bus and "device_1" will be probed by the "rpmsg bus" when the "rpmsg_i2c" channel will be created by the remote processor firmware. If the remote processor is stopped the I2C bus is not visible.

4.3.4. Related STM32Cube MPU implementation

Associated STM32CubeMP2 Package virtual I2C driver is available in Middlewares/Third_Party/OpenAMP/virtual_driver/virt_i2c.c

4.4. RPMsg interrupt controller

The irq_rpmsg driver simulates an IRQ controller that is able to dispatch Interupt generating by the remote processor. It can be used for instance to simulate a GPIO interruption.

4.4.1. Linux kernel configuration

Activate RPMSG_IRQ in kernel configuration with Linux Menuconfig tool: Menuconfig or how to configure kernel.

 Device Drivers  --->
    IRQ chip support  --->
        <*> RPMSG irq driver

4.4.2. RPMSg channel name

The associated RPMsg service name is "rpmsg_intc".

4.4.3. Device tree

The irq_rpmsg bus is declared as a standard interrupt controller in the device tree and described in Documentation/devicetree/bindings/interrupt-controller/rpmsg,intc.yaml .

An exemple of the irq_rpmsg use is available in arch/arm64/boot/dts/st/stm32mp257f-ev1.dts .

Information

The irq_rpmsg will be probed by the "rpmsg bus" when the "rpmsg_intc" channel will be created by the remote processor firmware. If the remoteprocessor is stopped the interrupt controller is not visible.

4.4.4. API description

Refer to interrupt API description page.

4.4.5. Related STM32Cube MPU implementation

Associated STM32CubeMP2 Package virtual interrupt driver is available in Middlewares/Third_Party/OpenAMP/virtual_driver/virt_intc.c

5. How to trace and debug the framework

5.1. How to trace

RPMsg and virtio dynamic debug traces can be added using the following commands:

echo -n 'file virtio_rpmsg_bus.c +p' > /sys/kernel/debug/dynamic_debug/control
echo -n 'file virtio_ring.c +p' > /sys/kernel/debug/dynamic_debug/control

Linux^® is a registered trademark of Linus Torvalds.

Remote processor messaging

Central processing unit

Open Asymmetric Multi-Processing (See https://www.openampproject.org/)

Application programming interface

TeleTYpewriter

System file system (see https://en.wikipedia.org/wiki/Sysfs for more details)

Terminal input output structure

Microprocessor unit

Universal asynchronous receiver/transmitter

Inter-integrated circuit (Bi-directional 2-wire bus standard for efficient inter-IC control.)

General-purpose input/output (A realization of open ended transmission between devices on an embedded level. These pins available on a processor can be programmed to be used to either accept input or provide output to external devices depending on user desires and applications requirements.)

Linux^® is a registered trademark of Linus Torvalds.

Remote processor messaging

Central processing unit

Open Asymmetric Multi-Processing (See https://www.openampproject.org/)

Application programming interface

TeleTYpewriter

System file system (see https://en.wikipedia.org/wiki/Sysfs for more details)

Terminal input output structure

Microprocessor unit

Universal asynchronous receiver/transmitter

Inter-integrated circuit (Bi-directional 2-wire bus standard for efficient inter-IC control.)

General-purpose input/output (A realization of open ended transmission between devices on an embedded level. These pins available on a processor can be programmed to be used to either accept input or provide output to external devices depending on user desires and applications requirements.)