Last edited one month ago

STM32MP13 V4L2 camera overview

Applicable for STM32MP13x lines

This article gives information about the Linux® V4L2 camera framework.

1. Framework purpose↑

The V4L2 Linux kernel framework[1] allows the control of both an external camera sensor, and the camera interface. This allows the capture of raw frames in various pixel formats or encoded stream data such as JPEG.

This could be typically used, with the help of other Linux multimedia frameworks, and applications, to take snapshot, to make preview, to make a video recording or even remotely stream images from the camera sensor.

2. System overview↑

CAMERAOverview MP13.png

2.1. Component description↑

  • media-ctl (User space)

A V4L2 utility relying on Media Controller Linux kernel interface[2] aiming to configure, and link each sub devices composing the camera subsystem in order to configure it.

  • Application (User space)

Any application relying on V4L2 Linux kernel interface[3] or libv4l abstraction layer. GStreamer framework provides such application.

  • V4L2 utilities (User space)

A set of tools to test, configure, and use the whole camera subsystem, including the external camera sensor, and the camera interface. v4l2-ctl is one of the most useful utility.

  • V4L2 libraries (libv4l) (User space)

A set of libraries on top of the V4L2 Linux kernel interface[3] which abstract the kernel interface in order to simplify, keep compatibility or add some hooks between V4L-based applications, and the V4L2 kernel interface.

  • V4L2 core (Kernel space)

This layer represents the standard Linux kernel V4L2 Framework.

  • stm32-dcmipp (Kernel space)

This V4L2 DCMIPP Linux device driver handles the DCMIPP hardware block.

  • Camera sensor X driver (Kernel space)

This V4L2 Linux device driver handles the camera sensor X external peripheral, handles some GPIOs lines, and potentially power supplies to power-up/down the camera sensor. The communication with camera sensor is done through the i2c bus.

  • DCMIPP (Hardware)

The Digital Camera Memory Interface Pixel Processor hardware block.

  • Camera sensor X (Hardware)

The camera sensor external peripheral.

2.2. APIs description↑

The V4L2 userland API is documented in the Linux Media subsystem documentation[3]

The V4L2 kernel framework internal API is documented in the V4L2 Kernel Support section of the Linux kernel documentation[4]

The Media Controller API is documented in the Linux Media subsystem documentation[2]

3. Configuration↑

3.1. Kernel configuration↑

The STM32 DCMIPP camera interface and the camera sensor are enabled by default in STMicroelectronics deliveries.

Nevertheless, this is not the case when using the upstream kernel version. In this case, the DCMIPP V4L2 driver can be enabled using Linux kernel menuconfig tool: 

  Device Drivers --->
    <M> Multimedia support --->
      Media drivers --->
        [*] Media platform devices  ---> 
          <M> STM32 Digital Camera Memory Interface Pixel Processor (DCMIPP) support

The external camera sensor connected to the camera interface must also be enabled, here is an example with the GC2145 GalaxyCore camera sensor located on the MB1897 camera daughter board[5] connected to the CN1 camera connector[6] of the STM32MP135x-DK Discovery kit More info green.png: 

  Device Drivers --->
    <M> Multimedia support --->
      Media ancillary drivers --->
        Camera sensor devices  --->
          <M> GalaxyCore GC2145 sensor support


Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

DCMIPP hardware block interconnects with the camera sensor connected on CN1 camera connector[6] of the STM32MP135x-DK Discovery kit More info green.png through the MIPID02 CSI-2 to parallel bridge soldered on STM32MP13 discovery board. Hence, MIPID02 driver must be enabled too: 

  Device Drivers --->
    <M> Multimedia support --->
      Media ancillary drivers --->
        Miscellaneous helper chips  --->
          <M> STMicroelectronics MIPID02 CSI-2 to PARALLEL bridge

3.2. Device tree configuration↑

Refer to DCMIPP device tree configuration article for a complete view of DCMIPP and sensor configuration thanks to Linux kernel device tree mechanism.

4. How to use the framework↑

The use cases described here are enabled using media-ctl, v4l2-ctl, gst-launch or gst-play command line utilities.


4.1. List the video devices and their capabilities↑

List all the available video devices using --list-devices option: 

v4l2-ctl --list-devices

dcmipp_bytecap (platform:dcmipp):
        /dev/video0
        /dev/media0

If several devices are available, use -d option after any v4l2-ctl commands to target a specific device. If -d option is not specified, /dev/video0 is targeted by default.

In order to have information on a specific device, use -D option: 

v4l2-ctl -d /dev/video0 -D

Driver Info:
        Driver name      : dcmipp
        Card type        : stm32_dcmipp
        Bus info         : platform:5a000000.dcmipp
        Driver version   : 6.6.48
        Capabilities     : 0xa4200001
                Video Capture
                I/O MC
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x24200001
                Video Capture
                I/O MC
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : dcmipp
        Model            : DCMIPP MDEV
        Serial           : 
        Bus info         : platform:5a000000.dcmipp
        Media version    : 6.6.48
        Hardware revision: 0x00000010 (16)
        Driver version   : 6.6.48
Interface Info:
        ID               : 0x03000009
        Type             : V4L Video
Entity Info:
        ID               : 0x00000007 (7)
        Name             : dcmipp_dump_capture
        Function         : V4L2 I/O
        Pad 0x01000008   : 0: Sink
          Link 0x0200000d: from remote pad 0x1000006 of entity 'dcmipp_dump_postproc' (Video Scaler): Data, Enablee


4.2. Get the topology of camera subsystem↑

Print the topology of camera subsystem using -p media-ctl option: 

media-ctl -p

Media controller API version 6.6.48

Media device information
------------------------
driver          dcmipp
model           DCMIPP MDEV
serial          
bus info        platform:5a000000.dcmipp
hw revision     0x10
driver version  6.6.48

Device topology
- entity 1: dcmipp_input (2 pads, 2 links, 0 routes)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [stream:0 fmt:RGB565_2X8_LE/640x480 field:none colorspace:srgb]
                <- "st-mipid02 1-0014":2 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:RGB565_2X8_LE/640x480 field:none colorspace:rec709]
                -> "dcmipp_dump_postproc":0 [ENABLED,IMMUTABLE]

- entity 4: dcmipp_dump_postproc (2 pads, 2 links, 0 routes)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [stream:0 fmt:RGB565_2X8_LE/640x480 field:none colorspace:rec709
                 compose.bounds:(0,0)/640x480
                 compose:(0,0)/640x480]
                <- "dcmipp_input":1 [ENABLED,IMMUTABLE]
        pad1: Source
                [stream:0 fmt:RGB565_2X8_LE/640x480 field:none colorspace:rec709
                 crop.bounds:(0,0)/640x480
                 crop:(0,0)/640x480]
                -> "dcmipp_dump_capture":0 [ENABLED,IMMUTABLE]

- entity 7: dcmipp_dump_capture (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "dcmipp_dump_postproc":1 [ENABLED,IMMUTABLE]

- entity 15: st-mipid02 1-0014 (3 pads, 2 links, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev2
        pad0: Sink
                [stream:0 fmt:RGB565_1X16/640x480 field:none colorspace:srgb]
                <- "gc2145 1-003c":0 [ENABLED,IMMUTABLE]
        pad1: Sink
        pad2: Source
                [stream:0 fmt:RGB565_2X8_LE/640x480 field:none colorspace:srgb]
                -> "dcmipp_input":0 [ENABLED,IMMUTABLE]

- entity 21: gc2145 1-003c (1 pad, 1 link, 0 routes)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
        pad0: Source
                [stream:0 fmt:RGB565_1X16/640x480 field:none colorspace:srgb
                 crop.bounds:(0,0)/1600x1200
                 crop:(0,0)/640x480]
                -> "st-mipid02 1-0014":0 [ENABLED,IMMUTABLE]

This gives the current links between camera subdevices, and each subdevice current pads configuration such as, format, resolution, framerate.

Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

This topology can also be displayed in a human readable graphical form thanks to --print-dot option: 

media-ctl --print-dot > graph.dot

Retrieve then this .dot file into your host PC in order to convert it to .png: 

dot -Tpng -Nfontname=Roboto -Nfontsize=10 -Efontname=Roboto -Efontsize=10 graph.dot > graph.png

MediaGraph MP13 GC2145.png

Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

4.3. Camera subsystem setup↑

In order to be able to capture frames from video device node with v4l2-ctl, GStreamer or any other V4L2 application, the camera subsystem must be first configured. To do so use media-ctl --set-v4l2 command giving the subdevice name & pad, the desired format, resolution and framerate: 

media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480 field:none]"

The configuration must be done from source to sink. Below is a configuration allowing to capture RGB565 480x272 frames (the display size of the MP13 discovery board): 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_1X16/640x480 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'st-mipid02 1-0014':2[fmt:RGB565_1X16/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(80,104)/480x272]"
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

Sensor is configured in 640x480 RGB565 at 30 fps, then DCMIPP is configured to crop area 480x272 in the middle of the 640x480 frame output by sensor. The configuration can be read back displaying the topology of graph, see #Get the topology of camera subsystem. Then, any application that read frames from V4L2 video device node can be executed: 

gst-launch-1.0 v4l2src ! video/x-raw, format=RGB16, width=480,height=272, framerate=30/1 ! queue ! waylandsink fullscreen=true
Warning white.png Warning
Application must configure V4L2 device node with exactly the same format, resolution and framerate that the ones being set at camera subsystem setup !

4.4. Controlling camera sensor↑

In order to control camera sensor, its corresponding entity device node must be found in the topology. To do so, the first entity of the graph must be found, ie the entity having no sink pad: 

- entity 21: ov5640 1-003c (1 pad, 1 link, 0 routes)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
        pad0: Source
                [stream:0 fmt:RGB565_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range
                 crop.bounds:(0,0)/2624x1964
                 crop:(16,14)/2592x1944]
                -> "st-mipid02 1-0014":0 [ENABLED,IMMUTABLE]

Here it is /dev/v4l-subdev3.

Use then v4l2-ctl with -L option to get the list of supported controls: 

v4l2-ctl -d /dev/v4l-subdev3 -L

User Controls

                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0

Image Source Controls

              vertical_blanking 0x009e0901 (int)    : min=0 max=8191 step=1 default=12 value=12
            horizontal_blanking 0x009e0902 (int)    : min=0 max=4095 step=1 default=304 value=304

Image Processing Controls

                 link_frequency 0x009f0901 (intmenu): min=0 max=2 default=0 value=0 (120000000 0x7270e00) flags=read-only
                                0: 120000000 (0x7270e00)
                                1: 192000000 (0xb71b000)
                                2: 240000000 (0xe4e1c00)
                     pixel_rate 0x009f0902 (int64)  : min=30000000 max=60000000 step=1 default=30000000 value=30000000 flags=read-only
                   test_pattern 0x009f0903 (menu)   : min=0 max=8 default=0 value=0 (Disabled)
                                0: Disabled
                                1: Colored patterns
                                2: Uniform white
                                3: Uniform yellow
                                4: Uniform cyan
                                5: Uniform green
                                6: Uniform magenta
                                7: Uniform red
                                8: Uniform black
Info white.png Information
"value=" field returns the current value of the control
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

The control value can be changed thanks to --set-ctrl option, for example: 

v4l2-ctl -d /dev/v4l-subdev3 --set-ctrl test_pattern=1

The control value can be changed dynamically. In the following example, the color bar is enabled/disabled while preview is running:

  • Start preview in background
gst-launch-1.0 v4l2src ! "video/x-raw, format=RGB16, width=480, height=272, framerate=(fraction)30/1" ! queue ! waylandsink fullscreen=true -e &
  • Then alternate the color bar activation or not
v4l2-ctl -d /dev/v4l-subdev3 --set-ctrl test_pattern=1;sleep 1;v4l2-ctl -d /dev/v4l-subdev3 --set-ctrl test_pattern=0;sleep 1;v4l2-ctl -d /dev/v4l-subdev3 --set-ctrl test_pattern=1;sleep 1;v4l2-ctl -d /dev/v4l-subdev3 --set-ctrl test_pattern=0;killall gst-launch-1.0
  • Horizontal/vertical flip can also be changed while preview is running:
v4l2-ctl -d /dev/v4l-subdev3 --set-ctrl horizontal_flip=1;sleep 2;v4l2-ctl -d /dev/v4l-subdev3 --set-ctrl horizontal_flip=0;sleep 2;v4l2-ctl -d /dev/v4l-subdev3 --set-ctrl vertical_flip=1;sleep 2;v4l2-ctl -d /dev/v4l-subdev3 --set-ctrl vertical_flip=0;killall gst-launch-1.0

4.5. Set the camera sensor pixel format, resolution and framerate↑

Camera sensor only supports a discrete set of formats, resolutions and framerates that can be get thanks to v4l2-ctl options --list-subdev-mbus-codes, --list-subdev-framesizes and --list-subdev-frameintervals: 

v4l2-ctl -d /dev/v4l-subdev3 --list-subdev-mbus-codes

ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0,stream=0)
        0x200f: MEDIA_BUS_FMT_UYVY8_1X16
        0x2010: MEDIA_BUS_FMT_VYUY8_1X16
        0x2011: MEDIA_BUS_FMT_YUYV8_1X16
        0x2012: MEDIA_BUS_FMT_YVYU8_1X16
        0x1017: MEDIA_BUS_FMT_RGB565_1X16
        0x3002: MEDIA_BUS_FMT_SGRBG8_1X8
        0x3014: MEDIA_BUS_FMT_SRGGB8_1X8
        0x3001: MEDIA_BUS_FMT_SBGGR8_1X8
        0x3013: MEDIA_BUS_FMT_SGBRG8_1X8
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

Frame size might depend on frame format, so precise it when asking for frame size: 

v4l2-ctl -d /dev/v4l-subdev3 --list-subdev-framesizes pad=0,code=0x1017


ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0,stream=0)
        Size Range: 640x480 - 640x480
        Size Range: 1280x720 - 1280x720
        Size Range: 1600x1200 - 1600x1200
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

Framerate depends on resolution and format, so precise them when asking for frame intervals: 

v4l2-ctl -d /dev/v4l-subdev3 --list-subdev-frameintervals pad=0,width=640,height=480,code=0x1007

ioctl: VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (pad=0)
	Interval: 0.033s (30.000 fps)
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

Now that camera sensor capabilities are known, the wanted configuration can be set following instructions described in chapter camera subsystem setup.

For example here to make a VGA preview at 30 fps: 


 media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_1X16/640x480 field:none]"
 media-ctl -d /dev/media0 --set-v4l2 "'st-mipid02 1-0014':2[fmt:RGB565_1X16/640x480]"
 media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':1[fmt:RGB565_2X8_LE/640x480]"
 media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:RGB565_2X8_LE/640x480]"

gst-launch-1.0 v4l2src ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! autovideosink
Warning white.png Warning
Please note that GStreamer parameters must fit exactly the same format, resolution and framerate that the ones being set with media-ctl ! Refer to #Camera subsystem setup for more information.
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

4.6. Downscale and crop↑

Two features of DCMIPP can help to reduce resolution: downsize and crop, these two features are configured on postproc subdev.

Downscale is basic and consists in skipping 1 pixel over 2 in width and/or height, allowing to divide by 2 in width and/or height. The wanted resolution must be set on postproc subdev using compose property of sink pad respecting /2 ratio. Here is an example to get 320x240 frames from 640x480 input frames: 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_1X16/640x480 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'st-mipid02 1-0014':2[fmt:RGB565_1X16/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[compose:(0,0)/320x240]"
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.
gst-launch-1.0 v4l2src ! video/x-raw, format=RGB16, width=320,height=240, framerate=30/1 ! queue ! autovideosink
Info white.png Information
Please note that (top,left) argument of compose property is ignored by driver


Crop can be made at 16 pixels precision (output is rounded to 16 inferior multiple). The wanted area must be set using crop property of postproc subdev source pad. Here is an example to crop a 480x272 area centered inside the 640x480 input frame: 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_1X16/640x480 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'st-mipid02 1-0014':2[fmt:RGB565_1X16/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(80,104)/480x272]"
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.
gst-launch-1.0 v4l2src ! video/x-raw, format=RGB16, width=480,height=272, framerate=30/1 ! queue ! autovideosink

Downscale and crop can be combined together. Here is an example to get small 320x176 frames from 1280x720 input frames: 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_1X16/1280x720 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'st-mipid02 1-0014':2[fmt:RGB565_1X16/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':1[fmt:RGB565_2X8_LE/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:RGB565_2X8_LE/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[compose:(0,0)/640x360]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(160,92)/320x176]"
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.
gst-launch-1.0 v4l2src ! video/x-raw, format=RGB16, width=320,height=176, framerate=30/1 ! queue ! autovideosink

Refer to the selection chapter of the sub device interface documentation[7] to get more details on how to control scaling and crop.

4.7. Grab a raw frame↑

Capture a VGA RGB565 raw frame on disk: 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_1X16/640x480 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'st-mipid02 1-0014':2[fmt:RGB565_1X16/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:RGB565_2X8_LE/640x480]"

v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=RGBP --stream-mmap --stream-count=1 --stream-to=grab-640x480-rgb565.raw
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

In order to display it, this raw frame must be converted first to JPEG: 

gst-launch-1.0 filesrc location= grab-640x480-rgb565.raw blocksize=614400 ! "video/x-raw, format=(string)RGB16, width=(int)640, height=(int)480, framerate=(fraction)30/1" ! videoconvert ! jpegenc ! filesink location=grab-640x480-rgb565.jpeg

Then weston-image utility can be used to display this JPEG file: 

weston-image grab-640x480-rgb565.jpeg

4.8. Fullscreen preview↑


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_1X16/640x480 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'st-mipid02 1-0014':2[fmt:RGB565_1X16/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(80,104)/480x272]"

gst-launch-1.0 v4l2src ! "video/x-raw, format=RGB16, width=480, height=272, framerate=(fraction)15/1" ! queue ! autovideosink -e
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

4.9. Take a picture in JPEG format↑

The GC2145 sensor doesn't support JPEG frame format natively.

Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

4.10. Preview in YUV422↑

Capture VGA YUV422 frames and display them using GStreamer : 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:YUYV8_1X16/640x480 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'st-mipid02 1-0014':2[fmt:YUYV8_1X16/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':0[fmt:YUYV8_2X8/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:YUYV8_2X8/640x480]"

gst-launch-1.0 v4l2src ! "video/x-raw, format=YUY2, width=640,height=480, framerate=(fraction)15/1" ! queue ! videoconvert ! queue ! autovideosink
Click on the right to see/hide another example with OV5640 Omnivision (end of life/deprecated) camera sensor.

4.11. Grab a raw-bayer frame↑

Capture a 720p raw-bayer frame on disk. The GC2145 only currently support the RGGB format. 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:SRGGB8_1X8/1280x720 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'st-mipid02 1-0014':2[fmt:SRGGB8_1X8/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':1[fmt:SRGGB8_1X8/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:SRGGB8_1X8/1280x720]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:SRGGB8_1X8/1280x720]"

v4l2-ctl --set-fmt-video=width=1280,height=720,pixelformat=BA81 --stream-mmap --stream-count=1 --stream-to=grab-1280x720-bayer.raw

In order to display it, this raw-bayer frame must be converted first to JPEG: 

gst-launch-1.0 filesrc location= grab-1280x720-bayer.raw blocksize=921600 ! "video/x-bayer, format=(string)rggb, width=(int)1280, height=(int)720, framerate=(fraction)30/1" ! bayer2rgb ! videoconvert ! jpegenc ! filesink location=grab-1280x720-bayer.jpeg

Then weston-image utility can be used to display this JPEG file: 

weston-image grab-1280x720-bayer.jpeg
Click on the right to see/hide an example with OV5640 Omnivision (end of life/deprecated) camera sensor.

4.12. Raw-bayer capture with preview↑

Camera subsystem is setup in order that VGA raw-bayer camera sensor output is reduced to 320x240 using crop (downscale is not possible with raw-bayer): 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:SRGGB8_1X8/640x480 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'st-mipid02 1-0014':2[fmt:SRGGB8_1X8/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':1[fmt:SRGGB8_1X8/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[fmt:SRGGB8_1X8/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':1[crop:(160,120)/320x240]"

Then GStreamer is used to grab frames on disk while previewing: 

gst-launch-1.0 v4l2src num-buffers=100 ! "video/x-bayer, format=rggb, width=320,height=240, framerate=(fraction)30/1" ! queue ! tee name=tee ! queue ! fpsdisplaysink name="capturerate" sync=false text-overlay=false video-sink="multifilesink location=320x240_%d-bayer.raw max-files=10"  tee. ! queue ! videorate max-rate=15 ! bayer2rgb ! queue ! videoconvert ! queue ! waylandsink sync=false -v

Raw-bayer frames are captured on disk looping over 10 files to not saturate the disk:

-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x240_90-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x240_91-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x240_92-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x240_93-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x240_94-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x240_95-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x240_96-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x240_97-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x240_98-bayer.raw
-rw-r--r-- 1 root root 56320 Sep 20 12:38 320x240_99-bayer.raw

Note that capture occurs at 30 fps while preview is done at 15 fps to not overload system with software debayering, see trace below: 

/GstPipeline:pipeline0/GstFPSDisplaySink:capturerate: last-message = rendered: 145, dropped: 0, current: 30.41, average: 30.63
/GstPipeline:pipeline0/GstFPSDisplaySink:capturerate: last-message = rendered: 161, dropped: 0, current: 30.41, average: 30.60
/GstPipeline:pipeline0/GstFPSDisplaySink:capturerate: last-message = rendered: 177, dropped: 0, current: 30.41, average: 30.59
Click on the right to see/hide an example with OV5640 Omnivision (end of life/deprecated) camera sensor.

5. How to trace and debug↑

5.1. How to monitor↑

5.1.1. Check of devicetree configuration↑

Here are some commands to verify that DCMIPP is enabled, check which camera sensor is used and log other details about devicetree configuration: 

cd ~ # go to $HOME directory
echo "#!/bin/bash" > dtdumpentry.sh;echo "hexdump -e '\"=\"'  -e '20/1 \"%c\"\"\t\"' -e '20/1 \"%02x\"\"\n\"' \$1" >> dtdumpentry.sh;chmod +x dtdumpentry.sh
echo "#!/bin/bash" > dtdump.sh;echo "find  \$1* -type f -print0 -exec ./dtdumpentry.sh {} \;" >> dtdump.sh;chmod +x dtdump.sh

cd ~ # go to $HOME directory
rm ~/devicetree.txt
echo "[devicetree]" >> ~/devicetree.txt
echo "|-[dcmipp]" >> devicetree.txt
./dtdump.sh /proc/device-tree/soc/bus@5c007000/dcmipp@5a000000 | sed 's/\/proc\/device-tree\/soc\//| |-/' >> devicetree.txt
echo "|" >> devicetree.txt
echo "|-[camera:" | tr -d "\n" >> devicetree.txt
cat /proc/device-tree/soc/bus@5c007000/i2c*/camera*/compatible >> devicetree.txt
echo "]" >> devicetree.txt
./dtdump.sh /proc/device-tree/soc/bus@5c007000/i2c*/camera* -type f -print0 -exec ./dtdump.sh {} \; | sed 's/\/proc\/device-tree\/soc\//| |-/' >> devicetree.txt
echo "|" >> devicetree.txt
echo "|-[mipi bridge:" | tr -d "\n" >> devicetree.txt
cat /proc/device-tree/soc/bus@5c007000/i2c*/*mipi*/compatible >> devicetree.txt
echo "]" >> devicetree.txt
./dtdump.sh /proc/device-tree/soc/bus@5c007000/i2c*/*mipi* -type f -print0 -exec ./dtdump.sh {} \; | sed 's/\/proc\/device-tree\/soc\//| |-/' >> devicetree.txt
echo "" >> devicetree.txt
cat ~/devicetree.txt # or cat -t ~/devicetree.txt

[devicetree]
|-[dcmipp]
| |-bus@5c007000/dcmipp@5a000000/compatible=st,stm32mp13-dcmipp 73742c73746d33326d7031332d64636d69707000
| |-bus@5c007000/dcmipp@5a000000/clocks=��      000000060000008f                        
| |-bus@5c007000/dcmipp@5a000000/access-controllers='�  0000002700000004                        
| |-bus@5c007000/dcmipp@5a000000/resets=�5�     00000006000035c1                        
| |-bus@5c007000/dcmipp@5a000000/pinctrl-1=`    00000060                                
| |-bus@5c007000/dcmipp@5a000000/port/endpoint/hsync-active=    00000000                                
| |-bus@5c007000/dcmipp@5a000000/port/endpoint/vsync-active=    00000000                                
| |-bus@5c007000/dcmipp@5a000000/port/endpoint/remote-endpoint=a        00000061                                
| |-bus@5c007000/dcmipp@5a000000/port/endpoint/pclk-max-frequency='     07270e00                                
| |-bus@5c007000/dcmipp@5a000000/port/endpoint/bus-width=       00000008                                
| |-bus@5c007000/dcmipp@5a000000/port/endpoint/pclk-sample=     00000000                                
| |-bus@5c007000/dcmipp@5a000000/port/endpoint/phandle=<        0000003c                                
| |-bus@5c007000/dcmipp@5a000000/port/endpoint/name=endpoint    656e64706f696e7400                      
| |-bus@5c007000/dcmipp@5a000000/port/name=port 706f727400                              
| |-bus@5c007000/dcmipp@5a000000/clock-names=kclk       6b636c6b00                              
| |-bus@5c007000/dcmipp@5a000000/status=okay    6f6b617900                              
| |-bus@5c007000/dcmipp@5a000000/interrupts=O�  000000000000004f00000004                
| |-bus@5c007000/dcmipp@5a000000/phandle=�      000000cf                                
| |-bus@5c007000/dcmipp@5a000000/reg=Z� 5a00000000000400                        
| |-bus@5c007000/dcmipp@5a000000/pinctrl-0=_    0000005f                                
| |-bus@5c007000/dcmipp@5a000000/name=dcmipp    64636d69707000                          
| |-bus@5c007000/dcmipp@5a000000/pinctrl-names=defaultsleep     64656661756c7400736c65657000            
|
|-[camera:ovti,ov5640]
| |-bus@5c007000/i2c@4c006000/camera@3c/compatible=ovti,ov5640  6f7674692c6f763536343000                
| |-bus@5c007000/i2c@4c006000/camera@3c/powerdown-gpios=3��     000000330000000300000001                
| |-bus@5c007000/i2c@4c006000/camera@3c/DOVDD-supply=%  00000025                                
| |-bus@5c007000/i2c@4c006000/camera@3c/clocks==        0000003d                                
| |-bus@5c007000/i2c@4c006000/camera@3c/port/endpoint/data-lanes=��     0000000100000002                        
| |-bus@5c007000/i2c@4c006000/camera@3c/port/endpoint/clock-lanes=      00000000                                
| |-bus@5c007000/i2c@4c006000/camera@3c/port/endpoint/phandle=ϻ 000000bb                                
| |-bus@5c007000/i2c@4c006000/camera@3c/port/endpoint/name=endpoint     656e64706f696e7400                      
| |-bus@5c007000/i2c@4c006000/camera@3c/port/name=port  706f727400                              
| |-bus@5c007000/i2c@4c006000/camera@3c/clock-names=xclk        78636c6b00                              
| |-bus@5c007000/i2c@4c006000/camera@3c/status=disabled 64697361626c656400                      
| |-bus@5c007000/i2c@4c006000/camera@3c/reset-gpios=3�� 000000330000000400000001                
| |-bus@5c007000/i2c@4c006000/camera@3c/phandle=�       000000ba                                
| |-bus@5c007000/i2c@4c006000/camera@3c/reg=<   0000003c                                
| |-bus@5c007000/i2c@4c006000/camera@3c/name=camera     63616d65726100                          
|
|-[mipi bridge:st,st-mipid02]
| |-bus@5c007000/i2c@4c006000/stmipi@14/compatible=st,st-mipid02        73742c73742d6d69706964303200            
| |-bus@5c007000/i2c@4c006000/stmipi@14/clocks=9        00000039                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/VDDIN-supply=:  0000003a                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/clock-names=xclk        78636c6b00                              
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/#address-cells=�  00000001                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/#size-cells=      00000000                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@2/endpoint/hsync-active=     00000000                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@2/endpoint/vsync-active=     00000000                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@2/endpoint/remote-endpoint=< 0000003c                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@2/endpoint/bus-width=00000008                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@2/endpoint/pclk-sample=      00000000                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@2/endpoint/phandle=a 00000061                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@2/endpoint/name=endpoint     656e64706f696e7400                      
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@2/reg=�      00000002                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@2/name=port  706f727400                              
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@0/endpoint/remote-endpoint=; 0000003b                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@0/endpoint/data-lanes=��     0000000100000002                        
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@0/endpoint/phandle=> 0000003e                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@0/endpoint/lane-polarities=  000000000000000000000000                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@0/endpoint/name=endpoint     656e64706f696e7400                      
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@0/reg=       00000000                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/port@0/name=port  706f727400                              
| |-bus@5c007000/i2c@4c006000/stmipi@14/ports/name=ports        706f72747300                            
| |-bus@5c007000/i2c@4c006000/stmipi@14/status=okay     6f6b617900                              
| |-bus@5c007000/i2c@4c006000/stmipi@14/reset-gpios=3�� 000000330000000200000001                
| |-bus@5c007000/i2c@4c006000/stmipi@14/VDDE-supply=:   0000003a                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/phandle=�       000000b8                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/reg=�   00000014                                
| |-bus@5c007000/i2c@4c006000/stmipi@14/name=stmipi     73746d69706900

5.1.2. Camera subsystem configuration debugging↑

Refer to section #Get the topology of camera subsystem for basics on how to get and set camera subsystem configuration. Find below some debugging tips to help configuration handling.

5.1.2.1. subdevice configuration syntax error↑

media-ctl utility returns an error when a configuration syntax mistake is made. Example here with erroneous entity name: 

media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]"
Unable to setup formats: Invalid argument (22)

In order to have details on the exact error, add -v option: 

media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]" -v
no such entity "dcmipp_dump_postpr"
'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]
 ^

Another example with pad number mistake (only pad number 0 or 1 are allowed): 

media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480]" -v
No pad '4' on entity "dcmipp_dump_postproc". Maximum pad number is 1
 'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480]
                        ^

Example with format mistake (BA81 is a raw-bayer V4L2 pixel format while subdevice expects V4L2 mbus code such as SBGGR8_1X8): 

media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:BA81/640x480]" -v
Invalid pixel code 'BA81'
 'dcmipp_dump_postproc':0[fmt:BA81/640x480]
                         ^
5.1.2.2. subdevice negotiation↑

Requested configuration may not be fully accepted by the subdevice, in this case, the subdevice configures itself with the part of configuration it can handle. media-ctl -v option when setting configuration shows the effective subdevice configuration.

Here is an example with 5 fps framerate not accepted by the subdevice and 30 fps selected instead: 

media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@1/5 field:none]" -v
Setting up format RGB565_2X8_BE 640x480 on pad gc2145 1-003c/0
Format set: RGB565_2X8_BE 640x480
Setting up frame interval 1/5 on pad gc2145 1-003c/0
Frame interval set: 1/30
Warning white.png Warning
No error or warning is returned by media-ctl, so take car to read back configuration with --set-v4l2 with -v or --get-v4l2 to check if it is fully accepted. Several negotiation rounds may be needed to find a fully compatible configuration.


Example with resolution negotiation (4000x4000 reduced to 2592x1944) 

media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/4000x4000@1/15 field:none]" -v
Setting up format RGB565_2X8_BE 4000x4000 on pad gc2145 1-003c/0
Format set: RGB565_2X8_BE 1600x1200


Example with format negotiation (AYUV8_1X32 not supported, defaults to RGB565_2X8_LE): 

media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:AYUV8_1X32/640x480]" -v
Setting up format  AYUV8_1X32 640x480 on pad dcmipp_dump_postproc/0
Format set: RGB565_2X8_LE

5.2. How to trace↑

5.2.1. V4L2 userland API tracing↑

Tracing of V4L2 userland API[3] can be enabled using command: 

echo 0x3 >  devices/platform/soc/5c007000.etzpc/5a000000.dcmipp/video4linux/video0/dev_debug

Traces are output in kernel log buffer: 

dmesg

[  655.479016] video0: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_bytecap, bus=platform:dcmipp, version=0x00050f00, capabilities=0x852000011
[  655.481591] video0: VIDIOC_G_FMT: type=vid-cap, width=640, height=480, pixelformat=RGBP little-endian (0x50424752), field=none, bytesperl0
[  655.481656] video0: VIDIOC_ENUM_FMT: index=0, type=vid-cap, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, des'
[  655.481696] video0: VIDIOC_ENUM_FMT: index=1, type=vid-cap, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, des'
[  655.481732] video0: VIDIOC_ENUM_FMT: index=2, type=vid-cap, flags=0x0, pixelformat=YVYU little-endian (0x55595659), mbus_code=0x0000, des'
[  655.481767] video0: VIDIOC_ENUM_FMT: index=3, type=vid-cap, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, des'
[  655.481802] video0: VIDIOC_ENUM_FMT: index=4, type=vid-cap, flags=0x0, pixelformat=VYUY little-endian (0x59555956), mbus_code=0x0000, des'
[  655.481836] video0: VIDIOC_ENUM_FMT: index=5, type=vid-cap, flags=0x0, pixelformat=GREY little-endian (0x59455247), mbus_code=0x0000, des'
[  655.481873] video0: VIDIOC_ENUM_FMT: index=6, type=vid-cap, flags=0x0, pixelformat=BA81 little-endian (0x31384142), mbus_code=0x0000, des'
[  655.481909] video0: VIDIOC_ENUM_FMT: index=7, type=vid-cap, flags=0x0, pixelformat=GBRG little-endian (0x47524247), mbus_code=0x0000, des'
[  655.481944] video0: VIDIOC_ENUM_FMT: index=8, type=vid-cap, flags=0x0, pixelformat=GRBG little-endian (0x47425247), mbus_code=0x0000, des'
[  655.481981] video0: VIDIOC_ENUM_FMT: index=9, type=vid-cap, flags=0x0, pixelformat=RGGB little-endian (0x42474752), mbus_code=0x0000, des'
[  655.482017] video0: VIDIOC_ENUM_FMT: index=10, type=vid-cap, flags=0x1, pixelformat=JPEG little-endian (0x4745504a), mbus_code=0x0000, de'
[  655.482061] video0: VIDIOC_S_FMT: type=vid-cap, width=2592, height=1952, pixelformat=JPEG little-endian (0x4745504a), field=none, bytespe0
[  655.526124] video0: VIDIOC_REQBUFS: count=4, type=vid-cap, memory=mmap
[  655.526221] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4
[  655.526272] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  655.526296] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4
[  655.526339] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  655.526359] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=2, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4
[  655.526401] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  655.526420] video0: VIDIOC_QUERYBUF: 00:00:00.000000 index=3, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory4
[  655.526462] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  655.527192] video0: VIDIOC_G_FMT: type=vid-cap, width=2592, height=1952, pixelformat=JPEG little-endian (0x4745504a), field=none, bytespe0
[  655.536625] video0: VIDIOC_STREAMON: type=vid-cap
[  655.536701] video0: VIDIOC_G_FMT: type=vid-cap, width=2592, height=1952, pixelformat=JPEG little-endian (0x4745504a), field=none, bytespe0
[  655.892309] video0: VIDIOC_STREAMOFF: type=vid-cap
[  655.892435] video0: VIDIOC_STREAMOFF: type=vid-cap
[  655.901481] video0: VIDIOC_REQBUFS: count=0, type=vid-cap, memory=mmap

5.2.2. V4L2 core framework tracing↑

Tracing of the V4L2 core framework[4] can be enabled using commands: 

echo 0x3 > /sys/module/videobuf2_common/parameters/debug
echo 0x3 > /sys/module/videobuf2_v4l2/parameters/debug

Traces are output in kernel log buffer: 

dmesg

[ 1218.626235] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 0, plane 0 offset 0x00000000
[ 1218.642445] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 1, plane 0 offset 0x004d4000
[ 1218.656398] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 2, plane 0 offset 0x009a8000
[ 1218.669843] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 3, plane 0 offset 0x00e7c000
[ 1218.677503] videobuf2_common: [cap-7b50f8bf] __vb2_queue_alloc: allocated 4 buffers, 1 plane(s) each
[ 1218.688361] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 0, plane 0 successfully mapped
[ 1218.696744] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 1, plane 0 successfully mapped
[ 1218.705291] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 2, plane 0 successfully mapped
[ 1218.712487] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 3, plane 0 successfully mapped
[ 1218.722474] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 0 succeeded
[ 1218.730528] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 1 succeeded
[ 1218.738153] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 2 succeeded
[ 1218.746012] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 3 succeeded
[ 1218.756237] videobuf2_common: [cap-7b50f8bf] vb2_core_streamon: successful
[ 1218.761969] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.822167] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.828383] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 0, state: dequeued
[ 1218.838812] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 0 succeeded
[ 1218.846549] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.855624] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.862118] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 1, state: dequeued
[ 1218.872406] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 1 succeeded
[ 1218.880029] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.889460] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.896756] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 2, state: dequeued
[ 1218.905383] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 2 succeeded
[ 1218.912309] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.923080] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.930392] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 3, state: dequeued
[ 1218.944499] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 3 succeeded
[ 1218.952210] videobuf2_common: [cap-7b50f8bf] vb2_core_streamoff: successful
[ 1218.959921] videobuf2_common: [cap-7b50f8bf] vb2_core_streamoff: successful
[ 1218.968800] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 0
[ 1218.978419] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 1
[ 1218.988204] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 2
[ 1218.997501] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 3

5.2.3. DCMIPP V4L2 kernel driver tracing↑

DCMIPP dynamic debug traces[8] can be enabled using command: 

echo "module stm32_dcmipp +p" > /sys/kernel/debug/dynamic_debug/control

Here is an example with a 5Mp jpeg capture: 


media-ctl -d /dev/media0 --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@1/30 field:none]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':0[fmt:RGB565_2X8_BE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_input':0[fmt:RGB565_2X8_LE/640x480]"
media-ctl -d /dev/media0 --set-v4l2 "'dcmipp_dump_postproc':0[fmt:RGB565_2X8_LE/640x480]"

 v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=RGBP --stream-mmap --stream-count=1 --stream-to=grab-640x480-rgb565.raw

dmesg

[ 6185.212240] dcmipp 5a000000.dcmipp: dcmipp_dump_capture: format update: old:480x272 (0x50424752, 8, 1, 2, 0) new:640x480 (0x50424752, 8, 1, 2, 0)
[ 6185.212629] dcmipp 5a000000.dcmipp: Setup queue, count=4, size=614400
[ 6185.216997] dcmipp 5a000000.dcmipp: Setup [0] phy=0xd8a00000 size=614400
[ 6185.217059] dcmipp 5a000000.dcmipp: Setup [1] phy=0xd8b00000 size=614400
[ 6185.217082] dcmipp 5a000000.dcmipp: Setup [2] phy=0xd8c00000 size=614400
[ 6185.217102] dcmipp 5a000000.dcmipp: Setup [3] phy=0xd8d00000 size=614400
[ 6185.217137] dcmipp 5a000000.dcmipp: Queue [0] d8b69f13 phy=0xd8a00000
[ 6185.217158] dcmipp 5a000000.dcmipp: Queue [1] b0942ca1 phy=0xd8b00000
[ 6185.217171] dcmipp 5a000000.dcmipp: Queue [2] 2d865547 phy=0xd8c00000
[ 6185.217183] dcmipp 5a000000.dcmipp: Queue [3] a70ff5be phy=0xd8d00000
[ 6185.217343] dcmipp 5a000000.dcmipp: CLR DCMIPP_P0FCTCR 0x00000003
[ 6185.217363] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0FCTCR 0x00000000
[ 6185.217374] dcmipp 5a000000.dcmipp: SET DCMIPP_P0FCTCR 0x00000000
[ 6185.217384] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0FCTCR 0x00000000
[ 6185.217395] dcmipp 5a000000.dcmipp: CLR DCMIPP_P0PPCR 0x00000180
[ 6185.217405] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0PPCR 0x00000000
[ 6185.217416] dcmipp 5a000000.dcmipp: CLR DCMIPP_P0PPCR 0x00000400
[ 6185.217426] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0PPCR 0x00000000
[ 6185.217436] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0SCSTR 0x00000000
[ 6185.217446] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0SCSZR 0x00000000
[ 6185.217457] dcmipp 5a000000.dcmipp: crop to 640x480
[ 6185.217468] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0SCSTR 0x00000000
[ 6185.217478] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0SCSZR 0x81e00140
[ 6185.217489] dcmipp 5a000000.dcmipp: "dcmipp_dump_postproc" is started
[ 6185.217502] dcmipp 5a000000.dcmipp: WR  DCMIPP_PRCR 0x02220000
[ 6185.217513] dcmipp 5a000000.dcmipp: WR  DCMIPP_CMCR 0x00000000
[ 6185.217524] dcmipp 5a000000.dcmipp: SET DCMIPP_PRCR 0x00004000
[ 6185.217535] dcmipp 5a000000.dcmipp: WR  DCMIPP_PRCR 0x02224000
[ 6185.217545] dcmipp 5a000000.dcmipp: "dcmipp_input" is started
[ 6185.219357] dcmipp 5a000000.dcmipp: "st-mipid02 1-0014" is started
[ 6185.375507] dcmipp 5a000000.dcmipp: "gc2145 1-003c" is started
[ 6185.375543] dcmipp 5a000000.dcmipp: SET DCMIPP_P0FSCR 0x80000000
[ 6185.375560] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0FSCR 0x80000000
[ 6185.375572] dcmipp 5a000000.dcmipp: Start with next [0] d8b69f13 phy=0xd8a00000
[ 6185.375592] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0PPM0AR1 0xd8a00000
[ 6185.375602] dcmipp 5a000000.dcmipp: Write [0] d8b69f13 phy=0xd8a00000
[ 6185.375615] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0DCLMTR 0x80025800
[ 6185.375625] dcmipp 5a000000.dcmipp: SET DCMIPP_P0FCTCR 0x00000008
[ 6185.375635] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0FCTCR 0x00000008
[ 6185.375646] dcmipp 5a000000.dcmipp: SET DCMIPP_CMIER 0x00008600
[ 6185.375656] dcmipp 5a000000.dcmipp: WR  DCMIPP_CMIER 0x00008600
[ 6185.626841] dcmipp 5a000000.dcmipp: RD  DCMIPP_CMSR2 0x00000500
[ 6185.626890] dcmipp 5a000000.dcmipp: WR  DCMIPP_CMFCR 0x00000400
[ 6185.626985] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0PPM0AR1 0xd8b00000
[ 6185.627012] dcmipp 5a000000.dcmipp: Write [1] b0942ca1 phy=0xd8b00000
[ 6185.660542] dcmipp 5a000000.dcmipp: RD  DCMIPP_CMSR2 0x00000300
[ 6185.660591] dcmipp 5a000000.dcmipp: WR  DCMIPP_CMFCR 0x00000200
[ 6185.660652] dcmipp 5a000000.dcmipp: RD  DCMIPP_P0DCCNTR 0x00096000
[ 6185.660693] dcmipp 5a000000.dcmipp: Done  [0] d8b69f13 phy=0xd8a00000
[ 6185.660821] dcmipp 5a000000.dcmipp: Queue [0] d8b69f13 phy=0xd8a00000
[ 6185.660953] dcmipp 5a000000.dcmipp: "dcmipp_dump_postproc" is stopped
[ 6185.660978] dcmipp 5a000000.dcmipp: CLR DCMIPP_PRCR 0x00004000
[ 6185.660998] dcmipp 5a000000.dcmipp: WR  DCMIPP_PRCR 0x02220000
[ 6185.661014] dcmipp 5a000000.dcmipp: "dcmipp_input" is stopped
[ 6185.662357] dcmipp 5a000000.dcmipp: "st-mipid02 1-0014" is stopped
[ 6185.662419] dcmipp 5a000000.dcmipp: "gc2145 1-003c" is stopped
[ 6185.662454] dcmipp 5a000000.dcmipp: CLR DCMIPP_CMIER 0x00008600
[ 6185.662474] dcmipp 5a000000.dcmipp: WR  DCMIPP_CMIER 0x00000000
[ 6185.662491] dcmipp 5a000000.dcmipp: CLR DCMIPP_P0FCTCR 0x00000008
[ 6185.662506] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0FCTCR 0x00000000
[ 6185.662523] dcmipp 5a000000.dcmipp: CLR DCMIPP_P0FSCR 0x80000000
[ 6185.662539] dcmipp 5a000000.dcmipp: WR  DCMIPP_P0FSCR 0x00000000
[ 6185.662566] dcmipp 5a000000.dcmipp: RD  DCMIPP_PRSR 0x00030000
[ 6185.662582] dcmipp 5a000000.dcmipp: [DCMIPP_PRSR]  =0x00030000
[ 6185.662597] dcmipp 5a000000.dcmipp: RD  DCMIPP_P0SR 0x00000007
[ 6185.662612] dcmipp 5a000000.dcmipp: [DCMIPP_P0SR] =0x00000007
[ 6185.662627] dcmipp 5a000000.dcmipp: RD  DCMIPP_P0DCCNTR 0x00096000
[ 6185.662642] dcmipp 5a000000.dcmipp: [DCMIPP_P0DCCNTR]=0x00096000
[ 6185.662657] dcmipp 5a000000.dcmipp: RD  DCMIPP_CMSR1 0x00000003
[ 6185.662672] dcmipp 5a000000.dcmipp: [DCMIPP_CMSR1] =0x00000003
[ 6185.662686] dcmipp 5a000000.dcmipp: RD  DCMIPP_CMSR2 0x00000100
[ 6185.662701] dcmipp 5a000000.dcmipp: [DCMIPP_CMSR2] =0x00000100
[ 6185.662722] dcmipp 5a000000.dcmipp: Stop streaming, errors=0 (overrun=0, limit=0, nactive=0), underrun=0, vsync=1, frame=1, buffers=1, it=2

5.3. How to debug↑

5.3.1. Errors↑

Errors are unconditionally traced in kernel log: 

dmesg
[ 1947.932915] dcmipp-bytecap dcmipp-bytecap.3.auto: Stop streaming, errors=1 (overrun=0, limit=0), vsync=2, frame=2, buffers=2, it=4


5.3.2. Memory tracking↑

Frames require large chunks of contiguous memory. They are allocated by V4L2 framework through DMA backend. Those allocations can be traced using: 


echo "module dma_contiguous +p" > /sys/kernel/debug/dynamic_debug/control
echo "module videobuf2_dma_contig +p" > /sys/kernel/debug/dynamic_debug/control

Here is the trace after a VGA YUV422 capture. 

[11311.617688] vb2_dc_mmap: mapped dma addr 0xf1900000 at 0xb3b6a000, size 614400
[11311.617986] vb2_dc_mmap: mapped dma addr 0xf1a00000 at 0xb3ad4000, size 614400
[11311.618071] vb2_dc_mmap: mapped dma addr 0xf1b00000 at 0xb3a3e000, size 614400
[11311.764146] vb2_dc_mmap: mapped dma addr 0xf1c00000 at 0xb307c000, size 614400

4 frames of VGA YUV422 frames: 640x480x2=614400 bytes

6. Source code location↑

6.1. User space↑

6.2. Kernel space↑

7. References↑

  1. ↑ Information about V4L2 Linux kernel framework on wikipedia.
  2. ↑ Jump up to: 2.0 2.1 Linux Media Infrastructure userspace API » Part IV - Media Controller API
  3. ↑ Jump up to: 3.0 3.1 3.2 3.3 Linux Media Infrastructure userspace API » Part I - Video for Linux API
  4. ↑ Jump up to: 4.0 4.1 Media subsystem kernel internal API » 1. Video4Linux devices
  5. ↑ MB1897 camera daughter board
  6. ↑ Jump up to: 6.0 6.1 6.2 CN1 Camera sensor connector
  7. ↑ Linux Media Infrastructure userspace API » Part I - Video for Linux API » 4. Interfaces » 4.13. Sub-device Interface » 4.13.3.2. Selections: cropping, scaling and composition
  8. ↑ How to use the kernel dynamic debug



Retrieved from "https://wiki.stmicroelectronics.cn/stm32mpu/index.php?title=STM32MP13_V4L2_camera_overview&oldid=114998"