Registered User Tag: 2017 source edit |
Registered User (→Grab a raw-bayer frame: remove conversion + display because GStreamer not able to handle raw 10 bits...) Tag: 2017 source edit |
||
| (10 intermediate revisions by 3 users not shown) | |||
| Line 3: | Line 3: | ||
|MPUs checklist=STM32MP13x, STM32MP15x, STM32MP25x | |MPUs checklist=STM32MP13x, STM32MP15x, STM32MP25x | ||
}} | }} | ||
This article gives information about the Linux<sup>®</sup> V4L2 camera framework. | This article gives information about the Linux<sup>®</sup> V4L2 camera framework. | ||
== Framework purpose == | == Framework purpose == | ||
| Line 9: | Line 9: | ||
</ref> allows the control of both an external camera sensor, and the camera interface. This allows the capture of raw frames in various pixel formats or encoded stream data such as JPEG. | </ref> allows the control of both an external camera sensor, and the camera interface. This allows the capture of raw frames in various pixel formats or encoded stream data such as JPEG. | ||
This could be typically used, with the help of other Linux multimedia frameworks | This could be typically used, with the help of other Linux multimedia frameworks and applications, to take a snapshot, to make a preview, to make a video recording or even remotely stream images from the camera sensor. | ||
== System overview == | == System overview == | ||
| Line 15: | Line 15: | ||
=== Component description === | === Component description === | ||
*'''media-ctl''' ( | *'''media-ctl''' (user space) | ||
A V4L2 utility relying on | A V4L2 utility relying on media controller Linux kernel interface<ref name=API_MC/> aiming to configure and link each sub devices composing the camera subsystem to configure it. | ||
*'''Application''' ( | *'''Application''' (user space) | ||
Any application relying on V4L2 Linux kernel interface<ref name=API/> or libv4l abstraction layer. [[GStreamer overview|GStreamer]] framework provides such application. | Any application relying on V4L2 Linux kernel interface<ref name=API/> or libv4l abstraction layer. [[GStreamer overview|GStreamer]] framework provides such application. | ||
*'''V4L2 utilities''' ( | *'''V4L2 utilities''' (user space) | ||
A set of tools to test, configure, and use the whole camera subsystem, including the external camera sensor | A set of tools to test, configure, and use the whole camera subsystem, including the external camera sensor and the camera interface. [[v4l2-ctl]] is one of the most useful utilities. | ||
*'''V4L2 libraries (libv4l)''' ( | *'''V4L2 libraries (libv4l)''' (user space) | ||
A set of libraries on top of the V4L2 Linux kernel interface<ref name=API/> which abstract the kernel interface | A set of libraries on top of the V4L2 Linux kernel interface<ref name=API/> which abstract the kernel interface to simplify, keep compatibility, or add some hooks between V4L-based applications and the V4L2 kernel interface. | ||
*'''V4L2 core''' ( | *'''V4L2 core''' (kernel space) | ||
This layer represents the standard Linux kernel V4L2 | This layer represents the standard Linux kernel V4L2 framework. | ||
*'''stm32-dcmipp''' ( | *'''stm32-dcmipp''' (kernel space) | ||
This V4L2 DCMIPP Linux device driver handles the [[DCMIPP internal peripheral|DCMIPP hardware block]]. | This V4L2 DCMIPP Linux device driver handles the [[DCMIPP internal peripheral|DCMIPP hardware block]]. | ||
*'''Camera sensor X driver''' ( | *'''Camera sensor X driver''' (kernel space) | ||
This V4L2 Linux device driver handles the camera sensor X external peripheral, | This V4L2 Linux device driver handles the camera sensor X external peripheral, some GPIOs lines, and potentially power supplies to power-up/down the camera sensor. | ||
The communication with camera sensor is done through the | The communication with the camera sensor is done through the I<sup>2</sup>C bus. | ||
*'''DCMIPP''' (Hardware) | *'''DCMIPP''' (Hardware) | ||
The [[DCMIPP internal peripheral| | The [[DCMIPP internal peripheral|DCMIPP hardware block]]. | ||
*'''Camera sensor X''' (Hardware) | *'''Camera sensor X''' (Hardware) | ||
| Line 44: | Line 44: | ||
=== APIs description === | === APIs description === | ||
The V4L2 userland API is documented in the Linux | The V4L2 userland API is documented in the Linux media subsystem documentation<ref name=API>{{DocSource | domain=Linux kernel | path=userspace-api/media/v4l/v4l2.html | text=Linux Media Infrastructure userspace API » Part I - Video for Linux API}}</ref>. | ||
The V4L2 kernel framework internal API is documented in the V4L2 | The V4L2 kernel framework internal API is documented in the V4L2 kernel support section of the Linux kernel documentation<ref name=core>{{DocSource | domain=Linux kernel | path=driver-api/media/v4l2-core.html | text=Media subsystem kernel internal API » 1. Video4Linux devices}}</ref>. | ||
The | The media controller API is documented in the Linux media subsystem documentation<ref name=API_MC>{{DocSource | domain=Linux kernel | path=userspace-api/media/mediactl/media-controller.html | text=Linux Media Infrastructure userspace API » Part IV - Media Controller API}}</ref>. | ||
== Configuration == | == Configuration == | ||
| Line 54: | Line 54: | ||
The STM32 DCMIPP camera interface and the camera sensor are enabled by default in STMicroelectronics deliveries. | The STM32 DCMIPP camera interface and the camera sensor are enabled by default in STMicroelectronics deliveries. | ||
Nevertheless, this is not the case when using the upstream kernel version. In this case, the DCMIPP V4L2 driver can be enabled using [[Menuconfig or how to configure kernel|Linux kernel menuconfig tool]]: | Nevertheless, this is not the case when using the upstream kernel version. In this case, the DCMIPP V4L2 driver can be enabled using the [[Menuconfig or how to configure kernel|Linux kernel menuconfig tool]]: | ||
<pre> | <pre> | ||
Device Drivers ---> | Device Drivers ---> | ||
| Line 63: | Line 63: | ||
</pre> | </pre> | ||
The external camera sensor connected to the camera interface must also be enabled. Here is an example with the IMX335 SONY camera sensor located on the MB1854 camera daughter board<ref>[[MB1854 | MB1854 camera daughter board]]</ref> connected to the CN4 CSI camera connector<ref name=CN4>[[STM32MP257x-EV1_-_hardware_description#MB1936-CN4|CN4 Camera sensor connector]]</ref> of the {{Board | type=257F-EV1}}: | |||
The external camera sensor connected to the camera interface must also be enabled | |||
<pre> | <pre> | ||
Device Drivers ---> | Device Drivers ---> | ||
| Line 74: | Line 73: | ||
=== Device tree configuration === | === Device tree configuration === | ||
Refer to [[DCMIPP device tree configuration]] article for a complete view of DCMIPP and sensor configuration thanks to Linux kernel device tree mechanism. | Refer to the [[DCMIPP device tree configuration]] article for a complete view of DCMIPP and sensor configuration thanks to Linux kernel device tree mechanism. | ||
== How to use the framework == | == How to use the framework == | ||
The use cases described here are enabled using [[media-ctl]], [[v4l2-ctl]], [https://gstreamer.freedesktop.org/documentation/tools/gst-launch.html# gst-launch] or [[Gst-play|gst-play]] command line utilities. | The use cases described here are enabled using [[media-ctl]], [[v4l2-ctl]], [https://gstreamer.freedesktop.org/documentation/tools/gst-launch.html# gst-launch], or [[Gst-play|gst-play]] command line utilities. | ||
{{WestonLaunch}} | {{WestonLaunch}} | ||
| Line 83: | Line 82: | ||
=== List the video devices and their capabilities === | === List the video devices and their capabilities === | ||
List all the available video devices using {{highlight|'''--list-devices'''}} option: | List all the available video devices using {{highlight|'''--list-devices'''}} option: | ||
{{Board$}} v4l2-ctl {{highlight|'''--list-devices'''}} | {{Board$}}v4l2-ctl {{highlight|'''--list-devices'''}} | ||
<pre> | <pre> | ||
st,stm32mp25-vdec-dec (platform: hantro-vpu): | st,stm32mp25-vdec-dec (platform: hantro-vpu): | ||
| Line 100: | Line 99: | ||
</pre> | </pre> | ||
If several devices are available, use {{HighlightParam|'''-d'''}} option after any [[v4l2-ctl]] commands to target a specific device. If {{HighlightParam|'''-d'''}} option is not specified, /dev/video0 is targeted by default. | If several devices are available, use the {{HighlightParam|'''-d'''}} option after any [[v4l2-ctl]] commands to target a specific device. If the {{HighlightParam|'''-d'''}} option is not specified, /dev/video0 is targeted by default. | ||
To have information on a specific device, use the {{highlight|'''-D'''}} option: | |||
{{Board$}} v4l2-ctl {{HighlightParam|'''-d'''}} /dev/video3 {{highlight|'''-D'''}} | {{Board$}}v4l2-ctl {{HighlightParam|'''-d'''}} /dev/video3 {{highlight|'''-D'''}} | ||
<pre> | <pre> | ||
root@stm32mp25:~# v4l2-ctl -d /dev/video2 -D | root@stm32mp25:~# v4l2-ctl -d /dev/video2 -D | ||
| Line 141: | Line 140: | ||
Link 0x02000032: from remote pad 0x1000014 of entity 'dcmipp_main_postproc' (Video Pixel Formatter): Data, Enabled, Immutable | Link 0x02000032: from remote pad 0x1000014 of entity 'dcmipp_main_postproc' (Video Pixel Formatter): Data, Enabled, Immutable | ||
</pre> | </pre> | ||
=== Get the topology of camera subsystem === | === Get the topology of the camera subsystem === | ||
Print the topology of camera subsystem using {{highlight|'''-p'''}} [[media-ctl]] option: | Print the topology of the camera subsystem using the {{highlight|'''-p'''}} [[media-ctl]] option: | ||
{{Board$}} media-ctl -d platform:dcmipp {{highlight|'''-p'''}} | {{Board$}}media-ctl -d platform:dcmipp {{highlight|'''-p'''}} | ||
<pre> | <pre> | ||
Media controller API version 6.1.82 | Media controller API version 6.1.82 | ||
| Line 298: | Line 295: | ||
-> "stm32_csi2host.48020000.csi2hos":0 [ENABLED,IMMUTABLE] | -> "stm32_csi2host.48020000.csi2hos":0 [ENABLED,IMMUTABLE] | ||
</pre> | </pre> | ||
This gives the current links between camera subdevices | This gives the current links between camera subdevices and each subdevice current pads configuration, such as, format, resolution, framerate. | ||
This topology can also be displayed in a human readable graphical form thanks to {{highlight|'''--print-dot'''}} option: | This topology can also be displayed in a human readable graphical form thanks to the {{highlight|'''--print-dot'''}} option: | ||
{{Board$}} media-ctl -d platform:dcmipp {{highlight|'''--print-dot'''}} > graph.dot | {{Board$}}media-ctl -d platform:dcmipp {{highlight|'''--print-dot'''}} > graph.dot | ||
Retrieve | Retrieve this .dot file into your host PC to convert it to .png: | ||
{{PC$}} dot -Tpng -Nfontname=Roboto -Nfontsize=10 -Efontname=Roboto -Efontsize=10 graph.dot > graph.png | {{PC$}} dot -Tpng -Nfontname=Roboto -Nfontsize=10 -Efontname=Roboto -Efontsize=10 graph.dot > graph.png | ||
[[File:MediaGraph_MP25_IMX335.png|frame|center]] | [[File:MediaGraph_MP25_IMX335.png|frame|center]] | ||
This picture shows the | This picture shows the two possible camera interfaces: MIPI CSI-2 or parallel, and the three pipes topology: dump pipe, main pipe, and auxiliary pipe: | ||
* dump pipe allows to directly read without any processing the camera sensor data whatever those data are: raw-bayer, RGB, YUV, JPEG or even image statistic data. The dump pipe output camera sensor specific data can be read from /dev/video1 capture node. | * The dump pipe allows to directly read without any processing the camera sensor data whatever those data are: raw-bayer, RGB, YUV, JPEG, or even image statistic data. The dump pipe output camera sensor specific data can be read from the /dev/video1 capture node. | ||
* main pipe is the most featured DCMIPP pipe, including frame skipping, | * The main pipe is the most featured DCMIPP pipe, including frame skipping, de-Bayering, image correction thanks to [[How_to_use_the_DCMIPP_ISP|DCMIPP ISP]], scaling, color conversion, ... The main pipe output pixel data can be read from the /dev/video2 capture node while the image statistic data can be read from the /dev/video4 capture node. Refer to [[How to use the DCMIPP ISP]] article to get more details on how to handle image statistic data. | ||
* | * The auxiliary pipe can perform limited operations such as scaling and color conversion with some restrictions regarding to main pipe. It can also be dynamically linked to the output of ISP to inherit of image correction. The auxiliary pipe output pixel data can be read from the /dev/video3 capture node. | ||
=== Camera subsystem setup === | === Camera subsystem setup === | ||
To be able to capture frames from a video device node with v4l2-ctl, GStreamer, or any other V4L2 application, the camera subsystem must be first configured. To do so use the [[media-ctl]] {{highlight|'''--set-v4l2'''}} command giving the {{STDarkGreen|'''subdevice name'''}} and {{STYellow|'''pad'''}} the desired {{STDarkBlue|'''format'''}}, {{STPurple|'''resolution'''}}, and {{HighlightParam|'''framerate'''}}: | |||
{{Board$}} | {{Board$}} | ||
| Line 319: | Line 316: | ||
The configuration must be done from source to sink. Below is a configuration allowing to capture RGB565 640x480 frames: | The configuration must be done from source to sink. Below is a configuration allowing to capture RGB565 640x480 frames: | ||
{{Board$}} | {{Board$}} | ||
media-ctl -d platform:dcmipp --set-v4l2 "'{{STDarkGreen|'''imx335 0-001a'''}}':0[fmt:{{STDarkBlue|'''SRGGB10_1X10'''}}/{{STPurple|'''2592x1940'''}}]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/{{STPurple|'''640x480'''|tint=50}}]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:{{STDarkBlue|'''RGB565_2X8_LE'''|tint=50}}/{{STPurple|'''640x480'''|tint=50}}]" | |||
export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture") | |||
The camera sensor is configured in {{STPurple|'''5 MP'''}} {{STDarkBlue|'''raw-bayer 10 bits'''}} at {{HighlightParam|30 fps}}, which is the only configuration supported by the IMX335 for now. Then DCMIPP is configured to convert to {{STDarkBlue|'''RGB565'''|tint=50}} and downscale to {{STPurple|'''640x480'''|tint=50}}. | |||
The configuration can be read back displaying the topology of graph | The configuration can be read back displaying the topology of the graph. See [[#Get the topology of camera subsystem]]. | ||
Then, any application that read frames from V4L2 video device node can be executed: | Then, any application that read frames from V4L2 video device node can be executed: | ||
{{Board$}} gst-launch-1.0 v4l2src device= | {{Board$}}gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format={{STDarkBlue|'''RGB16'''|tint=50}}, width={{STPurple|'''640'''|tint=50}},height={{STPurple|'''480'''|tint=50}}, framerate={{HighlightParam|'''30/1'''}} ! queue ! waylandsink fullscreen=true | ||
{{Warning| | {{Warning|The application must configure V4L2 device node with exactly the same {{STDarkBlue|'''format'''|tint=50}}, {{STPurple|'''resolution'''|tint=50}}, and {{HighlightParam|'''framerate'''}} that the ones being set at camera subsystem setup!}} | ||
{{Info| At this stage you can observe that the preview image is very dark and greenish, see next chapter to perform image correction.}} | |||
{{Info| At this stage you can observe that preview image is very dark and greenish, see next chapter to perform image correction.}} | |||
=== Image correction === | === Image correction === | ||
To correct an image coming from a raw-bayer camera sensor using DCMIPP ISP, use the STMicroelectronics [[DCMIPP ISP example utility]] command line utility with the {{highlight|-w}} option '''while capture is running''': | |||
{{Board$}} | {{Board$}} | ||
gst-launch-1.0 v4l2src device= | gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! waylandsink fullscreen=true & | ||
sleep 1 | sleep 1 | ||
/usr/local/demo/bin/{{highlight|dcmipp-isp-ctrl -i0 -g}} | /usr/local/demo/bin/{{highlight|dcmipp-isp-ctrl -i0 -g}} | ||
sleep 1 | sleep 1 | ||
kill %1 | kill %1 | ||
Image correction parameters are kept from a capture to another one, so you | Image correction parameters are kept from a capture to another one, so you do not have to recall the dcmipp-isp-ctrl utility if ambient light conditions have not changed. | ||
More details on image correction can be found in the dedicated article [[How to use the DCMIPP ISP]]. | More details on image correction can be found in the dedicated article [[How to use the DCMIPP ISP]]. | ||
=== Controlling camera sensor === | === Controlling camera sensor === | ||
To control a camera sensor its corresponding entity device node must be found in the topology. To do so, the first entity of the graph must be found, that is, the entity having no sink pad: | |||
<pre> | <pre> | ||
| Line 358: | Line 355: | ||
Here it is '''/dev/v4l-subdev6'''. | Here it is '''/dev/v4l-subdev6'''. | ||
Use then [[v4l2-ctl]] with {{highlight|'''-L'''}} option to get the list of supported controls: | Use then [[v4l2-ctl]] with the {{highlight|'''-L'''}} option to get the list of supported controls: | ||
{{Board$}} v4l2-ctl -d '''/dev/v4l-subdev6''' {{highlight|'''-L'''}} | {{Board$}}v4l2-ctl -d '''/dev/v4l-subdev6''' {{highlight|'''-L'''}} | ||
<pre> | <pre> | ||
User Controls | User Controls | ||
| Line 376: | Line 373: | ||
{{Info|'''"value{{=}}"''' field returns the current value of the control}} | {{Info|'''"value{{=}}"''' field returns the current value of the control}} | ||
<br> | <br> | ||
The control value can be changed thanks to {{highlight|'''--set-ctrl'''}} option, for example: | The control value can be changed thanks to the {{highlight|'''--set-ctrl'''}} option, for example: | ||
{{Board$}} v4l2-ctl -d /dev/v4l-subdev6 {{highlight|'''--set-ctrl analogue_gain{{=}}10'''}} | {{Board$}}v4l2-ctl -d /dev/v4l-subdev6 {{highlight|'''--set-ctrl analogue_gain{{=}}10'''}} | ||
The control value can be changed dynamically. In the following example, the analog gain is increased while preview is running: | The control value can be changed dynamically. In the following example, the analog gain is increased while the preview is running: | ||
* Start preview in background | * Start the preview in the background. | ||
{{Board$}} gst-launch-1.0 v4l2src device= | {{Board$}}gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! waylandsink fullscreen=true -e > /dev/null 2>&1 & | ||
* Then increase gain every second | * Then increase the gain every second. | ||
{{Board$}} v4l2-ctl -d /dev/v4l-subdev6 {{highlight|'''--set-ctrl analogue_gain{{=}}0'''}};sleep 1;v4l2-ctl -d /dev/v4l-subdev6 {{highlight|'''--set-ctrl analogue_gain{{=}}20'''}};sleep 1;v4l2-ctl -d /dev/v4l-subdev6 {{highlight|'''--set-ctrl analogue_gain{{=}}40'''}};sleep 1;v4l2-ctl -d /dev/v4l-subdev6 {{highlight|'''--set-ctrl analogue_gain{{=}}0'''}};killall gst-launch-1.0 | {{Board$}}v4l2-ctl -d /dev/v4l-subdev6 {{highlight|'''--set-ctrl analogue_gain{{=}}0'''}};sleep 1;v4l2-ctl -d /dev/v4l-subdev6 {{highlight|'''--set-ctrl analogue_gain{{=}}20'''}};sleep 1;v4l2-ctl -d /dev/v4l-subdev6 {{highlight|'''--set-ctrl analogue_gain{{=}}40'''}};sleep 1;v4l2-ctl -d /dev/v4l-subdev6 {{highlight|'''--set-ctrl analogue_gain{{=}}0'''}};killall gst-launch-1.0 | ||
=== Set the camera sensor pixel format, resolution and framerate === | === Set the camera sensor pixel format, resolution, and framerate === | ||
To control a camera sensor its corresponding entity device node must be found in the topology. To do so, the first entity of the graph must be found, that is, the entity having no sink pad: | |||
- entity 68: {{STDarkGreen|'''imx335 0-001a'''}} (1 pad, 1 link) | - entity 68: {{STDarkGreen|'''imx335 0-001a'''}} (1 pad, 1 link) | ||
type V4L2 subdev subtype Sensor flags 0 | type V4L2 subdev subtype Sensor flags 0 | ||
| Line 396: | Line 393: | ||
Here it is '''/dev/v4l-subdev6'''. | Here it is '''/dev/v4l-subdev6'''. | ||
Alternatively, media-ctl command be used to get the information, using following command: | Alternatively, the media-ctl command can be used to get the information, using the following command: | ||
media-ctl -d platform:dcmipp -e "imx335 0-001a" | media-ctl -d platform:dcmipp -e "imx335 0-001a" | ||
The camera sensor only supports a discrete set of formats and resolutions that can be obtained thanks to the v4l2-ctl options | |||
{{highlight|'''--list-subdev-mbus-codes'''}}, {{highlight|'''--list-subdev-framesizes'''}}: | {{highlight|'''--list-subdev-mbus-codes'''}}, {{highlight|'''--list-subdev-framesizes'''}}: | ||
{{Board$}} v4l2-ctl -d '''/dev/v4l-subdev6''' {{highlight|'''--list-subdev-mbus-codes'''}} | {{Board$}}v4l2-ctl -d '''/dev/v4l-subdev6''' {{highlight|'''--list-subdev-mbus-codes'''}} | ||
or | or | ||
{{Board$}} v4l2-ctl -d '''/dev/v4l-subdev6''' {{highlight|'''--list-subdev-mbus-codes'''}} {{highlight|<pad number>}} | {{Board$}}v4l2-ctl -d '''/dev/v4l-subdev6''' {{highlight|'''--list-subdev-mbus-codes'''}} {{highlight|<pad number>}} | ||
ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0) | ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0) | ||
0x300f: MEDIA_BUS_FMT_{{STDarkBlue|'''SRGGB10_1X10'''}} | 0x300f: MEDIA_BUS_FMT_{{STDarkBlue|'''SRGGB10_1X10'''}} | ||
The frame size might depend on the frame format, so precise it when asking for frame size: | |||
{{Board$}} v4l2-ctl -d '''/dev/v4l-subdev6''' {{highlight|'''--list-subdev-framesizes'''}} pad=0,code=0x300f | {{Board$}}v4l2-ctl -d '''/dev/v4l-subdev6''' {{highlight|'''--list-subdev-framesizes'''}} pad=0,code=0x300f | ||
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0) | ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0) | ||
Size Range: {{STPurple|'''2592x1940'''}} - {{STPurple|'''2592x1940'''}} | Size Range: {{STPurple|'''2592x1940'''}} - {{STPurple|'''2592x1940'''}} | ||
As often for RAW Bayer sensor, direct framerate control is not available, but blanking control | As often for RAW Bayer sensor, direct framerate control is not available, but blanking control can be used as a workaround. | ||
Now that camera sensor capabilities are known, the wanted configuration can be set following instructions described in chapter | Now that the camera sensor capabilities are known, the wanted configuration can be set following instructions described in chapter [[#Camera subsystem setup | camera subsystem setup]]: | ||
{{Board$}} | {{Board$}} | ||
media-ctl -d platform:dcmipp --set-v4l2 "'{{STDarkGreen|'''imx335 0-001a'''}}':0[fmt:{{STDarkBlue|'''SRGGB10_1X10'''}}/{{STPurple|'''2592x1940'''}}]" | media-ctl -d platform:dcmipp --set-v4l2 "'{{STDarkGreen|'''imx335 0-001a'''}}':0[fmt:{{STDarkBlue|'''SRGGB10_1X10'''}}/{{STPurple|'''2592x1940'''}}]" | ||
=== Downscale and crop === | === Downscale and crop === | ||
Three features of DCMIPP can help to reduce resolution: decimation, downsize and crop. | Three features of DCMIPP can help to reduce resolution: decimation, downsize, and crop. | ||
The main pipe can crop and downscale. To do so postproc subdev must be configured accordingly. | |||
The wanted resolution must be set on postproc subdev using compose property of sink pad. | The wanted resolution must be set on postproc subdev using the compose property of the sink pad. | ||
Here is an example to get {{highlight|'''640x480'''}} frames from {{STPurple|'''2592x1940'''}} input frames coming from camera sensor: | Here is an example to get {{highlight|'''640x480'''}} frames from {{STPurple|'''2592x1940'''}} input frames coming from a camera sensor: | ||
{{Board$}} | {{Board$}} | ||
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/{{STPurple|'''2592x1940'''}}]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/{{highlight|'''640x480'''}}]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/{{highlight|'''640x480'''}}]" | |||
export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture") | |||
{{Board$}} gst-launch-1.0 v4l2src device= | {{Board$}}gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=RGB16, width=640, height=480, framerate=30/1 ! queue ! autovideosink | ||
{{Info| | {{Info|Note that '''(top,left)''' argument of compose property is ignored by driver}} | ||
<br> | <br> | ||
The wanted crop area must be set using crop property of postproc subdev sink pad. | The wanted crop area must be set using the crop property of the postproc subdev sink pad. | ||
Here is an example to crop a {{STPurple|'''640x480'''|tint=50}} area centered inside the {{STPurple|'''2592x1940'''}} input frame coming from camera sensor: | Here is an example to crop a {{STPurple|'''640x480'''|tint=50}} area centered inside the {{STPurple|'''2592x1940'''}} input frame coming from camera sensor: | ||
{{Board$}} | {{Board$}} | ||
| Line 447: | Line 445: | ||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[crop:(976,300)/{{STPurple|'''640x480'''|tint=50}}]" | media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[crop:(976,300)/{{STPurple|'''640x480'''|tint=50}}]" | ||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/{{STPurple|'''640x480'''|tint=50}}]" | media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/{{STPurple|'''640x480'''|tint=50}}]" | ||
export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture") | |||
{{Board$}} gst-launch-1.0 v4l2src device= | {{Board$}}gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! autovideosink | ||
Downscale and crop can be combined together. | Downscale and crop can be combined together. | ||
| Line 461: | Line 460: | ||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/{{highlight|'''640x480'''}}]" | media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/{{highlight|'''640x480'''}}]" | ||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/{{highlight|'''640x480'''}}]" | media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/{{highlight|'''640x480'''}}]" | ||
export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture") | |||
{{Board$}} gst-launch-1.0 v4l2src device= | {{Board$}}gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! autovideosink | ||
Refer to the selection chapter of the sub device interface documentation<ref name=API_MC_SEL>{{DocSource | domain=Linux kernel | path=userspace-api/media/v4l/dev-subdev.html#selections-cropping-scaling-and-composition | text=Linux Media Infrastructure userspace API » Part I - Video for Linux API » 4. Interfaces » 4.13. Sub-device Interface » 4.13.3.2. Selections: cropping, scaling and composition}}</ref> to get more details on how to control scaling and crop. | Refer to the selection chapter of the sub device interface documentation<ref name=API_MC_SEL>{{DocSource | domain=Linux kernel | path=userspace-api/media/v4l/dev-subdev.html#selections-cropping-scaling-and-composition | text=Linux Media Infrastructure userspace API » Part I - Video for Linux API » 4. Interfaces » 4.13. Sub-device Interface » 4.13.3.2. Selections: cropping, scaling and composition}}</ref> to get more details on how to control scaling and crop. | ||
=== Color conversion === | === Color conversion === | ||
The main pipe and auxiliary pipe are able to do color conversion. | |||
The main pipe is the most featured pipe with conversions to RGB and YUV, either single-plane (for example YUV422 coplanar), dual-plane (for example YUV420 semiplanar NV12) or triple-plane (for example YUV420 planar I420). To do so the main postproc subdev must be configured accordingly. The wanted target format must be set on the main postproc subdev using the fmt property of the source pad. Here is an example to get {{highlight|'''YUV420 semiplanar'''}} in output of the main pipe: | |||
{{Board$}} | {{Board$}} | ||
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/{{STPurple|'''2592x1940'''}}]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:{{highlight|'''YUYV8_1_5X8'''}}/640x480]" | |||
export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture") | |||
{{Board$}} gst-launch-1.0 v4l2src device= | {{Board$}}gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format={{highlight|'''NV12'''}}, width=640, height=480, framerate=30/1 ! queue ! autovideosink | ||
Here are all the | Here are all the mbus codes supported at the main postproc subdevices level: | ||
{{Board$}} v4l2-ctl -d /dev/v4l-subdev3 --list-subdev-mbus-codes 1 | {{Board$}}v4l2-ctl -d /dev/v4l-subdev3 --list-subdev-mbus-codes 1 | ||
ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=1) | ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=1) | ||
0x100a: MEDIA_BUS_FMT_RGB888_1X24 | 0x100a: MEDIA_BUS_FMT_RGB888_1X24 | ||
| Line 498: | Line 499: | ||
0x2003: MEDIA_BUS_FMT_VYUY8_1_5X8 | 0x2003: MEDIA_BUS_FMT_VYUY8_1_5X8 | ||
And their V4L2 devices counterpart on main pipe capture node: | And their V4L2 devices counterpart on the main pipe capture node: | ||
{{Board$}} v4l2-ctl -d | {{Board$}}v4l2-ctl -d $main_capture_dev --list-formats-ext | ||
ioctl: VIDIOC_ENUM_FMT | ioctl: VIDIOC_ENUM_FMT | ||
Type: Video Capture | Type: Video Capture | ||
| Line 536: | Line 537: | ||
Size: Continuous 16x16 - 4096x2160 | Size: Continuous 16x16 - 4096x2160 | ||
Care must be taken on adequation between mbus-code, V4L2 pixel format and GStreamer pixel format | Care must be taken on adequation between mbus-code, V4L2 pixel format, and GStreamer pixel format. Find below a list of the most common pixel format: | ||
{|style="text-align: center; " | {|style="text-align: center; " | ||
|- | |- | ||
| Line 550: | Line 551: | ||
| YUYV8_2X8 || YUYV || YUY2 | | YUYV8_2X8 || YUYV || YUY2 | ||
|- | |- | ||
! scope="row" | YUV420 | ! scope="row" | YUV420 semiplanar 12 bits two planes | ||
| YUYV8_1_5X8 || NV12 || NV12 | | YUYV8_1_5X8 || NV12 || NV12 | ||
|- | |- | ||
| Line 568: | Line 569: | ||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" | media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" | ||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]" | media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]" | ||
export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture") | |||
{{Board$}} v4l2-ctl -d | {{Board$}}v4l2-ctl -d $main_capture_dev --set-fmt-video=width=640,height=480,pixelformat=RGBP --stream-mmap --stream-count=1 --stream-to=grab-640x480-rgb565.raw | ||
To display it, this raw frame must be converted first to JPEG: | |||
{{Board$}} gst-launch-1.0 filesrc location= grab-640x480-rgb565.raw blocksize=614400 ! "video/x-raw, format=(string)RGB16, width=(int)640, height=(int)480, framerate=(fraction)30/1" ! videoconvert ! jpegenc ! filesink location=grab-640x480-rgb565.jpeg | {{Board$}}gst-launch-1.0 filesrc location= grab-640x480-rgb565.raw blocksize=614400 ! "video/x-raw, format=(string)RGB16, width=(int)640, height=(int)480, framerate=(fraction)30/1" ! videoconvert ! jpegenc ! filesink location=grab-640x480-rgb565.jpeg | ||
Then [[Wayland Weston overview#Display_some_images|weston-image]] utility can be used to display this JPEG file: | Then the [[Wayland Weston overview#Display_some_images|weston-image]] utility can be used to display this JPEG file: | ||
{{Board$}} weston-image grab-640x480-rgb565.jpeg | {{Board$}}weston-image grab-640x480-rgb565.jpeg | ||
=== Fullscreen preview === | === Fullscreen preview === | ||
Camera preview demo application | Camera preview demo application, started by clicking on the camera icon menu, is a good example of script-based command lines that combines camera subsystem configuration using media-ctl, preview using GStreamer, and automatic image correction using isp utility in the background. You can manually start this demo script using the command line: | ||
{{Board$}} /usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh | {{Board$}}/usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh | ||
With GPU | With GPU | ||
/home/root | /home/root | ||
| Line 600: | Line 602: | ||
SCREEN_HEIGHT=600 | SCREEN_HEIGHT=600 | ||
GStreamer graph: | GStreamer graph: | ||
{{STPurple| v4l2src device<nowiki>=</nowiki>/dev/ | {{STPurple| v4l2src device<nowiki>=</nowiki>/dev/video2 ! video/x-raw, format<nowiki>=</nowiki>RGB16, width<nowiki>=</nowiki>640, height<nowiki>=</nowiki>480 ! queue ! gtkwaylandsink name<nowiki>=</nowiki>gtkwsink }} | ||
media-ctl camera subsystem configuration and GStreamer preview commands are displayed to ease further debugging and prototyping using copy/paste: | The media-ctl camera subsystem configuration and GStreamer preview commands are displayed to ease further debugging and prototyping using copy/paste: | ||
{{Board$}} | {{Board$}} | ||
{{STDarkGreen| media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" | {{STDarkGreen| media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" | ||
| Line 611: | Line 613: | ||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]" }} | media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]" }} | ||
{{Board$}} gst-launch-1.0 {{STPurple| v4l2src device<nowiki>=</nowiki>/dev/video2 ! video/x-raw, format<nowiki>=</nowiki>RGB16, width<nowiki>=</nowiki>640, height<nowiki>=</nowiki>480 ! queue ! gtkwaylandsink name<nowiki>=</nowiki>gtkwsink }} | {{Board$}}gst-launch-1.0 {{STPurple| v4l2src device<nowiki>=</nowiki>/dev/video2 ! video/x-raw, format<nowiki>=</nowiki>RGB16, width<nowiki>=</nowiki>640, height<nowiki>=</nowiki>480 ! queue ! gtkwaylandsink name<nowiki>=</nowiki>gtkwsink }} | ||
=== Take a picture === | === Take a picture === | ||
The IMX335 camera sensor | The IMX335 camera sensor does not support JPEG frame format natively, but you can use the [[V4L2 video codec overview|hardware JPEG video encoder]] to encode a raw camera sensor frame to JPEG. | ||
Capture a | Capture a 5 MP YUV422 raw frame: | ||
{{Board$}} | {{Board$}} | ||
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" | media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" | ||
| Line 624: | Line 626: | ||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/2592x1940]" | media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/2592x1940]" | ||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/2592x1940]" | media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/2592x1940]" | ||
export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture") | |||
{{Board$}} v4l2-ctl -d | {{Board$}}v4l2-ctl -d $main_capture_dev --set-fmt-video=width=2592,height=1940,pixelformat=YUYV --stream-mmap --stream-count=1 --stream-skip=3 --stream-to=grab-2592x1940-yuyv.raw | ||
Convert to JPEG using [[V4L2 video codec overview|hardware JPEG video encoder]]: | Convert to JPEG using the [[V4L2 video codec overview|hardware JPEG video encoder]]: | ||
{{Board$}} gst-launch-1.0 filesrc location= grab-2592x1940-yuyv.raw blocksize=10077696 ! "video/x-raw, format=YUY2, width=2592, height=1940, framerate=30/1, colorimetry=2:0:0:0" ! v4l2jpegenc ! filesink location=pic-5Mp.jpeg | {{Board$}}gst-launch-1.0 filesrc location= grab-2592x1940-yuyv.raw blocksize=10077696 ! "video/x-raw, format=YUY2, width=2592, height=1940, framerate=30/1, colorimetry=2:0:0:0" ! v4l2jpegenc ! filesink location=pic-5Mp.jpeg | ||
Then display it: | Then display it: | ||
{{Board$}} weston-image pic-5Mp.jpeg | {{Board$}}weston-image pic-5Mp.jpeg | ||
You can check the picture resolution using [[Gst-typefind|gst-typefind]]: | You can check the picture resolution using [[Gst-typefind|gst-typefind]]: | ||
{{Board$}} gst-typefind-1.0 pic-5Mp.jpeg | {{Board$}}gst-typefind-1.0 pic-5Mp.jpeg | ||
pic-5Mp.jpeg - image/jpeg, width=(int)2592, height=(int)1944, sof-marker=(int)0 | pic-5Mp.jpeg - image/jpeg, width=(int)2592, height=(int)1944, sof-marker=(int)0 | ||
=== Pictures streaming over network === | === Pictures streaming over network === | ||
Refer to [[How to stream RAW camera over network]] article to get some examples on how to stream camera content over network. | Refer to the [[How to stream RAW camera over network]] article to get some examples on how to stream camera content over network. | ||
=== Preview in YUV422 === | === Preview in YUV422 === | ||
| Line 650: | Line 653: | ||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" | media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" | ||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/640x480]" | media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/640x480]" | ||
export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture") | |||
{{Board$}} gst-launch-1.0 v4l2src device= | {{Board$}}gst-launch-1.0 v4l2src device=$main_capture_dev ! "video/x-raw, format=YUY2, width=640,height=480, framerate=(fraction)30/1" ! queue ! autovideosink | ||
=== Grab a raw-bayer frame === | === Grab a raw-bayer frame === | ||
To dump the raw-bayer frame output by camera sensor, the DCMIPP dump pipe can be used: | |||
{{Board$}} | {{Board$}} | ||
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':1[fmt:SRGGB10_1X10/2592x1940 field:none]" | |||
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postproc':1[fmt:SRGGB10_1X10/2592x1940]" | |||
export dump_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_dump_capture") | |||
{{Board$}}v4l2-ctl -d $dump_capture_dev --set-fmt-video=width=2592,height=1940,pixelformat=RG10 --stream-mmap --stream-count=1 --stream-to=grab-2592x1940-rggb10.raw | |||
{{Board$}} | |||
== How to trace and debug == | == How to trace and debug == | ||
=== How to monitor === | === How to monitor === | ||
==== Check of | ==== Check of device tree configuration ==== | ||
Here are some commands to verify that DCMIPP is enabled, check which camera sensor is used and log other details about | Here are some commands to verify that DCMIPP is enabled, check which camera sensor is used and log other details about device tree configuration: | ||
{{Board$}} | {{Board$}} | ||
echo "#!/bin/bash" > dtdumpentry.sh;echo "hexdump -e '\"=\"' -e '20/1 \"%c\"\"\t\"' -e '20/1 \"%02x\"\"\n\"' \$1" >> dtdumpentry.sh;chmod +x dtdumpentry.sh | echo "#!/bin/bash" > dtdumpentry.sh;echo "hexdump -e '\"=\"' -e '20/1 \"%c\"\"\t\"' -e '20/1 \"%02x\"\"\n\"' \$1" >> dtdumpentry.sh;chmod +x dtdumpentry.sh | ||
| Line 730: | Line 729: | ||
==== Camera subsystem configuration debugging ==== | ==== Camera subsystem configuration debugging ==== | ||
Refer to section [[#Get the topology of camera subsystem]] for basics on how to get and set camera subsystem configuration. Find below some debugging tips to help configuration handling. | Refer to the section [[#Get the topology of camera subsystem]] for basics on how to get and set the camera subsystem configuration. Find below some debugging tips to help configuration handling. | ||
===== | ===== Subdevice configuration syntax error ===== | ||
[[media-ctl]] utility returns an error when a configuration syntax mistake is made. Example here with erroneous entity name: | [[media-ctl]] utility returns an error when a configuration syntax mistake is made. Example here with an erroneous entity name: | ||
{{Board$}} media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]" | {{Board$}}media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]" | ||
Unable to setup formats: Invalid argument (22) | Unable to setup formats: Invalid argument (22) | ||
To have details on the exact error, add the {{HighlightParam|'''-v'''}} option: | |||
{{Board$}} media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]" {{HighlightParam|'''-v'''}} | {{Board$}}media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]" {{HighlightParam|'''-v'''}} | ||
{{highlight|'''no such entity "dcmipp_dump_postpr"'''}} | {{highlight|'''no such entity "dcmipp_dump_postpr"'''}} | ||
'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480] | 'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480] | ||
| Line 743: | Line 742: | ||
Another example with pad number mistake (only pad number 0 or 1 are allowed): | Another example with pad number mistake (only pad number 0 or 1 are allowed): | ||
{{Board$}} media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480]" {{HighlightParam|'''-v'''}} | {{Board$}}media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480]" {{HighlightParam|'''-v'''}} | ||
{{highlight|'''No pad '4' on entity "dcmipp_dump_postproc". Maximum pad number is 1'''}} | {{highlight|'''No pad '4' on entity "dcmipp_dump_postproc". Maximum pad number is 1'''}} | ||
'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480] | 'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480] | ||
| Line 749: | Line 748: | ||
Example with format mistake (BA81 is a raw-bayer V4L2 pixel format while subdevice expects V4L2 mbus code such as SBGGR8_1X8): | Example with format mistake (BA81 is a raw-bayer V4L2 pixel format while subdevice expects V4L2 mbus code such as SBGGR8_1X8): | ||
{{Board$}} media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postproc':0[fmt:BA81/640x480]" {{HighlightParam|'''-v'''}} | {{Board$}}media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postproc':0[fmt:BA81/640x480]" {{HighlightParam|'''-v'''}} | ||
{{highlight|'''Invalid pixel code 'BA81''''}} | {{highlight|'''Invalid pixel code 'BA81''''}} | ||
'dcmipp_dump_postproc':0[fmt:BA81/640x480] | 'dcmipp_dump_postproc':0[fmt:BA81/640x480] | ||
{{highlight|''' ^'''}} | {{highlight|''' ^'''}} | ||
===== | ===== Subdevice negotiation ===== | ||
Requested configuration may not be fully accepted by the subdevice, in this case, the subdevice configures itself with the part of configuration it can handle. [[media-ctl]] {{HighlightParam|'''-v'''}} option when setting configuration shows the effective subdevice configuration. | Requested configuration may not be fully accepted by the subdevice, in this case, the subdevice configures itself with the part of the configuration it can handle. The [[media-ctl]] {{HighlightParam|'''-v'''}} option when setting configuration shows the effective subdevice configuration. | ||
Here is an example with 5 fps framerate not accepted by the subdevice and 30 fps selected instead: | Here is an example with a 5 fps framerate not accepted by the subdevice and 30 fps selected instead: | ||
{{Board$}} media-ctl -d platform:dcmipp --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@{{HighlightParam|'''1/5'''}} field:none]" -v | {{Board$}}media-ctl -d platform:dcmipp --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@{{HighlightParam|'''1/5'''}} field:none]" -v | ||
Setting up format RGB565_2X8_BE 640x480 on pad gc2145 1-003c/0 | Setting up format RGB565_2X8_BE 640x480 on pad gc2145 1-003c/0 | ||
Format set: RGB565_2X8_BE 640x480 | Format set: RGB565_2X8_BE 640x480 | ||
Setting up frame interval {{HighlightParam|'''1/5'''}} on pad gc2145 1-003c/0 | Setting up frame interval {{HighlightParam|'''1/5'''}} on pad gc2145 1-003c/0 | ||
{{highlight|Frame interval set: '''1/30'''|tint=50}} | {{highlight|Frame interval set: '''1/30'''|tint=50}} | ||
{{Warning|No error or warning is returned by [[media-ctl]], so take | {{Warning|No error or warning is returned by [[media-ctl]], so take care to read back configuration with {{HighlightParam|'''--set-v4l2 with -v'''}} or {{HighlightParam|'''--get-v4l2'''}} to check if it is fully accepted. Several negotiation rounds may be needed to find a fully compatible configuration.}} | ||
<br> | <br> | ||
Example with resolution negotiation ({{STPurple|'''4000x4000'''}} reduced to {{STPurple|'''2592x1944'''|tint=50}}) | Example with resolution negotiation ({{STPurple|'''4000x4000'''}} reduced to {{STPurple|'''2592x1944'''|tint=50}}): | ||
{{Board$}} media-ctl -d platform:dcmipp --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/{{STPurple|'''4000x4000'''}}@1/15 field:none]" -v | {{Board$}}media-ctl -d platform:dcmipp --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/{{STPurple|'''4000x4000'''}}@1/15 field:none]" -v | ||
Setting up format RGB565_2X8_BE {{STPurple|'''4000x4000'''}} on pad gc2145 1-003c/0 | Setting up format RGB565_2X8_BE {{STPurple|'''4000x4000'''}} on pad gc2145 1-003c/0 | ||
Format set: RGB565_2X8_BE {{STPurple|'''1600x1200'''|tint=50}} | Format set: RGB565_2X8_BE {{STPurple|'''1600x1200'''|tint=50}} | ||
<br> | <br> | ||
Example with format negotiation ({{STDarkBlue|'''AYUV8_1X32'''}} not supported, defaults to {{STDarkBlue|'''RGB565_2X8_LE'''|tint=50}}): | Example with format negotiation ({{STDarkBlue|'''AYUV8_1X32'''}} not supported, defaults to {{STDarkBlue|'''RGB565_2X8_LE'''|tint=50}}): | ||
{{Board$}} media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postproc':0[fmt:{{STDarkBlue|'''AYUV8_1X32'''}}/640x480]" -v | {{Board$}}media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postproc':0[fmt:{{STDarkBlue|'''AYUV8_1X32'''}}/640x480]" -v | ||
Setting up format {{STDarkBlue|'''AYUV8_1X32'''}} 640x480 on pad dcmipp_dump_postproc/0 | Setting up format {{STDarkBlue|'''AYUV8_1X32'''}} 640x480 on pad dcmipp_dump_postproc/0 | ||
Format set: {{STDarkBlue|'''RGB565_2X8_LE'''|tint=50}} | Format set: {{STDarkBlue|'''RGB565_2X8_LE'''|tint=50}} | ||
| Line 777: | Line 776: | ||
=== How to trace === | === How to trace === | ||
==== V4L2 userland API tracing ==== | ==== V4L2 userland API tracing ==== | ||
Tracing of V4L2 userland API<ref name=API/> can be enabled using | Tracing of V4L2 userland API<ref name=API/> can be enabled using commands (video device number must be adjusted): | ||
(video device number must be adjusted) | |||
{{Board$}} | {{Board$}} | ||
echo 0x3 > /sys/devices/platform/soc@0/42080000.rifsc/48030000.dcmipp/video4linux/video2/dev_debug | echo 0x3 > /sys/devices/platform/soc@0/42080000.rifsc/48030000.dcmipp/video4linux/video2/dev_debug | ||
| Line 784: | Line 783: | ||
echo 0x3 > /sys/devices/platform/soc@0/42080000.rifsc/48030000.dcmipp/video4linux/video4/dev_debug | echo 0x3 > /sys/devices/platform/soc@0/42080000.rifsc/48030000.dcmipp/video4linux/video4/dev_debug | ||
echo 0x3 > /sys/devices/platform/soc@0/42080000.rifsc/48030000.dcmipp/video4linux/video5/dev_debug | echo 0x3 > /sys/devices/platform/soc@0/42080000.rifsc/48030000.dcmipp/video4linux/video5/dev_debug | ||
Traces are output in kernel log buffer: | Traces are output in the kernel log buffer: | ||
{{Board$}} dmesg | {{Board$}}dmesg | ||
<pre> | <pre> | ||
[ 6792.553216] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0 | [ 6792.553216] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0 | ||
| Line 1,028: | Line 1,027: | ||
echo 0x3 > /sys/module/videobuf2_v4l2/parameters/debug | echo 0x3 > /sys/module/videobuf2_v4l2/parameters/debug | ||
Traces are output in kernel log buffer: | Traces are output in the kernel log buffer: | ||
{{Board$}} dmesg | {{Board$}}dmesg | ||
[ 1218.626235] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 0, plane 0 offset 0x00000000 | [ 1218.626235] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 0, plane 0 offset 0x00000000 | ||
[ 1218.642445] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 1, plane 0 offset 0x004d4000 | [ 1218.642445] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 1, plane 0 offset 0x004d4000 | ||
| Line 1,068: | Line 1,067: | ||
==== DCMIPP V4L2 kernel driver tracing ==== | ==== DCMIPP V4L2 kernel driver tracing ==== | ||
DCMIPP dynamic debug traces<ref>[[How to use the kernel dynamic debug]]</ref> can be enabled using | DCMIPP dynamic debug traces<ref>[[How to use the kernel dynamic debug]]</ref> can be enabled using commands: | ||
{{Board$}} | {{Board$}} | ||
echo "module stm32_csi2host +p" > /sys/kernel/debug/dynamic_debug/control | echo "module stm32_csi2host +p" > /sys/kernel/debug/dynamic_debug/control | ||
echo "module stm32_dcmipp +p" > /sys/kernel/debug/dynamic_debug/control | echo "module stm32_dcmipp +p" > /sys/kernel/debug/dynamic_debug/control | ||
Here is an example | Here is an example of a preview: | ||
{{Board$}} dmesg | {{Board$}}dmesg | ||
<pre> | <pre> | ||
root@stm32mp2:~# v4l2-ctl -d /dev/video2 --set-fmt-video=width=640,height=480,pi | root@stm32mp2:~# v4l2-ctl -d /dev/video2 --set-fmt-video=width=640,height=480,pi | ||
| Line 1,229: | Line 1,228: | ||
=== How to debug === | === How to debug === | ||
==== Errors ==== | ==== Errors ==== | ||
Errors are unconditionally traced in kernel log: | Errors are unconditionally traced in the kernel log: | ||
{{Board$}} dmesg | {{Board$}}dmesg | ||
[ 1947.932915] dcmipp-bytecap dcmipp-bytecap.3.auto: Stop streaming, errors=1 (overrun=0, limit=0), vsync=2, frame=2, buffers=2, it=4 | [ 1947.932915] dcmipp-bytecap dcmipp-bytecap.3.auto: Stop streaming, errors=1 (overrun=0, limit=0), vsync=2, frame=2, buffers=2, it=4 | ||
==== Memory tracking ==== | ==== Memory tracking ==== | ||
Frames require large chunks of contiguous memory. They are allocated by V4L2 framework through DMA | Frames require large chunks of contiguous memory. They are allocated by V4L2 framework through the DMA back-end. | ||
Those allocations can be traced using: | Those allocations can be traced using: | ||
{{Board$}} | {{Board$}} | ||
| Line 1,248: | Line 1,247: | ||
[11311.764146] vb2_dc_mmap: mapped dma addr 0xf1c00000 at 0xb307c000, size 614400 | [11311.764146] vb2_dc_mmap: mapped dma addr 0xf1c00000 at 0xb307c000, size 614400 | ||
</pre> | </pre> | ||
Four frames of VGA YUV422 frames: 640x480x2=614400 bytes | |||
== Source code location == | == Source code location == | ||
| Line 1,267: | Line 1,266: | ||
<noinclude> | <noinclude> | ||
{{ReviewsComments|-- [[User:Philippe Cornu|Philippe Cornu]] ([[User talk:Philippe Cornu|talk]]) 15:54, 11 June 2024 (CEST)<br />Please check command line examples when STM32MP25 dk board will be available.}} | {{ReviewsComments|-- [[User:Philippe Cornu|Philippe Cornu]] ([[User talk:Philippe Cornu|talk]]) 15:54, 11 June 2024 (CEST)<br />Please check command line examples when STM32MP25 dk board will be available.}} | ||
{{ArticleBasedOnModel | Framework overview article model}} | {{ArticleBasedOnModel | Framework overview article model}} | ||
{{PublicationRequestId | | {{PublicationRequestId | 32060 | 2024-08-22 | }} | ||
[[Category:V4L2|V4L2 camera overview STM32MP25]] | [[Category:V4L2|V4L2 camera overview STM32MP25]] | ||
</noinclude> | </noinclude> | ||
Latest revision as of 12:59, 14 October 2024
This article gives information about the Linux® V4L2 camera framework.
1. Framework purpose
The V4L2 Linux® kernel framework[1] allows the control of both an external camera sensor, and the camera interface. This allows the capture of raw frames in various pixel formats or encoded stream data such as JPEG.
This could be typically used, with the help of other Linux multimedia frameworks and applications, to take a snapshot, to make a preview, to make a video recording or even remotely stream images from the camera sensor.
2. System overview
2.1. Component description
- media-ctl (user space)
A V4L2 utility relying on media controller Linux kernel interface[2] aiming to configure and link each sub devices composing the camera subsystem to configure it.
- Application (user space)
Any application relying on V4L2 Linux kernel interface[3] or libv4l abstraction layer. GStreamer framework provides such application.
- V4L2 utilities (user space)
A set of tools to test, configure, and use the whole camera subsystem, including the external camera sensor and the camera interface. v4l2-ctl is one of the most useful utilities.
- V4L2 libraries (libv4l) (user space)
A set of libraries on top of the V4L2 Linux kernel interface[3] which abstract the kernel interface to simplify, keep compatibility, or add some hooks between V4L-based applications and the V4L2 kernel interface.
- V4L2 core (kernel space)
This layer represents the standard Linux kernel V4L2 framework.
- stm32-dcmipp (kernel space)
This V4L2 DCMIPP Linux device driver handles the DCMIPP hardware block.
- Camera sensor X driver (kernel space)
This V4L2 Linux device driver handles the camera sensor X external peripheral, some GPIOs lines, and potentially power supplies to power-up/down the camera sensor. The communication with the camera sensor is done through the I2C bus.
- DCMIPP (Hardware)
- Camera sensor X (Hardware)
The camera sensor external peripheral.
2.2. APIs description
The V4L2 userland API is documented in the Linux media subsystem documentation[3].
The V4L2 kernel framework internal API is documented in the V4L2 kernel support section of the Linux kernel documentation[4].
The media controller API is documented in the Linux media subsystem documentation[2].
3. Configuration
3.1. Kernel configuration
The STM32 DCMIPP camera interface and the camera sensor are enabled by default in STMicroelectronics deliveries.
Nevertheless, this is not the case when using the upstream kernel version. In this case, the DCMIPP V4L2 driver can be enabled using the Linux kernel menuconfig tool:
Device Drivers --->
<M> Multimedia support --->
Media drivers --->
[*] Media platform devices --->
<M> STM32 Digital Camera Memory Interface Pixel Processor (DCMIPP) support
The external camera sensor connected to the camera interface must also be enabled. Here is an example with the IMX335 SONY camera sensor located on the MB1854 camera daughter board[5] connected to the CN4 CSI camera connector[6] of the STM32MP257F-EV1 Evaluation board 
:
Device Drivers --->
<M> Multimedia support --->
Media ancillary drivers --->
[*] Camera sensor devices --->
<M> Sony IMX335 sensor support
3.2. Device tree configuration
Refer to the DCMIPP device tree configuration article for a complete view of DCMIPP and sensor configuration thanks to Linux kernel device tree mechanism.
4. How to use the framework
The use cases described here are enabled using media-ctl, v4l2-ctl, gst-launch, or gst-play command line utilities.
4.1. List the video devices and their capabilities
List all the available video devices using --list-devices option:
v4l2-ctl --list-devices
st,stm32mp25-vdec-dec (platform: hantro-vpu):
/dev/video0
/dev/video6
/dev/media0
/dev/media2
stm32_dcmipp (platform:dcmipp):
/dev/video1
/dev/video2
/dev/video3
/dev/video4
/dev/video5
/dev/media1
If several devices are available, use the -d option after any v4l2-ctl commands to target a specific device. If the -d option is not specified, /dev/video0 is targeted by default.
To have information on a specific device, use the -D option:
v4l2-ctl -d /dev/video3 -D
root@stm32mp25:~# v4l2-ctl -d /dev/video2 -D
Driver Info:
Driver name : dcmipp
Card type : stm32_dcmipp
Bus info : platform:dcmipp
Driver version : 6.1.82
Capabilities : 0xa4200001
Video Capture
I/O MC
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x24200001
Video Capture
I/O MC
Streaming
Extended Pix Format
Media Driver Info:
Driver name : dcmipp
Model : DCMIPP MDEV
Serial :
Bus info : platform:dcmipp
Media version : 6.1.82
Hardware revision: 0x00000030 (48)
Driver version : 6.1.82
Interface Info:
ID : 0x03000017
Type : V4L Video
Entity Info:
ID : 0x00000015 (21)
Name : dcmipp_main_capture
Function : V4L2 I/O
Pad 0x01000016 : 0: Sink
Link 0x02000032: from remote pad 0x1000014 of entity 'dcmipp_main_postproc' (Video Pixel Formatter): Data, Enabled, Immutable
4.2. Get the topology of the camera subsystem
Print the topology of the camera subsystem using the -p media-ctl option:
media-ctl -d platform:dcmipp -p
Media controller API version 6.1.82
Media device information
------------------------
driver dcmipp
model DCMIPP MDEV
serial
bus info platform:dcmipp
hw revision 0x30
driver version 6.1.82
Device topology
- entity 1: dcmipp_input (4 pads, 4 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SRGGB10_1X10/2592x1940 field:none colorspace:rec709]
<- "stm32_csi2host.48020000.csi2hos":1 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:SRGGB10_1X10/2592x1940 field:none colorspace:rec709]
-> "dcmipp_dump_postproc":0 [ENABLED,IMMUTABLE]
pad2: Source
[fmt:SRGGB10_1X10/2592x1940 field:none colorspace:rec709]
-> "dcmipp_main_isp":0 [ENABLED,IMMUTABLE]
pad3: Source
[fmt:SRGGB10_1X10/2592x1940 field:none colorspace:rec709]
-> "dcmipp_aux_postproc":0 []
- entity 6: dcmipp_dump_postproc (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:RGB565_2X8_LE/640x480@1/30 field:none colorspace:rec709
compose.bounds:(0,0)/640x480
compose:(0,0)/640x480]
<- "dcmipp_input":1 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:RGB565_2X8_LE/640x480@1/30 field:none colorspace:rec709
crop.bounds:(0,0)/640x480
crop:(0,0)/640x480]
-> "dcmipp_dump_capture":0 [ENABLED,IMMUTABLE]
- entity 9: dcmipp_dump_capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video1
pad0: Sink
<- "dcmipp_dump_postproc":1 [ENABLED,IMMUTABLE]
- entity 13: dcmipp_main_isp (4 pads, 5 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Sink
[fmt:SRGGB10_1X10/2592x1940 field:none colorspace:rec709
crop.bounds:(0,0)/2592x1940
crop:(0,0)/2592x1940
compose:(0,0)/2592x1940]
<- "dcmipp_input":2 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:RGB888_1X24/2592x1940 field:none colorspace:rec709]
-> "dcmipp_main_postproc":0 [ENABLED,IMMUTABLE]
-> "dcmipp_aux_postproc":0 []
pad2: Sink
[fmt:RGB888_1X24/2592x1940 field:none colorspace:rec709]
<- "dcmipp_main_isp_params_output":0 [ENABLED,IMMUTABLE]
pad3: Source
[fmt:RGB888_1X24/2592x1940 field:none colorspace:rec709]
-> "dcmipp_main_isp_stat_capture":0 [ENABLED,IMMUTABLE]
- entity 18: dcmipp_main_postproc (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev3
pad0: Sink
[fmt:RGB888_1X24/2592x1940@1/30 field:none colorspace:rec709
crop.bounds:(0,0)/2592x1940
crop:(0,0)/2592x1940
compose:(0,0)/640x480]
<- "dcmipp_main_isp":1 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:RGB565_2X8_LE/640x480@1/30 field:none colorspace:rec709]
-> "dcmipp_main_capture":0 [ENABLED,IMMUTABLE]
- entity 21: dcmipp_main_capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video2
pad0: Sink
<- "dcmipp_main_postproc":1 [ENABLED,IMMUTABLE]
- entity 25: dcmipp_aux_postproc (2 pads, 3 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev4
pad0: Sink
[fmt:RGB888_1X24/640x480@1/30 field:none colorspace:rec709
crop.bounds:(0,0)/640x480
crop:(0,0)/640x480
compose:(0,0)/640x480]
<- "dcmipp_input":3 []
<- "dcmipp_main_isp":1 []
pad1: Source
[fmt:RGB565_2X8_LE/640x480@1/30 field:none colorspace:rec709]
-> "dcmipp_aux_capture":0 [ENABLED,IMMUTABLE]
- entity 28: dcmipp_aux_capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video3
pad0: Sink
<- "dcmipp_aux_postproc":1 [ENABLED,IMMUTABLE]
- entity 32: dcmipp_main_isp_stat_capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video4
pad0: Sink
<- "dcmipp_main_isp":3 [ENABLED,IMMUTABLE]
- entity 36: dcmipp_main_isp_params_output (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video5
pad0: Source
-> "dcmipp_main_isp":2 [ENABLED,IMMUTABLE]
- entity 60: stm32_csi2host.48020000.csi2hos (9 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev5
pad0: Sink
[fmt:SRGGB10_1X10/2592x1940 field:none colorspace:raw xfer:none]
<- "imx335 0-001a":0 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:SRGGB10_1X10/2592x1940]
-> "dcmipp_input":0 [ENABLED,IMMUTABLE]
pad2: Source
[fmt:UYVY8_2X8/0x0]
pad3: Source
[fmt:UYVY8_2X8/0x0]
pad4: Source
[fmt:UYVY8_2X8/0x0]
pad5: Source
[fmt:UYVY8_2X8/0x0]
pad6: Source
[fmt:UYVY8_2X8/0x0]
pad7: Source
[fmt:UYVY8_2X8/0x0]
pad8: Source
[fmt:UYVY8_2X8/0x0]
- entity 72: imx335 0-001a (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev6
pad0: Source
[fmt:SRGGB10_1X10/2592x1940 field:none colorspace:raw xfer:none]
-> "stm32_csi2host.48020000.csi2hos":0 [ENABLED,IMMUTABLE]
This gives the current links between camera subdevices and each subdevice current pads configuration, such as, format, resolution, framerate.
This topology can also be displayed in a human readable graphical form thanks to the --print-dot option:
media-ctl -d platform:dcmipp --print-dot > graph.dot
Retrieve this .dot file into your host PC to convert it to .png:
dot -Tpng -Nfontname=Roboto -Nfontsize=10 -Efontname=Roboto -Efontsize=10 graph.dot > graph.png
This picture shows the two possible camera interfaces: MIPI CSI-2 or parallel, and the three pipes topology: dump pipe, main pipe, and auxiliary pipe:
- The dump pipe allows to directly read without any processing the camera sensor data whatever those data are: raw-bayer, RGB, YUV, JPEG, or even image statistic data. The dump pipe output camera sensor specific data can be read from the /dev/video1 capture node.
- The main pipe is the most featured DCMIPP pipe, including frame skipping, de-Bayering, image correction thanks to DCMIPP ISP, scaling, color conversion, ... The main pipe output pixel data can be read from the /dev/video2 capture node while the image statistic data can be read from the /dev/video4 capture node. Refer to How to use the DCMIPP ISP article to get more details on how to handle image statistic data.
- The auxiliary pipe can perform limited operations such as scaling and color conversion with some restrictions regarding to main pipe. It can also be dynamically linked to the output of ISP to inherit of image correction. The auxiliary pipe output pixel data can be read from the /dev/video3 capture node.
4.3. Camera subsystem setup
To be able to capture frames from a video device node with v4l2-ctl, GStreamer, or any other V4L2 application, the camera subsystem must be first configured. To do so use the media-ctl --set-v4l2 command giving the subdevice name and pad the desired format, resolution, and framerate:
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]"
The configuration must be done from source to sink. Below is a configuration allowing to capture RGB565 640x480 frames:
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]" export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture")
The camera sensor is configured in 5 MP raw-bayer 10 bits at 30 fps, which is the only configuration supported by the IMX335 for now. Then DCMIPP is configured to convert to RGB565 and downscale to 640x480. The configuration can be read back displaying the topology of the graph. See #Get the topology of camera subsystem. Then, any application that read frames from V4L2 video device node can be executed:
gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! waylandsink fullscreen=true
4.4. Image correction
To correct an image coming from a raw-bayer camera sensor using DCMIPP ISP, use the STMicroelectronics DCMIPP ISP example utility command line utility with the -w option while capture is running:
gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! waylandsink fullscreen=true & sleep 1 /usr/local/demo/bin/dcmipp-isp-ctrl -i0 -g sleep 1 kill %1
Image correction parameters are kept from a capture to another one, so you do not have to recall the dcmipp-isp-ctrl utility if ambient light conditions have not changed. More details on image correction can be found in the dedicated article How to use the DCMIPP ISP.
4.5. Controlling camera sensor
To control a camera sensor its corresponding entity device node must be found in the topology. To do so, the first entity of the graph must be found, that is, the entity having no sink pad:
- entity 68: imx335 0-001a (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev6
pad0: Source
[fmt:SRGGB10_1X10/2592x1940 field:none colorspace:raw xfer:none]
-> "stm32_csi2host.48020000.csi2hos":0 [ENABLED,IMMUTABLE]
Here it is /dev/v4l-subdev6.
Use then v4l2-ctl with the -L option to get the list of supported controls:
v4l2-ctl -d /dev/v4l-subdev6 -L
User Controls
exposure 0x00980911 (int) : min=1 max=4491 step=1 default=1608 value=1608
Image Source Controls
vertical_blanking 0x009e0901 (int) : min=2560 max=133060 step=1 default=2560 value=2560
horizontal_blanking 0x009e0902 (int) : min=0 max=1048575 step=1 default=342 value=342 flags=read-only
analogue_gain 0x009e0903 (int) : min=0 max=240 step=1 default=0 value=0
Image Processing Controls
link_frequency 0x009f0901 (intmenu): min=0 max=0 default=0 value=0 (594000000 0x2367b880) flags=read-only
0: 594000000 (0x2367b880)
pixel_rate 0x009f0902 (int64) : min=396000000 max=396000000 step=1 default=396000000 value=396000000 flags=read-only
The control value can be changed thanks to the --set-ctrl option, for example:
v4l2-ctl -d /dev/v4l-subdev6 --set-ctrl analogue_gain=10
The control value can be changed dynamically. In the following example, the analog gain is increased while the preview is running:
- Start the preview in the background.
gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! waylandsink fullscreen=true -e > /dev/null 2>&1 &
- Then increase the gain every second.
v4l2-ctl -d /dev/v4l-subdev6 --set-ctrl analogue_gain=0;sleep 1;v4l2-ctl -d /dev/v4l-subdev6 --set-ctrl analogue_gain=20;sleep 1;v4l2-ctl -d /dev/v4l-subdev6 --set-ctrl analogue_gain=40;sleep 1;v4l2-ctl -d /dev/v4l-subdev6 --set-ctrl analogue_gain=0;killall gst-launch-1.0
4.6. Set the camera sensor pixel format, resolution, and framerate
To control a camera sensor its corresponding entity device node must be found in the topology. To do so, the first entity of the graph must be found, that is, the entity having no sink pad:
- entity 68: imx335 0-001a (1 pad, 1 link) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev6 pad0: Source [fmt:SRGGB10_1X10/2592x1940 field:none colorspace:raw xfer:none] -> "stm32_csi2host.48020000.csi2hos":0 [ENABLED,IMMUTABLE]
Here it is /dev/v4l-subdev6.
Alternatively, the media-ctl command can be used to get the information, using the following command:
media-ctl -d platform:dcmipp -e "imx335 0-001a"
The camera sensor only supports a discrete set of formats and resolutions that can be obtained thanks to the v4l2-ctl options --list-subdev-mbus-codes, --list-subdev-framesizes:
v4l2-ctl -d /dev/v4l-subdev6 --list-subdev-mbus-codes
or
v4l2-ctl -d /dev/v4l-subdev6 --list-subdev-mbus-codes <pad number> ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0) 0x300f: MEDIA_BUS_FMT_SRGGB10_1X10
The frame size might depend on the frame format, so precise it when asking for frame size:
v4l2-ctl -d /dev/v4l-subdev6 --list-subdev-framesizes pad=0,code=0x300f ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0) Size Range: 2592x1940 - 2592x1940
As often for RAW Bayer sensor, direct framerate control is not available, but blanking control can be used as a workaround.
Now that the camera sensor capabilities are known, the wanted configuration can be set following instructions described in chapter camera subsystem setup:
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]"
4.7. Downscale and crop
Three features of DCMIPP can help to reduce resolution: decimation, downsize, and crop.
The main pipe can crop and downscale. To do so postproc subdev must be configured accordingly. The wanted resolution must be set on postproc subdev using the compose property of the sink pad. Here is an example to get 640x480 frames from 2592x1940 input frames coming from a camera sensor:
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]" export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture")
gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=RGB16, width=640, height=480, framerate=30/1 ! queue ! autovideosink
The wanted crop area must be set using the crop property of the postproc subdev sink pad.
Here is an example to crop a 640x480 area centered inside the 2592x1940 input frame coming from camera sensor:
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[crop:(976,300)/640x480]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]" export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture")
gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! autovideosink
Downscale and crop can be combined together. Here is an example to get 640x480 frames from 2592x1940 input frames cropped to 1292x970:
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[crop:(976,300)/1292x970]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]" export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture")
gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=RGB16, width=640,height=480, framerate=30/1 ! queue ! autovideosink
Refer to the selection chapter of the sub device interface documentation[7] to get more details on how to control scaling and crop.
4.8. Color conversion
The main pipe and auxiliary pipe are able to do color conversion.
The main pipe is the most featured pipe with conversions to RGB and YUV, either single-plane (for example YUV422 coplanar), dual-plane (for example YUV420 semiplanar NV12) or triple-plane (for example YUV420 planar I420). To do so the main postproc subdev must be configured accordingly. The wanted target format must be set on the main postproc subdev using the fmt property of the source pad. Here is an example to get YUV420 semiplanar in output of the main pipe:
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_1_5X8/640x480]" export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture")
gst-launch-1.0 v4l2src device=$main_capture_dev ! video/x-raw, format=NV12, width=640, height=480, framerate=30/1 ! queue ! autovideosink
Here are all the mbus codes supported at the main postproc subdevices level:
v4l2-ctl -d /dev/v4l-subdev3 --list-subdev-mbus-codes 1
ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=1)
0x100a: MEDIA_BUS_FMT_RGB888_1X24
0x1013: MEDIA_BUS_FMT_BGR888_1X24
0x1008: MEDIA_BUS_FMT_RGB565_2X8_LE
0x2008: MEDIA_BUS_FMT_YUYV8_2X8
0x2009: MEDIA_BUS_FMT_YVYU8_2X8
0x2006: MEDIA_BUS_FMT_UYVY8_2X8
0x2007: MEDIA_BUS_FMT_VYUY8_2X8
0x2001: MEDIA_BUS_FMT_Y8_1X8
0x2004: MEDIA_BUS_FMT_YUYV8_1_5X8
0x2005: MEDIA_BUS_FMT_YVYU8_1_5X8
0x2011: MEDIA_BUS_FMT_YUYV8_1X16
0x2012: MEDIA_BUS_FMT_YVYU8_1X16
0x2002: MEDIA_BUS_FMT_UYVY8_1_5X8
0x2003: MEDIA_BUS_FMT_VYUY8_1_5X8
And their V4L2 devices counterpart on the main pipe capture node:
v4l2-ctl -d $main_capture_dev --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'RGBP' (16-bit RGB 5-6-5)
Size: Continuous 16x16 - 4096x2160
[1]: 'YUYV' (YUYV 4:2:2)
Size: Continuous 16x16 - 4096x2160
[2]: 'YVYU' (YVYU 4:2:2)
Size: Continuous 16x16 - 4096x2160
[3]: 'UYVY' (UYVY 4:2:2)
Size: Continuous 16x16 - 4096x2160
[4]: 'VYUY' (VYUY 4:2:2)
Size: Continuous 16x16 - 4096x2160
[5]: 'GREY' (8-bit Greyscale)
Size: Continuous 16x16 - 4096x2160
[6]: 'RGB3' (24-bit RGB 8-8-8)
Size: Continuous 16x16 - 4096x2160
[7]: 'BGR3' (24-bit BGR 8-8-8)
Size: Continuous 16x16 - 4096x2160
[8]: 'BA24' (32-bit ARGB 8-8-8-8)
Size: Continuous 16x16 - 4096x2160
[9]: 'AYUV' (32-bit AYUV 8-8-8-8)
Size: Continuous 16x16 - 4096x2160
[10]: 'NV12' (Y/UV 4:2:0)
Size: Continuous 16x16 - 4096x2160
[11]: 'NV21' (Y/VU 4:2:0)
Size: Continuous 16x16 - 4096x2160
[12]: 'NV16' (Y/UV 4:2:2)
Size: Continuous 16x16 - 4096x2160
[13]: 'NV61' (Y/VU 4:2:2)
Size: Continuous 16x16 - 4096x2160
[14]: 'YU12' (Planar YUV 4:2:0)
Size: Continuous 16x16 - 4096x2160
[15]: 'YV12' (Planar YVU 4:2:0)
Size: Continuous 16x16 - 4096x2160
Care must be taken on adequation between mbus-code, V4L2 pixel format, and GStreamer pixel format. Find below a list of the most common pixel format:
| mbus-code (media-ctl) | V4L2 pixel format (v4l-ctl) | GStreamer (gst-launch-1.0) | |
|---|---|---|---|
| RGB565 16 bits single plane | RGB565_2X8_LE | RGBP | RGB16 |
| YUV422 16 bits single plane | YUYV8_2X8 | YUYV | YUY2 |
| YUV420 semiplanar 12 bits two planes | YUYV8_1_5X8 | NV12 | NV12 |
| YUV420 planar 12 bits three planes | UYVY8_1_5X8 | YU12 | I420 |
4.9. Grab a frame from a RAW sensor, with ISP demosaicing
Capture a VGA RGB565 frame on disk from the RAW sensor:
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]"
media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]"
export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture")
v4l2-ctl -d $main_capture_dev --set-fmt-video=width=640,height=480,pixelformat=RGBP --stream-mmap --stream-count=1 --stream-to=grab-640x480-rgb565.raw
To display it, this raw frame must be converted first to JPEG:
gst-launch-1.0 filesrc location= grab-640x480-rgb565.raw blocksize=614400 ! "video/x-raw, format=(string)RGB16, width=(int)640, height=(int)480, framerate=(fraction)30/1" ! videoconvert ! jpegenc ! filesink location=grab-640x480-rgb565.jpeg
Then the weston-image utility can be used to display this JPEG file:
weston-image grab-640x480-rgb565.jpeg
4.10. Fullscreen preview
Camera preview demo application, started by clicking on the camera icon menu, is a good example of script-based command lines that combines camera subsystem configuration using media-ctl, preview using GStreamer, and automatic image correction using isp utility in the background. You can manually start this demo script using the command line:
/usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh With GPU /home/root mediadev=/dev/media2 sensorsubdev="imx335 0-001a" sensordev= /dev/v4l-subdev6 interfacesubdev=stm32_csi2host.48020000.csi2hos sensorbuscode=SRGGB10_1X10 SENSORWIDTH=2592 SENSORHEIGHT=1940 Mediacontroller graph: media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]" . SCREEN_WIDTH= 1024 SCREEN_HEIGHT=600 GStreamer graph: v4l2src device=/dev/video2 ! video/x-raw, format=RGB16, width=640, height=480 ! queue ! gtkwaylandsink name=gtkwsink
The media-ctl camera subsystem configuration and GStreamer preview commands are displayed to ease further debugging and prototyping using copy/paste:
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:RGB565_2X8_LE/640x480]"
gst-launch-1.0 v4l2src device=/dev/video2 ! video/x-raw, format=RGB16, width=640, height=480 ! queue ! gtkwaylandsink name=gtkwsink
4.11. Take a picture
The IMX335 camera sensor does not support JPEG frame format natively, but you can use the hardware JPEG video encoder to encode a raw camera sensor frame to JPEG.
Capture a 5 MP YUV422 raw frame:
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]"
media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/2592x1940]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/2592x1940]"
export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture")
v4l2-ctl -d $main_capture_dev --set-fmt-video=width=2592,height=1940,pixelformat=YUYV --stream-mmap --stream-count=1 --stream-skip=3 --stream-to=grab-2592x1940-yuyv.raw
Convert to JPEG using the hardware JPEG video encoder:
gst-launch-1.0 filesrc location= grab-2592x1940-yuyv.raw blocksize=10077696 ! "video/x-raw, format=YUY2, width=2592, height=1940, framerate=30/1, colorimetry=2:0:0:0" ! v4l2jpegenc ! filesink location=pic-5Mp.jpeg
Then display it:
weston-image pic-5Mp.jpeg
You can check the picture resolution using gst-typefind:
gst-typefind-1.0 pic-5Mp.jpeg
pic-5Mp.jpeg - image/jpeg, width=(int)2592, height=(int)1944, sof-marker=(int)0
4.12. Pictures streaming over network
Refer to the How to stream RAW camera over network article to get some examples on how to stream camera content over network.
4.13. Preview in YUV422
Capture VGA YUV422 frames and display them using GStreamer :
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]"
media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':2[fmt:SRGGB10_1X10/2592x1940 field:none]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_isp':1[fmt:RGB888_1X24/2592x1940 field:none]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/640x480]"
export main_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_main_capture")
gst-launch-1.0 v4l2src device=$main_capture_dev ! "video/x-raw, format=YUY2, width=640,height=480, framerate=(fraction)30/1" ! queue ! autovideosink
4.14. Grab a raw-bayer frame
To dump the raw-bayer frame output by camera sensor, the DCMIPP dump pipe can be used:
media-ctl -d platform:dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1940]"
media-ctl -d platform:dcmipp --set-v4l2 "'stm32_csi2host.48020000.csi2hos':1[fmt:SRGGB10_1X10/2592x1940]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_input':1[fmt:SRGGB10_1X10/2592x1940 field:none]"
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postproc':1[fmt:SRGGB10_1X10/2592x1940]"
export dump_capture_dev=$(media-ctl -d "platform:dcmipp" -e "dcmipp_dump_capture")
v4l2-ctl -d $dump_capture_dev --set-fmt-video=width=2592,height=1940,pixelformat=RG10 --stream-mmap --stream-count=1 --stream-to=grab-2592x1940-rggb10.raw
5. How to trace and debug
5.1. How to monitor
5.1.1. Check of device tree configuration
Here are some commands to verify that DCMIPP is enabled, check which camera sensor is used and log other details about device tree configuration:
echo "#!/bin/bash" > dtdumpentry.sh;echo "hexdump -e '\"=\"' -e '20/1 \"%c\"\"\t\"' -e '20/1 \"%02x\"\"\n\"' \$1" >> dtdumpentry.sh;chmod +x dtdumpentry.sh
echo "#!/bin/bash" > dtdump.sh;echo "find \$1* -type f -print0 -exec ./dtdumpentry.sh {} \;" >> dtdump.sh;chmod +x dtdump.sh
rm devicetree.txt
echo "[devicetree]" >> devicetree.txt
echo "|-[dcmipp]" >> devicetree.txt
./dtdump.sh /proc/device-tree/soc@0/rifsc@42080000/dcmipp@48030000 | sed 's/\/proc\/device-tree\/soc@0\/rifsc@42080000\//| |-/' >> devicetree.txt
echo "|" >> devicetree.txt
echo "|-[camera:" | tr -d "\n" >> devicetree.txt
cat /proc/device-tree/soc@0/rifsc@42080000/i2c@40130000/imx335@1a/compatible >> devicetree.txt
echo "]" >> devicetree.txt
./dtdump.sh /proc/device-tree/soc@0/rifsc@42080000/i2c@40130000/imx335@1a -type f -print0 -exec ./dtdump.sh {} \; | sed 's/\/proc\/device-tree\/soc@0\/rifsc@42080000\//| |-/' >> devicetree.txt
cat devicetree.txt
[devicetree]
|-[dcmipp]
| |-dcmipp@48030000/power-domains=
0000000c
| |-dcmipp@48030000/clock-names=kclkmclk 6b636c6b006d636c6b00
| |-dcmipp@48030000/port/endpoint/remote-endpoint=N 0000004e
| |-dcmipp@48030000/port/endpoint/bus-type= 00000004
| |-dcmipp@48030000/port/endpoint/phandle=M 0000004d
| |-dcmipp@48030000/port/endpoint/name=endpoint 656e64706f696e7400
| |-dcmipp@48030000/port/name=port 706f727400
| |-dcmipp@48030000/resets=B� 00000013000042e0
| |-dcmipp@48030000/interrupts=� 00000000000000c600000004
| |-dcmipp@48030000/clocks=�" 00000013000000d00000001300000122
| |-dcmipp@48030000/feature-domains=W 0000001800000057
| |-dcmipp@48030000/compatible=st,stm32mp25-dcmipp 73742c73746d33326d7032352d64636d69707000
| |-dcmipp@48030000/status=okay 6f6b617900
| |-dcmipp@48030000/reg=H 4803000000001000
| |-dcmipp@48030000/phandle= 0000011f
| |-dcmipp@48030000/name=dcmipp 64636d69707000
|
|-[camera:sony,imx335]
| |-i2c@40130000/imx335@1a/port/endpoint/data-lanes= 0000000100000002
| |-i2c@40130000/imx335@1a/port/endpoint/clock-lanes= 00000000
| |-i2c@40130000/imx335@1a/port/endpoint/remote-endpoint=/ 0000002f
| |-i2c@40130000/imx335@1a/port/endpoint/phandle=L 0000004c
| |-i2c@40130000/imx335@1a/port/endpoint/link-frequencies=#g�� 000000002367b880
| |-i2c@40130000/imx335@1a/port/endpoint/name=endpoint 656e64706f696e7400
| |-i2c@40130000/imx335@1a/port/name=port 706f727400
| |-i2c@40130000/imx335@1a/clocks=- 0000002d
| |-i2c@40130000/imx335@1a/powerdown-gpios=. 0000002e0000000000000000
| |-i2c@40130000/imx335@1a/reset-gpios=. 0000002e0000000700000000
| |-i2c@40130000/imx335@1a/compatible=sony,imx335 736f6e792c696d7833333500
| |-i2c@40130000/imx335@1a/status=okay 6f6b617900
| |-i2c@40130000/imx335@1a/reg=� 0000001a
| |-i2c@40130000/imx335@1a/phandle=� 000000d2
| |-i2c@40130000/imx335@1a/name=imx335 696d7833333500
5.1.2. Camera subsystem configuration debugging
Refer to the section #Get the topology of camera subsystem for basics on how to get and set the camera subsystem configuration. Find below some debugging tips to help configuration handling.
5.1.2.1. Subdevice configuration syntax error
media-ctl utility returns an error when a configuration syntax mistake is made. Example here with an erroneous entity name:
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]"
Unable to setup formats: Invalid argument (22)
To have details on the exact error, add the -v option:
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480]" -v no such entity "dcmipp_dump_postpr" 'dcmipp_dump_postpr':0[fmt:RGB565_2X8_LE/640x480] ^
Another example with pad number mistake (only pad number 0 or 1 are allowed):
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480]" -v No pad '4' on entity "dcmipp_dump_postproc". Maximum pad number is 1 'dcmipp_dump_postproc':4[fmt:RGB565_2X8_LE/640x480] ^
Example with format mistake (BA81 is a raw-bayer V4L2 pixel format while subdevice expects V4L2 mbus code such as SBGGR8_1X8):
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postproc':0[fmt:BA81/640x480]" -v Invalid pixel code 'BA81' 'dcmipp_dump_postproc':0[fmt:BA81/640x480] ^
5.1.2.2. Subdevice negotiation
Requested configuration may not be fully accepted by the subdevice, in this case, the subdevice configures itself with the part of the configuration it can handle. The media-ctl -v option when setting configuration shows the effective subdevice configuration.
Here is an example with a 5 fps framerate not accepted by the subdevice and 30 fps selected instead:
media-ctl -d platform:dcmipp --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/640x480@1/5 field:none]" -v Setting up format RGB565_2X8_BE 640x480 on pad gc2145 1-003c/0 Format set: RGB565_2X8_BE 640x480 Setting up frame interval 1/5 on pad gc2145 1-003c/0 Frame interval set: 1/30
Example with resolution negotiation (4000x4000 reduced to 2592x1944):
media-ctl -d platform:dcmipp --set-v4l2 "'gc2145 1-003c':0[fmt:RGB565_2X8_BE/4000x4000@1/15 field:none]" -v Setting up format RGB565_2X8_BE 4000x4000 on pad gc2145 1-003c/0 Format set: RGB565_2X8_BE 1600x1200
Example with format negotiation (AYUV8_1X32 not supported, defaults to RGB565_2X8_LE):
media-ctl -d platform:dcmipp --set-v4l2 "'dcmipp_dump_postproc':0[fmt:AYUV8_1X32/640x480]" -v Setting up format AYUV8_1X32 640x480 on pad dcmipp_dump_postproc/0 Format set: RGB565_2X8_LE
5.2. How to trace
5.2.1. V4L2 userland API tracing
Tracing of V4L2 userland API[3] can be enabled using commands (video device number must be adjusted):
echo 0x3 > /sys/devices/platform/soc@0/42080000.rifsc/48030000.dcmipp/video4linux/video2/dev_debug
echo 0x3 > /sys/devices/platform/soc@0/42080000.rifsc/48030000.dcmipp/video4linux/video3/dev_debug
echo 0x3 > /sys/devices/platform/soc@0/42080000.rifsc/48030000.dcmipp/video4linux/video4/dev_debug
echo 0x3 > /sys/devices/platform/soc@0/42080000.rifsc/48030000.dcmipp/video4linux/video5/dev_debug
Traces are output in the kernel log buffer:
dmesg
[ 6792.553216] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6792.553291] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6792.553314] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6792.553378] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6792.553398] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6792.553423] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6792.553463] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6792.553488] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6792.628964] video5: VIDIOC_STREAMON: type=meta-cap
[ 6794.652333] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6794.652407] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6794.652430] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6794.652492] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6794.652512] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6794.652537] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6794.652578] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6794.652603] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6794.731598] video5: VIDIOC_STREAMON: type=meta-cap
[ 6796.755153] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6796.755228] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6796.755252] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6796.755314] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6796.755335] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6796.755360] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6796.755399] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6796.755424] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6796.829979] video5: VIDIOC_STREAMON: type=meta-cap
[ 6798.853142] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6798.853217] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6798.853241] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6798.853303] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6798.853324] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6798.853348] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6798.853388] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6798.853413] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6798.929892] video5: VIDIOC_STREAMON: type=meta-cap
[ 6800.953179] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6800.953256] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6800.953280] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6800.953339] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6800.953360] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6800.953385] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6800.953422] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6800.953448] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6801.029010] video5: VIDIOC_STREAMON: type=meta-cap
[ 6803.052299] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6803.052379] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6803.052403] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6803.052464] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6803.052484] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6803.052509] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6803.052550] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6803.052575] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6803.129992] video5: VIDIOC_STREAMON: type=meta-cap
[ 6805.153551] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6805.153627] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6805.153650] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6805.153713] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6805.153734] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6805.153759] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6805.153798] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6805.153823] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6805.229465] video5: VIDIOC_STREAMON: type=meta-cap
[ 6807.253213] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6807.253290] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6807.253314] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6807.253374] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6807.253395] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6807.253420] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6807.253462] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6807.253488] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6807.329788] video5: VIDIOC_STREAMON: type=meta-cap
[ 6809.353330] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6809.353418] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6809.353446] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6809.353516] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6809.353540] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6809.353571] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6809.353617] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6809.353649] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6809.429740] video5: VIDIOC_STREAMON: type=meta-cap
[ 6811.452775] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6811.452850] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6811.452875] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6811.452936] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6811.452956] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6811.452981] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6811.453018] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6811.453044] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6811.529944] video5: VIDIOC_STREAMON: type=meta-cap
[ 6813.552902] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6813.552978] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6813.553001] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6813.553063] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6813.553083] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6813.553108] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6813.553149] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6813.553174] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6813.628612] video5: VIDIOC_STREAMON: type=meta-cap
[ 6815.651969] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=65535, name=ISP Stat Location, id/val=0x9f090b/0x0
[ 6815.652045] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6815.652068] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6815.652128] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6815.652149] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=
60
[ 6815.652174] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6815.652214] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=
60
[ 6815.652239] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6815.731233] video5: VIDIOC_STREAMON: type=meta-cap
[ 6816.637563] video2: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_bytecap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85200001, device_caps=0x05200001
[ 6816.637865] video3: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_pixelcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85200001, device_caps=0x05200001
[ 6816.638158] video4: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_pixelcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85200001, device_caps=0x05200001
[ 6816.638525] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6816.920651] video3: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_pixelcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85200001, device_caps=0x05200001
[ 6816.920755] video3: VIDIOC_ENUMINPUT: error -25: index=0, name=, type=0, audioset=0x0, tuner=0, std=0x00000000, status=0x0, capabilities=0x0
[ 6816.920783] video3: VIDIOC_ENUMSTD: error -25: index=0, id=0x0, name=, fps=0/0, framelines=0
[ 6816.920803] video3: VIDIOC_QUERYCTRL: error -25: id=0x80000000, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000
[ 6816.920825] video3: VIDIOC_QUERYCTRL: error -25: id=0x980900, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000
[ 6816.920846] video3: VIDIOC_QUERYCTRL: error -25: id=0x8000000, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000
[ 6816.920873] video3: VIDIOC_G_STD: error -25: std=0x00000000
[ 6816.920891] video3: VIDIOC_G_INPUT: error -25: value=0
[ 6816.929786] video3: VIDIOC_G_INPUT: error -25: value=0
[ 6816.929827] video3: VIDIOC_G_SELECTION: error -25: type=vid-cap, target=2, flags=0x0, wxh=0x0, x,y=0,0
[ 6816.929851] video3: VIDIOC_CROPCAP: error -25: type=0, bounds wxh=0x0, x,y=0,0, defrect wxh=0x0, x,y=0,0, pixelaspect 0/0
[ 6816.933127] video3: VIDIOC_ENUM_FMT: index=0, type=vid-cap, flags=0x0, pixelformat=RGBP little-endian (0x50424752), mbus_code=0x0000, description='16-bit RGB 5-6-5'
[ 6816.933177] video3: VIDIOC_ENUM_FMT: index=1, type=vid-cap, flags=0x0, pixelformat=YUYV little-endian (0x56595559), mbus_code=0x0000, description='YUYV 4:2:2'
[ 6816.933202] video3: VIDIOC_ENUM_FMT: index=2, type=vid-cap, flags=0x0, pixelformat=YVYU little-endian (0x55595659), mbus_code=0x0000, description='YVYU 4:2:2'
[ 6816.933224] video3: VIDIOC_ENUM_FMT: index=3, type=vid-cap, flags=0x0, pixelformat=UYVY little-endian (0x59565955), mbus_code=0x0000, description='UYVY 4:2:2'
[ 6816.933245] video3: VIDIOC_ENUM_FMT: index=4, type=vid-cap, flags=0x0, pixelformat=VYUY little-endian (0x59555956), mbus_code=0x0000, description='VYUY 4:2:2'
[ 6816.933266] video3: VIDIOC_ENUM_FMT: index=5, type=vid-cap, flags=0x0, pixelformat=GREY little-endian (0x59455247), mbus_code=0x0000, description='8-bit Greyscale'
[ 6816.933288] video3: VIDIOC_ENUM_FMT: index=6, type=vid-cap, flags=0x0, pixelformat=RGB3 little-endian (0x33424752), mbus_code=0x0000, description='24-bit RGB 8-8-8'
[ 6816.933310] video3: VIDIOC_ENUM_FMT: index=7, type=vid-cap, flags=0x0, pixelformat=BA24 little-endian (0x34324142), mbus_code=0x0000, description='32-bit ARGB 8-8-8-8'
[ 6816.933332] video3: VIDIOC_ENUM_FMT: index=8, type=vid-cap, flags=0x0, pixelformat=AYUV little-endian (0x56555941), mbus_code=0x0000, description='32-bit AYUV 8-8-8-8'
[ 6816.933355] video3: VIDIOC_ENUM_FMT: index=9, type=vid-cap, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/UV 4:2:0'
[ 6816.933377] video3: VIDIOC_ENUM_FMT: index=10, type=vid-cap, flags=0x0, pixelformat=NV21 little-endian (0x3132564e), mbus_code=0x0000, description='Y/VU 4:2:0'
[ 6816.933400] video3: VIDIOC_ENUM_FMT: index=11, type=vid-cap, flags=0x0, pixelformat=NV16 little-endian (0x3631564e), mbus_code=0x0000, description='Y/UV 4:2:2'
[ 6816.933423] video3: VIDIOC_ENUM_FMT: index=12, type=vid-cap, flags=0x0, pixelformat=NV61 little-endian (0x3136564e), mbus_code=0x0000, description='Y/VU 4:2:2'
[ 6816.933446] video3: VIDIOC_ENUM_FMT: index=13, type=vid-cap, flags=0x0, pixelformat=YU12 little-endian (0x32315559), mbus_code=0x0000, description='Planar YUV 4:2:0'
[ 6816.933468] video3: VIDIOC_ENUM_FMT: index=14, type=vid-cap, flags=0x0, pixelformat=YV12 little-endian (0x32315659), mbus_code=0x0000, description='Planar YVU 4:2:0'
[ 6816.933489] video3: VIDIOC_ENUM_FMT: error -22: index=15, type=vid-cap, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description=''
[ 6816.933518] video3: VIDIOC_CROPCAP: error -25: type=vid-cap, bounds wxh=0x0, x,y=0,0, defrect wxh=0x0, x,y=0,0, pixelaspect 0/0
[ 6816.933559] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=YUYV little-endian (0x56595559), type=2
[ 6816.933581] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=YUYV little-endian (0x56595559), wxh=16x16, type=0
[ 6816.933696] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=UYVY little-endian (0x59565955), type=2
[ 6816.933719] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=UYVY little-endian (0x59565955), wxh=16x16, type=0
[ 6816.933774] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=NV12 little-endian (0x3231564e), type=2
[ 6816.933795] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=NV12 little-endian (0x3231564e), wxh=16x16, type=0
[ 6816.933848] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=YU12 little-endian (0x32315559), type=2
[ 6816.933869] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=YU12 little-endian (0x32315559), wxh=16x16, type=0
[ 6816.933918] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=YV12 little-endian (0x32315659), type=2
[ 6816.933939] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=YV12 little-endian (0x32315659), wxh=16x16, type=0
[ 6816.934006] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=BA24 little-endian (0x34324142), type=2
[ 6816.934027] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=BA24 little-endian (0x34324142), wxh=16x16, type=0
[ 6816.934077] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=RGB3 little-endian (0x33424752), type=2
[ 6816.934098] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=RGB3 little-endian (0x33424752), wxh=16x16, type=0
[ 6816.934147] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=NV61 little-endian (0x3136564e), type=2
[ 6816.934168] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=NV61 little-endian (0x3136564e), wxh=16x16, type=0
[ 6816.934217] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=NV16 little-endian (0x3631564e), type=2
[ 6816.943391] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=NV16 little-endian (0x3631564e), wxh=16x16, type=0
[ 6816.943582] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=NV21 little-endian (0x3132564e), type=2
[ 6816.943606] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=NV21 little-endian (0x3132564e), wxh=16x16, type=0
[ 6816.943659] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=RGBP little-endian (0x50424752), type=2
[ 6816.943680] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=RGBP little-endian (0x50424752), wxh=16x16, type=0
[ 6816.943753] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=GREY little-endian (0x59455247), type=2
[ 6816.943775] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=GREY little-endian (0x59455247), wxh=16x16, type=0
[ 6816.943826] video3: VIDIOC_ENUM_FRAMESIZES: index=0, pixelformat=YVYU little-endian (0x55595659), type=2
[ 6816.943847] video3: VIDIOC_ENUM_FRAMEINTERVALS: error -25: index=0, pixelformat=YVYU little-endian (0x55595659), wxh=16x16, type=0
[ 6816.944287] video3: VIDIOC_TRY_FMT: type=vid-cap, width=640, height=480, pixelformat=RGBP little-endian (0x50424752), field=none, bytesperline=1280, sizeimage=614400, colorspace=8, flags=0x0, ycbc
r_enc=0, quantization=1, xfer_func=2
[ 6816.950801] video3: VIDIOC_G_SELECTION: error -25: type=vid-cap, target=1, flags=0x0, wxh=0x0, x,y=0,0
[ 6816.950845] video3: VIDIOC_CROPCAP: error -25: type=0, bounds wxh=0x0, x,y=0,0, defrect wxh=0x0, x,y=0,0, pixelaspect 0/0
[ 6816.950929] video3: VIDIOC_S_FMT: type=vid-cap, width=640, height=480, pixelformat=RGBP little-endian (0x50424752), field=none, bytesperline=1280, sizeimage=614400, colorspace=8, flags=0x0, ycbcr_
enc=0, quantization=1, xfer_func=2
[ 6816.950980] video3: VIDIOC_G_PARM: error -25: type=vid-cap, capability=0x0, capturemode=0x0, timeperframe=0/0, extendedmode=0, readbuffers=0
[ 6816.951010] video3: VIDIOC_G_SELECTION: error -25: type=vid-cap, target=257, flags=0x0, wxh=0x0, x,y=0,0
[ 6816.951033] video3: VIDIOC_EXPBUF: error -22: fd=0, type=vid-cap, index=4294967295, plane=4294967295, flags=0x00080002
[ 6816.951055] video3: VIDIOC_G_CTRL: error -25: name=Min Number of Capture Buffers, id=0x980927, value=0
[ 6816.951490] video3: VIDIOC_REQBUFS: count=0, type=vid-cap, memory=mmap
[ 6816.951516] video3: VIDIOC_CREATE_BUFS: index=0, count=0, memory=mmap, capabilities=0x00000015, type=vid-cap, width=640, height=480, pixelformat=RGBP little-endian (0x50424752), field=none, bytesp
erline=1280, sizeimage=614400, colorspace=8, flags=0x0, ycbcr_enc=0, quantization=1, xfer_func=2
[ 6816.951552] video3: VIDIOC_REQBUFS: error -22: count=0, type=vid-cap, memory=userptr
[ 6816.951571] video3: VIDIOC_REQBUFS: count=0, type=vid-cap, memory=dmabuf
[ 6816.951587] video3: VIDIOC_CREATE_BUFS: index=0, count=0, memory=dmabuf, capabilities=0x00000015, type=vid-cap, width=640, height=480, pixelformat=RGBP little-endian (0x50424752), field=none, byte
sperline=1280, sizeimage=614400, colorspace=8, flags=0x0, ycbcr_enc=0, quantization=1, xfer_func=2
[ 6816.951749] video3: VIDIOC_G_INPUT: error -25: value=0
[ 6816.951770] video3: VIDIOC_SUBSCRIBE_EVENT: error -25: type=0x5, id=0x0, flags=0x0
[ 6816.951804] video3: VIDIOC_G_CTRL: error -25: name=Min Number of Capture Buffers, id=0x980927, value=0
[ 6816.955377] video3: VIDIOC_REQBUFS: count=4, type=vid-cap, memory=mmap
[ 6816.955436] video3: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0x0, length=614400
[ 6816.955462] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6816.955477] video3: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0x96000, length=614400
[ 6816.955501] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6816.955512] video3: VIDIOC_QUERYBUF: 00:00:00.000000 index=2, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0x12c000, length=614400
[ 6816.955536] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6816.955547] video3: VIDIOC_QUERYBUF: 00:00:00.000000 index=3, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0x1c2000, length=614400
[ 6816.955571] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6816.955669] video3: VIDIOC_EXPBUF: fd=16, type=vid-cap, index=0, plane=0, flags=0x00080002
[ 6816.955833] video3: VIDIOC_EXPBUF: fd=17, type=vid-cap, index=1, plane=0, flags=0x00080002
[ 6816.955884] video3: VIDIOC_EXPBUF: fd=18, type=vid-cap, index=2, plane=0, flags=0x00080002
[ 6816.955930] video3: VIDIOC_EXPBUF: fd=22, type=vid-cap, index=3, plane=0, flags=0x00080002
[ 6816.956006] video3: VIDIOC_STREAMON: type=vid-cap
[ 6817.188158] video5: VIDIOC_STREAMOFF: type=meta-cap
[ 6817.188442] video5: VIDIOC_S_EXT_CTRLS: which=0x9f0000, count=1, error_idx=0, request_fd=0, name=ISP Stat Location, id/val=0x9f090b/0x1
[ 6817.188523] video5: VIDIOC_QUERYCAP: driver=dcmipp, card=dcmipp_statcap, bus=platform:dcmipp, version=0x0006011c, capabilities=0x85a00000, device_caps=0x05a00000
[ 6817.188553] video5: VIDIOC_G_FMT: type=meta-cap, dataformat=STIS little-endian (0x53495453), buffersize=60
[ 6817.188618] video5: VIDIOC_REQBUFS: count=2, type=meta-cap, memory=mmap
[ 6817.188641] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=0, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00000000, length=60
[ 6817.188672] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[ 6817.188725] video5: VIDIOC_QUERYBUF: 00:00:00.000000 index=1, type=meta-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xffff00001000, length=60
[ 6817.188757] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
5.2.2. V4L2 core framework tracing
Tracing of the V4L2 core framework[4] can be enabled using commands:
echo 0x3 > /sys/module/videobuf2_common/parameters/debug
echo 0x3 > /sys/module/videobuf2_v4l2/parameters/debug
Traces are output in the kernel log buffer:
dmesg
[ 1218.626235] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 0, plane 0 offset 0x00000000
[ 1218.642445] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 1, plane 0 offset 0x004d4000
[ 1218.656398] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 2, plane 0 offset 0x009a8000
[ 1218.669843] videobuf2_common: [cap-7b50f8bf] __setup_offsets: buffer 3, plane 0 offset 0x00e7c000
[ 1218.677503] videobuf2_common: [cap-7b50f8bf] __vb2_queue_alloc: allocated 4 buffers, 1 plane(s) each
[ 1218.688361] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 0, plane 0 successfully mapped
[ 1218.696744] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 1, plane 0 successfully mapped
[ 1218.705291] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 2, plane 0 successfully mapped
[ 1218.712487] videobuf2_common: [cap-7b50f8bf] vb2_mmap: buffer 3, plane 0 successfully mapped
[ 1218.722474] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 0 succeeded
[ 1218.730528] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 1 succeeded
[ 1218.738153] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 2 succeeded
[ 1218.746012] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 3 succeeded
[ 1218.756237] videobuf2_common: [cap-7b50f8bf] vb2_core_streamon: successful
[ 1218.761969] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.822167] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.828383] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 0, state: dequeued
[ 1218.838812] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 0 succeeded
[ 1218.846549] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.855624] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.862118] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 1, state: dequeued
[ 1218.872406] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 1 succeeded
[ 1218.880029] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.889460] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.896756] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 2, state: dequeued
[ 1218.905383] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 2 succeeded
[ 1218.912309] videobuf2_common: [cap-7b50f8bf] __vb2_wait_for_done_vb: will sleep waiting for buffers
[ 1218.923080] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: returning done buffer
[ 1218.930392] videobuf2_common: [cap-7b50f8bf] vb2_core_dqbuf: dqbuf of buffer 3, state: dequeued
[ 1218.944499] videobuf2_common: [cap-7b50f8bf] vb2_core_qbuf: qbuf of buffer 3 succeeded
[ 1218.952210] videobuf2_common: [cap-7b50f8bf] vb2_core_streamoff: successful
[ 1218.959921] videobuf2_common: [cap-7b50f8bf] vb2_core_streamoff: successful
[ 1218.968800] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 0
[ 1218.978419] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 1
[ 1218.988204] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 2
[ 1218.997501] videobuf2_common: [cap-7b50f8bf] __vb2_buf_mem_free: freed plane 0 of buffer 3
5.2.3. DCMIPP V4L2 kernel driver tracing
DCMIPP dynamic debug traces[8] can be enabled using commands:
echo "module stm32_csi2host +p" > /sys/kernel/debug/dynamic_debug/control
echo "module stm32_dcmipp +p" > /sys/kernel/debug/dynamic_debug/control
Here is an example of a preview:
dmesg
root@stm32mp2:~# v4l2-ctl -d /dev/video2 --set-fmt-video=width=640,height=480,pi
xelformat=YUYV --stream-mmap --stream-count=1
[259799.630709] dcmipp 48030000.dcmipp: dcmipp_main_capture: format update: old:2592x1940 (0x56595559, 8, 1, 2, 0) new:640x480 (0x56595559, 8, 1, 2, 0)
[259799.638552] dcmipp 48030000.dcmipp: Setup queue, count=4, size=614400
[259799.649146] dcmipp 48030000.dcmipp: buffer[0] phy=0x00000000f2a00000 size=614400
[259799.652512] dcmipp 48030000.dcmipp: buffer[1] phy=0x00000000f2b00000 size=614400
[259799.659894] dcmipp 48030000.dcmipp: buffer[2] phy=0x00000000f2c00000 size=614400
[259799.667382] dcmipp 48030000.dcmipp: buffer[3] phy=0x00000000f2d00000 size=614400
[259799.674776] dcmipp 48030000.dcmipp: Queue [0] 00000000120b7308 phy=0x00000000f2a00000
[259799.682661] dcmipp 48030000.dcmipp: Queue [1] 0000000080a04423 phy=0x00000000f2b00000
[259799.690435] dcmipp 48030000.dcmipp: Queue [2] 00000000aa660f9b phy=0x00000000f2c00000
[259799.698313] dcmipp 48030000.dcmipp: Queue [3] 00000000ba83d671 phy=0x00000000f2d00000
[259799.706332] dcmipp 48030000.dcmipp: CLR DCMIPP_PxFCTCR(pixelproc->pipe_id) 0x00000003
[259799.714117] dcmipp 48030000.dcmipp: WR DCMIPP_PxFCTCR(pixelproc->pipe_id) 0x00000000
[259799.722017] dcmipp 48030000.dcmipp: SET DCMIPP_PxFCTCR(pixelproc->pipe_id) 0x00000000
[259799.729913] dcmipp 48030000.dcmipp: WR DCMIPP_PxFCTCR(pixelproc->pipe_id) 0x00000000
[259799.737893] dcmipp 48030000.dcmipp: WR DCMIPP_PxCRSTR(pixelproc->pipe_id) 0x00000000
[259799.745679] dcmipp 48030000.dcmipp: WR DCMIPP_PxCRSZR(pixelproc->pipe_id) 0x87940a20
[259799.753556] dcmipp 48030000.dcmipp: dcmipp_main_postproc: decimation config: hdec: 0x0, vdec: 0x0
[259799.762434] dcmipp 48030000.dcmipp: dcmipp_main_postproc: downsize config: hratio: 0x8199, vratio: 0x8155, hdiv: 0xfc, vdiv: 0xfd
[259799.774135] dcmipp 48030000.dcmipp: CLR DCMIPP_PxDCCR(pixelproc->pipe_id) 0x00000001
[259799.781908] dcmipp 48030000.dcmipp: WR DCMIPP_PxDCCR(pixelproc->pipe_id) 0x00000000
[259799.789681] dcmipp 48030000.dcmipp: CLR DCMIPP_PxDSCR(pixelproc->pipe_id) 0x80000000
[259799.797556] dcmipp 48030000.dcmipp: WR DCMIPP_PxDSCR(pixelproc->pipe_id) 0x00fd00fc
[259799.805329] dcmipp 48030000.dcmipp: WR DCMIPP_PxDSRTIOR(pixelproc->pipe_id) 0x81558199
[259799.813400] dcmipp 48030000.dcmipp: WR DCMIPP_PxDSSZR(pixelproc->pipe_id) 0x01e00280
[259799.821273] dcmipp 48030000.dcmipp: WR DCMIPP_PxDSCR(pixelproc->pipe_id) 0x80fd00fc
[259799.829044] dcmipp 48030000.dcmipp: cconv_matrices[0][1][1][1]=000000008be3128f
[259799.836428] dcmipp 48030000.dcmipp: color conversion RGB-limited => YUV601-limited
[259799.844001] dcmipp 48030000.dcmipp: WR DCMIPP_P1YUVRR1 + (4 * i) 0x07920083
[259799.851174] dcmipp 48030000.dcmipp: WR DCMIPP_P1YUVRR1 + (4 * i) 0x008007eb
[259799.858252] dcmipp 48030000.dcmipp: WR DCMIPP_P1YUVRR1 + (4 * i) 0x0096004d
[259799.865430] dcmipp 48030000.dcmipp: WR DCMIPP_P1YUVRR1 + (4 * i) 0x0000001d
[259799.872497] dcmipp 48030000.dcmipp: WR DCMIPP_P1YUVRR1 + (4 * i) 0x07a907d4
[259799.879566] dcmipp 48030000.dcmipp: WR DCMIPP_P1YUVRR1 + (4 * i) 0x00800083
[259799.886737] dcmipp 48030000.dcmipp: WR DCMIPP_P1YUVCR 0x00000005
[259799.892804] dcmipp 48030000.dcmipp: WR DCMIPP_PxPPCR(pixelproc->pipe_id) 0x00000006
[259799.900689] dcmipp 48030000.dcmipp: WR DCMIPP_PxGMCR(pixelproc->pipe_id) 0x00000001
[259799.908467] dcmipp 48030000.dcmipp: "dcmipp_main_postproc" is started
[259799.914936] dcmipp 48030000.dcmipp: SET DCMIPP_P1FSCR 0x00040000
[259799.921001] dcmipp 48030000.dcmipp: WR DCMIPP_P1FSCR 0x0004002b
[259799.927063] dcmipp 48030000.dcmipp: WR DCMIPP_P1SRCR 0x00008794
[259799.933129] dcmipp 48030000.dcmipp: WR DCMIPP_P1DECR 0x00000000
[259799.939194] dcmipp 48030000.dcmipp: CLR DCMIPP_P1DMCR 0x00000007
[259799.945262] dcmipp 48030000.dcmipp: WR DCMIPP_P1DMCR 0x00000000
[259799.951323] dcmipp 48030000.dcmipp: Input is RawBayer, enable Demosaicing
[259799.958190] dcmipp 48030000.dcmipp: SET DCMIPP_P1DMCR 0x00000001
[259799.964260] dcmipp 48030000.dcmipp: WR DCMIPP_P1DMCR 0x00000001
[259799.970330] dcmipp 48030000.dcmipp: WR DCMIPP_P1CCRR1 + (4 * i) 0x00000000
[259799.977304] dcmipp 48030000.dcmipp: WR DCMIPP_P1CCRR1 + (4 * i) 0x00000000
[259799.984379] dcmipp 48030000.dcmipp: WR DCMIPP_P1CCRR1 + (4 * i) 0x00000000
[259799.991347] dcmipp 48030000.dcmipp: WR DCMIPP_P1CCRR1 + (4 * i) 0x00000000
[259799.998429] dcmipp 48030000.dcmipp: WR DCMIPP_P1CCRR1 + (4 * i) 0x00000000
[259800.005405] dcmipp 48030000.dcmipp: WR DCMIPP_P1CCRR1 + (4 * i) 0x00000000
[259800.012476] dcmipp 48030000.dcmipp: WR DCMIPP_P1CCCR 0x00000000
[259800.018536] dcmipp 48030000.dcmipp: "dcmipp_main_isp" is started
[259800.024611] dcmipp 48030000.dcmipp: CLR DCMIPP_P0FSCR 0x0003003f
[259800.030675] dcmipp 48030000.dcmipp: WR DCMIPP_P0FSCR 0x00000000
[259800.036740] dcmipp 48030000.dcmipp: SET DCMIPP_P0FSCR 0x0000002b
[259800.042807] dcmipp 48030000.dcmipp: WR DCMIPP_P0FSCR 0x0000002b
[259800.048874] dcmipp 48030000.dcmipp: CLR DCMIPP_P1FSCR 0x0000003f
[259800.054936] dcmipp 48030000.dcmipp: WR DCMIPP_P1FSCR 0x00040000
[259800.061001] dcmipp 48030000.dcmipp: SET DCMIPP_P1FSCR 0x0000002b
[259800.067061] dcmipp 48030000.dcmipp: WR DCMIPP_P1FSCR 0x0004002b
[259800.073179] dcmipp 48030000.dcmipp: WR DCMIPP_CMCR 0x00000001
[259800.079054] dcmipp 48030000.dcmipp: "dcmipp_input" is started
[259800.084836] stm32-csi2host 48020000.csi2host: Starting the CSI2
[259800.090802] stm32-csi2host 48020000.csi2host: Computed Mbps: 1188
[259800.096968] stm32-csi2host 48020000.csi2host: PHY settings: (1200 Mbps, 11 HS FRange, 460 OSC Freq)
[259800.106227] stm32-csi2host 48020000.csi2host: VC0: enable DT0 (0x2b) - DT0FT (0x3)
[259800.113730] dcmipp 48030000.dcmipp: "stm32_csi2host.48020000.csi2hos" is started
[259800.173894] dcmipp 48030000.dcmipp: "imx335 0-001a" is started
[259800.174240] dcmipp 48030000.dcmipp: SET DCMIPP_CMIER 0x00860000
[259800.180219] dcmipp 48030000.dcmipp: WR DCMIPP_CMIER 0x00860000
[259800.186202] dcmipp 48030000.dcmipp: SET DCMIPP_PxFSCR(vcap->pipe_id) 0x80000000
[259800.193582] dcmipp 48030000.dcmipp: WR DCMIPP_PxFSCR(vcap->pipe_id) 0x8004002b
[259800.200964] dcmipp 48030000.dcmipp: Start with next [0] 00000000120b7308 phy=0x00000000f2a00000
[259800.210355] dcmipp 48030000.dcmipp: WR DCMIPP_PxPPM0AR1(vcap->pipe_id) 0xf2a00000
[259800.217351] dcmipp 48030000.dcmipp: WR DCMIPP_PxPPM0PR(vcap->pipe_id) 0x00000500
[259800.224833] dcmipp 48030000.dcmipp: SET DCMIPP_PxFCTCR(vcap->pipe_id) 0x00000008
[259800.232349] dcmipp 48030000.dcmipp: RD DCMIPP_CMSR2 0x00050000
[259800.232367] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.232375] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00040000
[259800.232382] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.256245] dcmipp 48030000.dcmipp: WR DCMIPP_PxPPM0AR1(vcap->pipe_id) 0xf2b00000
[259800.263806] dcmipp 48030000.dcmipp: Write [1] 0000000080a04423 phy=0x00000000f2b00000
[259800.271818] dcmipp 48030000.dcmipp: RD DCMIPP_CMSR2 0x00050000
[259800.275336] dcmipp 48030000.dcmipp: WR DCMIPP_PxFCTCR(vcap->pipe_id) 0x00000008
[259800.277763] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.291141] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00040000
[259800.297084] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.303171] dcmipp 48030000.dcmipp: WR DCMIPP_PxPPM0AR1(vcap->pipe_id) 0xf2a00000
[259800.310729] dcmipp 48030000.dcmipp: Write [0] 00000000120b7308 phy=0x00000000f2a00000
[259800.318617] dcmipp 48030000.dcmipp: RD DCMIPP_CMSR2 0x00050000
[259800.324565] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.330609] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00040000
[259800.336548] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.342638] dcmipp 48030000.dcmipp: WR DCMIPP_PxPPM0AR1(vcap->pipe_id) 0xf2b00000
[259800.350197] dcmipp 48030000.dcmipp: Write [1] 0000000080a04423 phy=0x00000000f2b00000
[259800.358091] dcmipp 48030000.dcmipp: RD DCMIPP_CMSR2 0x00030000
[259800.364040] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.370085] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00020000
[259800.376026] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.382129] dcmipp 48030000.dcmipp: RD DCMIPP_CMSR2 0x00050000
[259800.382171] dcmipp 48030000.dcmipp: Queue [0] 00000000120b7308 phy=0x00000000f2a00000
[259800.387978] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.401853] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00040000
[259800.401887] dcmipp 48030000.dcmipp: "dcmipp_main_postproc" is stopped
[259800.407892] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.407947] dcmipp 48030000.dcmipp: WR DCMIPP_PxPPM0AR1(vcap->pipe_id) 0xf2c00000
[259800.417742] dcmipp 48030000.dcmipp: "dcmipp_main_isp" is stopped
[259800.420301] dcmipp 48030000.dcmipp: Write [2] 00000000aa660f9b phy=0x00000000f2c00000
[259800.420343] dcmipp 48030000.dcmipp: RD DCMIPP_CMSR2 0x00030000
[259800.420351] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.431354] dcmipp 48030000.dcmipp: "dcmipp_input" is stopped
[259800.434024] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00020000
[259800.434031] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.434079] dcmipp 48030000.dcmipp: RD DCMIPP_CMSR2 0x00050000
[259800.460308] stm32-csi2host 48020000.csi2host: Call csi2host_stop for test purpose
[259800.465714] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.465722] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00040000
[259800.465729] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.471715] stm32-csi2host 48020000.csi2host: Stopping the CSI2
[259800.477737] dcmipp 48030000.dcmipp: WR DCMIPP_PxPPM0AR1(vcap->pipe_id) 0xf2d00000
[259800.490239] stm32-csi2host 48020000.csi2host: Call csi2host_stop for test purpose
[259800.491190] dcmipp 48030000.dcmipp: Write [3] 00000000ba83d671 phy=0x00000000f2d00000
[259800.499136] stm32-csi2host 48020000.csi2host: Stopping the CSI2
[259800.503199] dcmipp 48030000.dcmipp: RD DCMIPP_CMSR2 0x00030000
[259800.503208] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.511662] stm32-csi2host 48020000.csi2host: Runtime PM usage count underflow!
[259800.516785] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00020000
[259800.516792] dcmipp 48030000.dcmipp: WR DCMIPP_CMFCR 0x00000000
[259800.624700] dcmipp 48030000.dcmipp: "stm32_csi2host.48020000.csi2hos" is stopped
[259800.632355] dcmipp 48030000.dcmipp: "imx335 0-001a" is stopped
[259800.638050] dcmipp 48030000.dcmipp: CLR DCMIPP_CMIER 0x00860000
[259800.644023] dcmipp 48030000.dcmipp: WR DCMIPP_CMIER 0x00000000
[259800.649986] dcmipp 48030000.dcmipp: CLR DCMIPP_PxFCTCR(vcap->pipe_id) 0x00000008
[259800.657464] dcmipp 48030000.dcmipp: WR DCMIPP_PxFCTCR(vcap->pipe_id) 0x00000000
[259800.664941] dcmipp 48030000.dcmipp: CLR DCMIPP_PxFSCR(vcap->pipe_id) 0x80000000
[259800.672223] dcmipp 48030000.dcmipp: WR DCMIPP_PxFSCR(vcap->pipe_id) 0x0004002b
[259800.679632] dcmipp 48030000.dcmipp: RD DCMIPP_PRSR 0x00030000
[259800.685475] dcmipp 48030000.dcmipp: [DCMIPP_PRSR] =0x00030000
[259800.691415] dcmipp 48030000.dcmipp: RD DCMIPP_PxSR(vcap->pipe_id) 0x00000007
[259800.698560] dcmipp 48030000.dcmipp: [DCMIPP_PxSR] =0x00000007
[259800.704396] dcmipp 48030000.dcmipp: RD DCMIPP_CMSR1 0x00000003
[259800.710434] dcmipp 48030000.dcmipp: [DCMIPP_CMSR1] =0x00000003
[259800.716272] dcmipp 48030000.dcmipp: RD DCMIPP_CMSR2 0x00010000
[259800.722310] dcmipp 48030000.dcmipp: [DCMIPP_CMSR2] =0x00010000
[259800.728785] dcmipp 48030000.dcmipp: Stop streaming, errors=0 (overrun=0, nactive=0), underrun=0, vsync=5, frame=3, buffers=3, it=8
5.3. How to debug
5.3.1. Errors
Errors are unconditionally traced in the kernel log:
dmesg
[ 1947.932915] dcmipp-bytecap dcmipp-bytecap.3.auto: Stop streaming, errors=1 (overrun=0, limit=0), vsync=2, frame=2, buffers=2, it=4
5.3.2. Memory tracking
Frames require large chunks of contiguous memory. They are allocated by V4L2 framework through the DMA back-end. Those allocations can be traced using:
echo "module dma_contiguous +p" > /sys/kernel/debug/dynamic_debug/control
echo "module videobuf2_dma_contig +p" > /sys/kernel/debug/dynamic_debug/control
Here is the trace after a VGA YUV422 capture.
[11311.617688] vb2_dc_mmap: mapped dma addr 0xf1900000 at 0xb3b6a000, size 614400
[11311.617986] vb2_dc_mmap: mapped dma addr 0xf1a00000 at 0xb3ad4000, size 614400
[11311.618071] vb2_dc_mmap: mapped dma addr 0xf1b00000 at 0xb3a3e000, size 614400
[11311.764146] vb2_dc_mmap: mapped dma addr 0xf1c00000 at 0xb307c000, size 614400
Four frames of VGA YUV422 frames: 640x480x2=614400 bytes
6. Source code location
6.1. User space
6.2. Kernel space
- V4L2 core source code
- stm32-dcmipp V4L2 driver source code
- i2c camera sensor V4L2 drivers source code
7. References
- ↑ Information about V4L2 Linux kernel framework on wikipedia.
- ↑ Jump up to: 2.0 2.1 Linux Media Infrastructure userspace API » Part IV - Media Controller API
- ↑ Jump up to: 3.0 3.1 3.2 3.3 Linux Media Infrastructure userspace API » Part I - Video for Linux API
- ↑ Jump up to: 4.0 4.1 Media subsystem kernel internal API » 1. Video4Linux devices
- ↑ MB1854 camera daughter board
- ↑ CN4 Camera sensor connector
- ↑ Linux Media Infrastructure userspace API » Part I - Video for Linux API » 4. Interfaces » 4.13. Sub-device Interface » 4.13.3.2. Selections: cropping, scaling and composition
- ↑ How to use the kernel dynamic debug