How to stream RAW camera over network

Applicable for

STM32MP25x lines

---

1. Overview

This article will explain how to stream camera content over network thanks to GStreamer application on top of V4L2 Linux^® kernel framework.

This article focus on camera sensor which doesn't output compressed image or video content but raw content such as YUV, RGB or Raw-bayer images.

Remote sending of images with a decent framerate requires compression in order to respect a reasonable bandwidth on network. This could be achieved compressing to JPEG image format or video formats like VP8 or H264.

Find below some examples of command lines allowing to capture a continuous stream of raw images then compress to JPEG, VP8 or H264 stream while playing it using various multimedia players, either local or remote.

2. Local streaming

Here is an example of a local preview loopback using gst-launch to capture raw pictures then compress to JPEG then decode and display.

For STM32MP255 line , setup the camera subsystem first:

/usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh #wait few seconds that image is correct then hit "CTRL+C" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/640x480]"

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="image/jpeg" ! decodebin ! autovideosink

3. UDP streaming

An internet connection is required, for example by plugging an ethernet cable on the:

STM32MP257x-EV1 Evaluation board

CN17 Ethernet connector 1

For STM32MP255 line , setup the camera subsystem first:

/usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh <wait few seconds that image is correct then hit "CTRL+C"> media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/640x480]"

Get the IP address aa.bb.cc.dd of the host PC using ifconfig command:

 ifconfig | grep "inet"
   inet addr:aa.bb.cc.dd  Bcast:10.201.23.255  Mask:255.255.252.0
   inet addr:127.0.0.1  Mask:255.0.0.0

Then fill the host= udpsink property with this IP address on the remote side in order to send UDP JPEG stream:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="image/jpeg" ! rtpjpegpay ! udpsink host=aa.bb.cc.dd port=5000

Then play the UDP JPEG stream on host PC:

 gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=JPEG ! rtpjpegdepay ! jpegparse ! decodebin ! videoconvert ! autovideosink sync=false

A new window will popup on host PC displaying the camera content.

Same can be done with a lower bandwidth thanks to video compression. Here is an example with VP8:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-vp8" ! rtpvp8pay ! udpsink host=aa.bb.cc.dd port=5000

Then play the UDP VP8 stream on host PC:

 gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=VP8 ! rtpvp8depay ! decodebin ! videoconvert ! autovideosink sync=false

Another example with H264:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-h264" ! h264parse config-interval=1 ! rtph264pay ! udpsink host=aa.bb.cc.dd port=5000

Then play the UDP H264 stream on host PC:

 gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=H264 ! rtph264depay ! decodebin ! videoconvert ! autovideosink sync=false

Another H264 streaming example giving a 200Kb/s bitrate constraint to limit network bandwidth:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-h264|element-properties,rate-control=1,bitrate=200000" ! h264parse config-interval=1 ! rtph264pay ! udpsink host=aa.bb.cc.dd port=5000

Information

Refer to V4L2_video_codec_overview#Controlling_encoder section for more details about video encoder available controls