Last edited 3 weeks ago

How to stream RAW camera over network

Applicable for STM32MP25x lines

1. Overview[edit source]

This article will explain how to stream camera content over network thanks to GStreamer application on top of V4L2 Linux® kernel framework.

This article focus on camera sensor which doesn't output compressed image or video content but raw content such as YUV, RGB or Raw-bayer images.

Remote sending of images with a decent framerate requires compression in order to respect a reasonable bandwidth on network. This could be achieved compressing to JPEG image format or video formats like VP8 or H264.

Find below some examples of command lines allowing to capture a continuous stream of raw images then compress to JPEG, VP8 or H264 stream while playing it using various multimedia players, either local or remote.

2. Local streaming[edit source]

Here is an example of a local preview loopback using gst-launch to capture raw pictures then compress to JPEG then decode and display.



For STM32MP255 line More info.png, setup the camera subsystem first:

/usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh
#wait few seconds that image is correct then hit "CTRL+C"
media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" 
media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/640x480]"
 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="image/jpeg" ! decodebin ! autovideosink

3. UDP streaming[edit source]

An internet connection is required, for example by plugging an ethernet cable on the:

STM32MP257x-EV1 Evaluation board More info green.png CN17 Ethernet connector 1


For STM32MP255 line More info.png, setup the camera subsystem first:
 /usr/local/demo/application/camera/bin/launch_camera_preview_mp25.sh
<wait few seconds that image is correct then hit "CTRL+C">
 media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':0[compose:(0,0)/640x480]" 
 media-ctl -d /dev/media2 --set-v4l2 "'dcmipp_main_postproc':1[fmt:YUYV8_2X8/640x480]"


Get the IP address aa.bb.cc.dd of the host PC using ifconfig command:

 ifconfig | grep "inet"
   inet addr:aa.bb.cc.dd  Bcast:10.201.23.255  Mask:255.255.252.0
   inet addr:127.0.0.1  Mask:255.0.0.0

Then fill the host= udpsink property with this IP address on the remote side in order to send UDP JPEG stream:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="image/jpeg" ! rtpjpegpay ! udpsink host=aa.bb.cc.dd port=5000

Then play the UDP JPEG stream on host PC:

 gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=JPEG ! rtpjpegdepay ! jpegparse ! decodebin ! videoconvert ! autovideosink sync=false

A new window will popup on host PC displaying the camera content.

Same can be done with a lower bandwidth thanks to video compression. Here is an example with VP8:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-vp8" ! rtpvp8pay ! udpsink host=aa.bb.cc.dd port=5000

Then play the UDP VP8 stream on host PC:

 gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=VP8 ! rtpvp8depay ! decodebin ! videoconvert ! autovideosink sync=false

Another example with H264:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-h264" ! h264parse config-interval=1 ! rtph264pay ! udpsink host=aa.bb.cc.dd port=5000

Then play the UDP H264 stream on host PC:

 gst-launch-1.0 udpsrc port=5000 ! application/x-rtp, encoding-name=H264 ! rtph264depay ! decodebin ! videoconvert ! autovideosink sync=false

Another H264 streaming example giving a 200Kb/s bitrate constraint to limit network bandwidth:

 gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw, format=YUY2, width=640, height=480, framerate=30/1 ! encodebin profile="video/x-h264|element-properties,rate-control=1,bitrate=200000" ! h264parse config-interval=1 ! rtph264pay ! udpsink host=aa.bb.cc.dd port=5000
Info white.png Information
Refer to V4L2_video_codec_overview#Controlling_encoder section for more details about video encoder available controls