X-LINUX-AI is an STM32 MPU OpenSTLinux Expansion Package that targets artificial intelligence for STM32MP1 Series devices. It contains Linux AI frameworks, as well as application examples to get started with some basic use cases such as computer vision (CV). It is composed of an OpenEmbedded meta layer, named meta-st-stm32mpu-ai, to be added on top of the STM32MP1 Distribution Package. It brings a complete and coherent easy-to-build / install environment to take advantage of AI on STM32MP1 Series devices.
1. X-LINUX-AI v2.2.0[edit source]
1.1. Contents[edit source]
TensorFlow™ Lite[1] 2.8.0- Coral Edge TPU™[2] accelerator native support
- libedgetpu 2.0.0 (Gouper) aligned with TensorFlow™ Lite 2.8.0
- libcoral 2.0.0 (Gouper) aligned with TensorFlow™ Lite 2.8.0
- PyCoral 2.0.0 (Gouper) aligned with TensorFlow™ Lite 2.8.0
- OpenCV[3] 4.5.x
- Python™[4] 3.10.x (enabling Pillow module)
- Application samples
- C++ / Python image classification using TensorFlow™ Lite based on MobileNet v1 quantized model
- C++ / Python object detection using TensorFlow™ Lite based on COCO SSD MobileNet v1 quantized model
- C++ / Python image classification using Coral Edge TPU™ based on MobileNet v1 quantized model and compiled for the Coral Edge TPU™
- C++ / Python object detection using Coral Edge TPU™ based on COCO SSD MobileNet v1 quantized model and compiled for the Coral Edge TPU™
- C++ face recognition using TensorFlow™ Lite models capable of recognizing the face of a known (enrolled) user (available on demand)
- Application support for the 720p, 480p, and 272p display configurations
- Application user interface with updated look and feel
- Python™ and C++ application rework for better performance
- X-LINUX-AI SDK add-on extending the OpenSTLinux SDK with AI functionality to develop and build an AI application easily. The X-LINUX-AI SDK add-on provides support for all the above frameworks. It is available from the X-LINUX-AI product page
1.2. Validated hardware[edit source]
As any software expansion package, the X-LINUX-AI is supported on all STM32MP1 Series and it has been validated on the following boards:
1.3. Software structure[edit source]
2. Starter package: install from the OpenSTLinux AI package repository[edit source]
All the generated X-LINUX-AI packages are available from the OpenSTLinux AI package repository service hosted at the non-browsable URL http://extra.packages.openstlinux.st.com/AI.
This repository contains AI packages that can be simply installed using apt-* utilities, which the same as those used on a Debian system:
- the main group contains the selection of AI packages whose installation is automatically tested by STMicroelectronics
- the updates group is reserved for future uses such as package revision update.
You can install them individually or by package group.
2.1. Prerequisites[edit source]
- Flash the Starter Package on your SDCard
- For OpenSTLinux ecosystem release v4.1.0
:
- For OpenSTLinux ecosystem release v4.1.0
- For OpenSTLinux ecosystem release v3.1.0 :
- Your board has an internet connection either through the network cable or through a WiFi connection.
2.2. Configure the AI OpenSTLinux package repository[edit source]
Once the board is booted, execute the following command in the console in order to configure the AI OpenSTLinux package repository:
For ecosystem release v4.1.0
For ecosystem release v3.1.0 :
wget http://extra.packages.openstlinux.st.com/AI/3.1/pool/config/a/apt-openstlinux-ai/apt-openstlinux-ai_1.0_armhf.deb
dpkg -i apt-openstlinux-ai_1.0_armhf.deb
Then synchronize the AI OpenSTLinux package repository.
apt-get update
2.3. Install AI packages[edit source]
2.3.1. Install all X-LINUX-AI packages[edit source]
| Command | Description |
|---|---|
apt-get install packagegroup-x-linux-ai |
Install all the X-LINUX-AI packages (TensorFlow™ Lite, Edge TPU™, application samples and tools) |
[edit source]
| Command | Description |
|---|---|
apt-get install packagegroup-x-linux-ai-tflite |
Install X-LINUX-AI packages related to TensorFlow™ Lite framework (including application samples) |
apt-get install packagegroup-x-linux-ai-tflite-edgetpu |
Install X-LINUX-AI packages related to the Coral Edge TPU™ framework (including application samples) |
2.3.3. Install individual packages[edit source]
| Command | Description |
|---|---|
apt-get install libedgetpu2 |
Install libedgetpu for Coral Edge TPU™ |
apt-get install libcoral2 |
Install libcoral API for Coral Edge TPU™ |
apt-get install libtensorflow-lite-tools |
Install Tensorflow™ Lite utilities |
apt-get install libtensorflow-lite2 |
Install Tensorflow™ Lite runtime |
apt-get install python3-libtensorflow-lite |
Install Python TensorFlow™ Lite inference engine |
apt-get install python3-pycoral |
Install Python PyCoral API for Coral Edge TPU™ |
apt-get install tflite-cv-apps-edgetpu-image-classification-c++ |
Install C++ image classification example using Coral Edge TPU™ TensorFlow™ Lite API |
apt-get install tflite-cv-apps-edgetpu-image-classification-python |
Install Python image classification example using Coral Edge TPU™ TensorFlow™ Lite API |
apt-get install tflite-cv-apps-edgetpu-object-detection-c++ |
Install C++ object detection example using Coral Edge TPU™ TensorFlow™ Lite API |
apt-get install tflite-cv-apps-edgetpu-object-detection-python |
Install Python object detection example using Coral Edge TPU™ TensorFlow™ Lite API |
apt-get install tflite-cv-apps-image-classification-c++ |
Install C++ image classification using TensorFlow™ Lite |
apt-get install tflite-cv-apps-image-classification-python |
Install Python image classification example using TensorFlow™ Lite |
apt-get install tflite-cv-apps-object-detection-c++ |
Install C++ object detection example using TensorFlow™ Lite |
apt-get install tflite-cv-apps-object-detection-python |
Install Python object detection example using TensorFlow™ Lite |
apt-get install tflite-edgetpu-benchmark |
Install benchmark application for Coral Edge TPU™ models |
apt-get install tflite-models-coco-ssd-mobilenetv1 |
Install TensorFlow™ Lite COCO SSD Mobilenetv1 model |
apt-get install tflite-models-coco-ssd-mobilenetv1-edgetpu |
Install TensorFlow™ Lite COCO SSD Mobilenetv1 model for Coral Edge TPU™ |
apt-get install tflite-models-mobilenetv1 |
Install TensorFlow™ Lite Mobilenetv1 model |
apt-get install tflite-models-mobilenetv1-edgetpu |
Install TensorFlow™ Lite Mobilenetv1 model for Coral Edge TPU™ |
3. Developer package: build AI application from the X-LINUX-AI add-on SDK[edit source]
In order to easily develop and build an AI application without using the Yocto build system, X-LINUX-AI comes with a SDK add-on extending the OpenSTLinux SDK with AI functionality.
All the needed information to install and use the X-LINUX-AI SDK add-on can be found in the following article : How to install and use the X-LINUX-AI SDK add-on
4. Distribution package: re-generate X-LINUX-AI OpenSTLinux distribution[edit source]
With the following procedure, you can re-generate the complete distribution enabling the X-LINUX-AI expansion package.
This procedure is mandatory if you want to update frameworks by yourself, or if you want to modify the application samples.
4.1. Download the STM32MP1 Distribution Package[edit source]
For ecosystem release v4.1.0 :
Install the STM32MP1 Distribution Package v4.1.0, but do not initialize the OpenEmbedded environment (do not source the envsetup.sh).
For ecosystem release v3.1.0 :
Install the STM32MP1 Distribution Package v3.1.0, but do not initialize the OpenEmbedded environment (do not source the envsetup.sh).
4.2. Install X-LINUX-AI environment[edit source]
- Clone the meta-st-stm32mpu-ai git repositories
For X-LINUX-AI v2.2.0: cd <Distribution Package installation directory>/layers/meta-st git clone https://github.com/STMicroelectronics/meta-st-stm32mpu-ai.git -b v2.2.0
- Set up the build environment
cd ../..
DISTRO=openstlinux-weston MACHINE=stm32mp1 BSP_DEPENDENCY='layers/meta-st/meta-st-stm32mpu-ai' source layers/meta-st/scripts/envsetup.sh
4.3. Build the image[edit source]
bitbake st-image-ai
4.4. Flash the built image[edit source]
Follow this link to see how to flash the built image.
5. How to use the X-LINUX-AI Expansion Package[edit source]
5.1. Material needed[edit source]
To use the X-LINUX-AI OpenSTLinux Expansion Package, choose one of the following materials:
- STM32MP157F-DK2[5] + an UVC USB WebCam
- STM32MP157F-EV1[6] with the built in OV5640 parallel camera
- STM32MP157D-EV1[7] with the built in OV5640 parallel camera
Optional:
- Coral USB Edge TPU™[2] accelerator
5.2. Boot the OpenSTlinux Starter Package[edit source]
At the end of the boot sequence, the demo launcher application appears on the screen.
5.3. Install the X-LINUX-AI[edit source]
After having configured the AI OpenSTLinux package you can install the X-LINUX-AI components.
apt-get install packagegroup-x-linux-ai
And restart the demo launcher:
- For OpenSTLinux distribution with a version lower than 4.0 use
systemctl restart weston@root
- For other OpenSTLinux distribution use :
systemctl restart weston-launch
5.4. Launch an AI application sample[edit source]
Once the demo launcher is restarted, notice that it is slightly different because new AI application samples have been installed.
The demo laucher has the following appearance, and you can navigate into the different screens by using the NEXT or BACK buttons.
Screens 2, 3 and 4 contain AI application samples that are described within dedicated article available in the X-LINUX-AI application samples zoo page.
5.5. Enjoy running your own NN models[edit source]
The above examples provide application samples to demonstrate how to execute models easily on the STM32MP1.
You are free to update the C/C++ application or Python scripts for your own purposes, using your own NN models.
Source code locations are provided in application sample pages.
6. References[edit source]