X-LINUX-AI OpenSTLinux Expansion Package

X-LINUX-AI is an STM32 MPU OpenSTLinux Expansion Package that targets artificial intelligence for STM32MP1 Series devices.

It contains Linux AI frameworks, as well as application examples to get started with some basic use cases such as computer vision (CV).

It is composed of an OpenEmbedded meta layer, named meta-st-stm32mpu-ai, to be added on top of the STM32MP1 Distribution Package.

It brings a complete and coherent easy-to-build / install environment to take advantage of AI on STM32MP1 Series devices.

1. Versions

1.1. X-LINUX-AI v2.1.0

Information

This version is compatible with Yocto Project® build system Thud and Dunfell and has been validated against the OpenSTLinux ecosystem release v2.1.0 , ecosystem release v2.0.0 , ecosystem release v1.2.0 and validated on STM32MP157x-DKx, STM32MP157x-EV1 and STM32MP157 Avenger96 boards.

1.1.1. Contents

• /!\ New /!\ TensorFlow Lite 2.4.1
• Coral Edge TPU accelerator support
  • /!\ New /!\ libedgetpu 2.4.1 aligned with TensorFlow Lite 2.4.1 (built from source)
• /!\ New /!\ armNN 20.11
• /!\ New /!\ OpenCV 4.1.x
• Python 3.8.x (enabling Pillow module)
• Support STM32MP15xF devices operating at up to 800MHz
• Application samples
  • C++ / Python image classification using TensorFlow Lite based on MobileNet v1 quantized model
  • C++ / Python object detection using TensorFlow Lite based on COCO SSD MobileNet v1 quantized model
  • C++ / Python image classification using Coral Edge TPU based on MobileNet v1 quantized model and compiled for the Coral Edge TPU
  • C++ / Python object detection using Coral Edge TPU based on COCO SSD MobileNet v1 quantized model and compiled for the Coral Edge TPU
  • C++ image classification using armNN TensorFlow Lite parser based on MobileNet v1 float model
  • C++ object detection using armNN TensorFlow Lite parser based on COCO SSD MobileNet v1 quantized model
  • /!\ New /!\ C++ face recognition using TensorFlow Lite models capable of recognizing the face of a known (enrolled) user (available on demand)

Warning

The face recognition binary is available on demand. Please contact the local STMicroelectronics support for more information about this application or send a request to edge.ai@st.com

1.1.2. Validated hardware

As any software expansion package, the X-LINUX-AI is supported on all STM32MP1 Series and it has been validated on the following boards:

• STM32MP157C-DK2
• STM32MP157C-EV1
• STM32MP157A-EV1
• STM32MP157 Avenger96 board

1.1.3. Software structure

1.2. X-LINUX-AI v2.0.0

Information

This version has been validated against the OpenSTLinux ecosystem release v2.0.0 and validated on STM32MP157x-DKx and STM32MP157x-EV1 boards.

1.2.1. Contents

• TensorFlow Lite 2.2.0
• Coral Edge TPU accelerator support
• armNN 20.05
• OpenCV 4.1.x
• Python 3.8.x (enabling Pillow module)
• Support STM32MP15xF devices operating at up to 800MHz
• Python and C++ application samples
  • Image classification using TensorFlow Lite based on MobileNet v1 quantized model
  • Object detection using TensorFlow Lite based on COCO SSD MobileNet v1 quantized model
  • Image classification using Coral Edge TPU based on MobileNet v1 quantized model and compiled for the Coral Edge TPU
  • Object detection using Coral Edge TPU based on COCO SSD MobileNet v1 quantized model and compiled for the Coral Edge TPU
  • Image classification using armNN TensorFlow Lite parser based on MobileNet v1 float model
  • Object detection using armNN TensorFlow Lite parser based on COCO SSD MobileNet v1 quantized model

1.2.2. Validated hardware

As any software expansion package, the X-LINUX-AI is supported on all STM32MP1 Series and it has been validated on the following boards:

• STM32MP157C-DK2
• STM32MP157C-EV1
• STM32MP157A-EV1

1.2.3. Software structure

2. Install from the OpenSTLinux AI package repository

Information

The STMicroelectronics packages repository service is provided for evaluation purposes only, its content may be updated at any time without notice and is therefore not approved for use in production.

All the generated X-LINUX-AI packages are available from the OpenSTLinux AI package repository service hosted at the non-browsable URL http://extra.packages.openstlinux.st.com/AI.

This repository contains AI packages that can be simply installed using apt-* utilities, which the same as those used on a Debian system:

• the main group contains the selection of AI packages whose installation is automatically tested by STMicroelectronics
• the updates group is reserved for future uses such as package revision update.

You can install them individually or by package group.

2.1. Prerequisites

ST boards prerequisites:

• Flash the Starter Package on your SDCard

For OpenSTLinux ecosystem release v2.1.0 and ecosystem release v2.0.0  :

STM32MP157x-DKx Starter Package procedure

or

STM32MP157x-EV1 Starter Package procedure

For OpenSTLinux ecosystem release v1.2.0 :

STM32MP157x-DKx Starter Package procedure

or

STM32MP157x-EV1 Starter Package procedure

• Your board has an internet connection either through the network cable or through a WiFi connection.

Information

If your internet access depends on a proxy server, you should define the http_proxy environment variable with the following command before any apt-* commands: export http_proxy='http://<proxy url>:<proxy port>/'

Avenger96 board prerequisites:

• The Avenger96 board starter image supporting OpenSTLinux v2.1.0 must be flashed on to your SD Card

OpenSTLinux-2.1 based on Yocto Dunfell LTS and Linux 5.4.56 - v6.5 Starter Image

• Your board has an internet connection either through the network cable or through a WiFi connection.

Information

If your internet access depends on a proxy server, you should define the http_proxy environment variable with the following command before any apt-* commands: export http_proxy='http://<proxy url>:<proxy port>/'

2.2. Configure the AI OpenSTLinux package repository

Once the board is booted, execute the following command in the console in order to configure the AI OpenSTLinux package repository:

For ecosystem release  v2.1.0  :

 wget http://extra.packages.openstlinux.st.com/AI/2.1/pool/config/a/apt-openstlinux-ai/apt-openstlinux-ai_1.0_armhf.deb
 dpkg -i apt-openstlinux-ai_1.0_armhf.deb

For ecosystem release  v2.0.0  :
 wget http://extra.packages.openstlinux.st.com/AI/2.0/pool/config/a/apt-openstlinux-ai/apt-openstlinux-ai_1.0_armhf.deb
 dpkg -i apt-openstlinux-ai_1.0_armhf.deb


For ecosystem release  v1.2.0 :
 wget http://extra.packages.openstlinux.st.com/AI/1.2/pool/config/a/apt-openstlinux-ai/apt-openstlinux-ai_1.0_armhf.deb
 dpkg -i apt-openstlinux-ai_1.0_armhf.deb

Then synchronize the AI OpenSTLinux package repository.

 apt-get update

2.3. Install AI packages

Warning

The software package is provided AS IS, and by downloading it, you agree to be bound to the terms of the software license agreement (SLA). The detailed content licenses can be found here.

2.3.1. Install all X-LINUX-AI packages

Command	Description
apt-get install packagegroup-x-linux-ai	Install all the X-LINUX-AI packages (TensorFlow Lite, Edge TPU, armNN, application samples and tools)

2.3.2. Install AI framework related packages

Command	Description
apt-get install packagegroup-x-linux-ai-tflite	Install X-LINUX-AI packages related to TensorFlow Lite framework (including application samples)
apt-get install packagegroup-x-linux-ai-tflite-edgetpu	Install X-LINUX-AI packages related to the Edge TPU framework (including application samples)
apt-get install packagegroup-x-linux-ai-armnn-tflite	Install X-LINUX-AI packages related to the armNN framework (including application samples)

2.3.3. Install individual packages

X-LINUX-AI v2.1.0 packages

Command	Description
apt-get install arm-compute-library	Install Arm Compute Library (ACL)
apt-get install arm-compute-library-tools	Install Arm Compute Library utilities (graph examples and benchmarks)
apt-get install armnn	Install arm Neural Network SDK (armNN)
apt-get install armnn-tensorflow-lite	Install armNN TensorFlow Lite parser
apt-get install armnn-tensorflow-lite-examples	Install armNN TensorFlow Lite examples
apt-get install armnn-tfl-cv-apps-image-classification-c++	Install C++ image classification example using armNN TensorFlow Lite parser
apt-get install armnn-tfl-cv-apps-object-detection-c++	Install C++ object detection example using armNN TensorFlow Lite parser
apt-get install armnn-tools	Install armNN utilitites such as unitary tests
apt-get install python3-tensorflow-lite	Install Python TensorFlow Lite inference engine
apt-get install python3-tensorflow-lite-edgetpu	Install Python TensorFlow Lite inference engine for Edge TPU
apt-get install tensorflow-lite-edgetpu	Install Edge TPU libraries and the USB rules
apt-get install tensorflow-lite-tools	Install Tensorflow Lite utilities
apt-get install tflite-cv-apps-edgetpu-image-classification-c++	Install C++ image classification example using Coral Edge TPU TensorFlow Lite API
apt-get install tflite-cv-apps-edgetpu-image-classification-python	Install Python image classification example using Coral Edge TPU TensorFlow Lite API
apt-get install tflite-cv-apps-edgetpu-object-detection-c++	Install C++ object detection example using Coral Edge TPU TensorFlow Lite API
apt-get install tflite-cv-apps-edgetpu-object-detection-python	Install Python object detection example using Coral Edge TPU TensorFlow Lite API
apt-get install tflite-cv-apps-image-classification-c++	Install C++ image classification using TensorFlow Lite
apt-get install tflite-cv-apps-image-classification-python	Install Python image classification example using TensorFlow Lite
apt-get install tflite-cv-apps-object-detection-c++	Install C++ object detection example using TensorFlow Lite
apt-get install tflite-cv-apps-object-detection-python	Install Python object detection example using TensorFlow Lite
apt-get install tflite-edgetpu-benchmark	Install benchmark application for Edge TPU models
apt-get install tflite-models-coco-ssd-mobilenetv1	Install TensorFlow Lite COCO SSD Mobilenetv1 model
apt-get install tflite-models-coco-ssd-mobilenetv1-edgetpu	Install TensorFlow Lite COCO SSD Mobilenetv1 model for Edge TPU
apt-get install tflite-models-mobilenetv1	Install TensorFlow Lite Mobilenetv1 model
apt-get install tflite-models-mobilenetv1-edgetpu	Install TensorFlow Lite Mobilenetv1 model for Edge TPU

X-LINUX-AI v2.0.0 packages

Command	Description
apt-get install arm-compute-library	Install Arm Compute Library (ACL)
apt-get install arm-compute-library-tools	Install Arm Compute Library utilities (graph examples and benchmarks)
apt-get install armnn	Install arm Neural Network SDK (armNN)
apt-get install armnn-tensorflow-lite	Install armNN TensorFlow Lite parser
apt-get install armnn-tensorflow-lite-examples	Install armNN TensorFlow Lite examples
apt-get install armnn-tfl-benchmark	Install armNN benchmark application for TensorFlow Lite models
apt-get install armnn-tfl-cv-apps-image-classification-c++	Install C++ image classification example using armNN TensorFlow Lite parser
apt-get install armnn-tfl-cv-apps-object-detection-c++	Install C++ object detection example using armNN TensorFlow Lite parser
apt-get install armnn-tools	Install armNN utilitites such as unitary tests
apt-get install libedgetpu1	Install Edge TPU libraries and the USB rules
apt-get install python3-tensorflow-lite	Install Python TensorFlow Lite inference engine
apt-get install python3-tensorflow-lite-edgetpu	Install Python TensorFlow Lite inference engine for Edge TPU
apt-get install tensorflow-lite-tools	Install Tensorflow Lite utilities
apt-get install tflite-cv-apps-edgetpu-image-classification-c++	Install C++ image classification example using Coral Edge TPU TensorFlow Lite API
apt-get install tflite-cv-apps-edgetpu-image-classification-python	Install Python image classification example using Coral Edge TPU TensorFlow Lite API
apt-get install tflite-cv-apps-edgetpu-object-detection-c++	Install C++ object detection example using Coral Edge TPU TensorFlow Lite API
apt-get install tflite-cv-apps-edgetpu-object-detection-python	Install Python object detection example using Coral Edge TPU TensorFlow Lite API
apt-get install tflite-cv-apps-image-classification-c++	Install C++ image classification using TensorFlow Lite
apt-get install tflite-cv-apps-image-classification-python	Install Python image classification example using TensorFlow Lite
apt-get install tflite-cv-apps-object-detection-c++	Install C++ object detection example using TensorFlow Lite
apt-get install tflite-cv-apps-object-detection-python	Install Python object detection example using TensorFlow Lite
apt-get install tflite-edgetpu-benchmark	Install benchmark application for Edge TPU models
apt-get install tflite-models-coco-ssd-mobilenetv1	Install TensorFlow Lite COCO SSD Mobilenetv1 model
apt-get install tflite-models-coco-ssd-mobilenetv1-edgetpu	Install TensorFlow Lite COCO SSD Mobilenetv1 model for Edge TPU
apt-get install tflite-models-mobilenetv1	Install TensorFlow Lite Mobilenetv1 model
apt-get install tflite-models-mobilenetv1-edgetpu	Install TensorFlow Lite Mobilenetv1 model for Edge TPU

Information

If you need more information about how to use apt-* utilities check the Package repository for OpenSTLinux distribution article.

3. Re-generate X-LINUX-AI OpenSTLinux distribution

With the following procedure, you can re-generate the complete distribution enabling the X-LINUX-AI expansion package.
This procedure is mandatory if you want to update frameworks by yourself, or if you want to modify the application samples.
For further details, please expand the contents...

3.1. Download the STM32MP1 Distribution Package

For ecosystem release v2.1.0 :
Install the STM32MP1 Distribution Package v2.1.0, but do not initialize the OpenEmbedded environment (do not source the envsetup.sh).

For ecosystem release v2.0.0 :

Install the STM32MP1 Distribution Package v2.0.0, but do not initialize the OpenEmbedded environment (do not source the envsetup.sh).

For ecosystem release v1.2.0 :

Install the STM32MP1 Distribution Package v1.2.0, but do not initialize the OpenEmbedded environment (do not source the envsetup.sh).

3.2. Install X-LINUX-AI environment for ST boards

• Clone the meta-st-stm32mpu-ai git repositories

Warning

The software package is provided AS IS, and by downloading it, you agree to be bound to the terms of the software license agreement (SLA). The detailed content licenses can be found here.

For X-LINUX-AI v2.1.0:

 cd <Distribution Package installation directory>/layers/meta-st
 git clone https://github.com/STMicroelectronics/meta-st-stm32mpu-ai.git -b v2.1.0

For X-LINUX-AI v2.0.0:
 cd <Distribution Package installation directory>/layers/meta-st
 git clone https://github.com/STMicroelectronics/meta-st-stm32mpu-ai.git -b v2.0.0

• Set up the build environment

 cd ../..
 DISTRO=openstlinux-weston MACHINE=stm32mp1 BSP_DEPENDENCY='layers/meta-st/meta-st-stm32mpu-ai' source layers/meta-st/scripts/envsetup.sh

3.3. Install X-LINUX-AI environment for Avenger96 board

• Clone the meta-av96 and meta-st-stm32mpu-ai git repositories

Warning

The software package is provided AS IS, and by downloading it, you agree to be bound to the terms of the software license agreement (SLA). The detailed content licenses can be found here.

For X-LINUX-AI v2.1.0:

 cd <Distribution Package installation directory>/layers
 git clone https://github.com/dh-electronics/meta-av96.git -b av96_v65
 cd <Distribution Package installation directory>/layers/meta-st
 git clone https://github.com/STMicroelectronics/meta-st-stm32mpu-ai.git -b v2.1.0

For X-LINUX-AI v2.0.0:
 git clone https://github.com/dh-electronics/meta-av96.git -b av96_v62
 cd <Distribution Package installation directory>/layers/meta-st
 git clone https://github.com/STMicroelectronics/meta-st-stm32mpu-ai.git -b v2.0.0

• Set up the build environment

 cd ../..
 META_LAYER_ROOT=layers DISTRO=openstlinux-weston MACHINE=stm32mp1-av96 BSP_DEPENDENCY='layers/meta-st/meta-st-stm32mp-addons layers/meta-st/meta-st-stm32mpu-ai' source layers/meta-st/scripts/envsetup.sh

3.4. Build the image

 bitbake st-image-ai

Information

Note that building the image could take long time depending on the host computer performance.

3.5. Flash the built image

Follow this link to see how to flash the built image.

4. How to use the X-LINUX-AI Expansion Package

4.1. Material needed

To use the X-LINUX-AI OpenSTLinux Expansion Package, choose one of the following materials:

• STM32MP157C-DK2 + an UVC USB WebCam
• STM32MP157C-EV1 with the built in OV5640 parallel camera
• STM32MP157A-EV1 with the built in OV5640 parallel camera
• STM32MP157 Avenger96 board + an UVC USB WebCam or the OV5640 CSI Camera mezzanine board

Optional:

• Coral USB Edge TPU accelerator

4.2. Boot the OpenSTlinux Starter Package

At the end of the boot sequence, the demo launcher application appears on the screen.

4.3. Install the X-LINUX-AI

Warning

The software package is provided AS IS, and by downloading it, you agree to be bound to the terms of the software license agreement (SLA). The detailed content licenses can be found here.

After having configured the AI OpenSTLinux package you can install the X-LINUX-AI components.

 apt-get install packagegroup-x-linux-ai

And restart the demo launcher

 systemctl restart weston@root

4.4. Launch an AI application sample

Once the demo launcher is restarted, notice that it is slightly different because new AI application samples have been installed.
The demo laucher has the following appearance, and you can navigate into the different screens by using the NEXT or BACK buttons.

Screens 2, 3 and 4 contain AI application samples that are described within dedicated article available in the X-LINUX-AI application samples zoo page.

4.5. Enjoy running your own NN models

The above examples provide application samples to demonstrate how to execute models easily on the STM32MP1.

You are free to update the C/C++ application or Python scripts for your own purposes, using your own NN models.

Source code locations are provided in application sample pages.

Artificial Intelligence

Microprocessor Unit

Linux^® is a registered trademark of Linus Torvalds.

Computer Vision

Flash memory shortened to gain space in titles, tables and block diagrams

Secure digital

Arm^® is a registered trademark of Arm Limited (or its subsidiaries) in the US and/or elsewhere.

Automatic current limit (LCD power improvement solution)

Software development kit (A programming package that enables a programmer to develop applications for a specific platform.)

Application programming interface

Board support package

USB Video Class

Multi Speed Internal oscillator (STM32 clock source)

Neural Network

Artificial Intelligence

Microprocessor Unit

Linux^® is a registered trademark of Linus Torvalds.

Computer Vision

Flash memory shortened to gain space in titles, tables and block diagrams

Secure digital

Arm^® is a registered trademark of Arm Limited (or its subsidiaries) in the US and/or elsewhere.

Automatic current limit (LCD power improvement solution)

Software development kit (A programming package that enables a programmer to develop applications for a specific platform.)

Application programming interface

Board support package

USB Video Class

Multi Speed Internal oscillator (STM32 clock source)

Neural Network

Flash memories combine high density and cost effectiveness of EPROMs with the electrical erasability of EEPROMs. For this reason, the Flash memory market is one of the most exciting areas of the semiconductor industry today and new applications requiring in system reprogramming, such as cellular telephones, automotive engine management systems, hard disk drives, PC BIOS software for Plug & Play, digital TV, set top boxes, fax and other modems, PC cards and multimedia CD-ROMs, offer the prospect of very high volume demand.