NanoEdge AI Library for anomaly detection (AD)

Warning white.png Warning

Major update for NanoEdge AI v3.
Pay close attention to the new function names, new arguments and new return types.


1 What is NanoEdge AI Library for anomaly detection?

NanoEdge™ AI Library is an Artificial Intelligence (AI) static library originally developed by Cartesiam, for embedded C software running on Arm® Cortex® microcontrollers.

When embedded on microcontrollers, it gives them the ability to easily "learn" and "understand" sensor patterns, by themselves, without the need for the user to have additional skills in Mathematics, Machine Learning, or data science.

The NanoEdge AI static library for anomaly detection is the code that contains an AI model (for example, as a bundle of signal treatment, Machine Learning model, and optimally tuned hyperparameters). It is designed to gather knowledge incrementally during a learning phase to become able to detect potential anomalous machine behaviors, and possibly predict them.

Info white.png Information
  • In the free version of NanoEdgeAIStudio (the Studio), the NanoEdge AI Library is only compatible with a selection of STM32 evaluation boards. See the full list here in the Studio documentation.

2 Install / Getting started

The main functions available via the library are:

init() run first before learning/detecting, or to reset the knowledge of the library/emulator
learn() start a number of learning iterations (to establish an initial knowledge, or enrich an existing one)
detect() start a number of detection iterations (inference), once a minimum knowledge base has been established
set_sensitivity() adjust the preset, internal detection sensitivity (does not affect learning, only returned similarity scores)
get_sensitivity() returns the sensitivity of the model
Warning white.png Important

When building a smart device, the final features heavily depend on the way those functions are called. It is entirely up to the developer to design relevant learning and detection strategies, depending on the project specificities and constraints.

NanoEdgeAI ild.png

For example for a hypothetical machine, one possible strategy is to:

  1. initialize the model
  2. establish an initial knowledge base by calling learn() every minute for 24 hours on that machine
  3. switch to the inference mode by calling detect() 10 times every hour (and averaging the returned scores), each day
  4. blink a LED and ring alarms whenever detect() returns any anomaly (average score < 90%)
  5. run another learning cycle to enrich the existing knowledge, if the temperature rises above 60°C (and the machine is still OK)
  6. send a daily report (average number of anomalies per hour, with date, time, and machine ID for instance) using Bluetooth® or LoRa®.

In summary, those smart functions can be triggered by external data (for example from sensors, buttons, to account for and adapt to environment changes).
The scores returned by the smart functions can trigger all kinds of behaviors on your device.
The possibilities are endless.

2.1 Static library

  • In NanoEdge AI Studio, after obtaining a library, click Compile (on the "Deployment" screen, which follows the "Benchmark" and "Emulator" screens)
  • Open the .zip obtained
  • Select and copy the static library libneai.a
  • Link this static library to your project code

2.2 NanoEdge AI Library functions

Most NanoEdge AI function return the status of the library in the following enum, neai_state:

enum neai_state {
	NEAI_OK = 0,
	NEAI_INIT_FCT_NOT_CALLED = 123,
	NEAI_BOARD_ERROR, 
	NEAI_KNOWLEDGE_BUFFER_ERROR,
	NEAI_NOT_ENOUGH_CALL_TO_LEARNING,
	NEAI_MINIMAL_RECOMMENDED_LEARNING_DONE,
	NEAI_UNKNOWN_ERROR
};
#endif

Here are the possible statuses:

NEAI_OK: the library is working as expected
NEAI_INIT_FCT_NOT_CALLED: the learn or detect function has been called without running the init function before. Initialize your library.
NEAI_BOARD_ERROR: the board detected is not authorized. For instance, it may happen if you are trying to use a limited library (for instance obtained from the free version of NanoEdge AI Studio) with a non-supported STM32 board.
NEAI_KNOWLEDGE_BUFFER_ERROR: the knowledge loaded is not compatible with this library. Make sure that the knowledge being used is the one obtained with this exact library.
NEAI_NOT_ENOUGH_CALL_TO_LEARNING: this is a fail-safe to prevent users from running an insufficient (only one or a few) number of iterations of the learning function. Run more learning iterations.
NEAI_MINIMAL_RECOMMENDED_LEARNING_DONE: Minimal amount of recommended learning have been done.
NEAI_UNKNOWN_ERROR: there is an unknown error with the library.

2.2.1 Initialization

enum neai_state neai_anomalydetection_init(void);

Initialization can be run at the beginning to initialize the model and/or later to initialize a new model and reset all knowledge.
Returns the neai_state enum (NEAI_OK == 0, in case of success).

Info white.png Information

If you are using a limited library (for instance obtained using the free version of NanoEdge AI Studio), make sure that your board is a compatible STM32 development board.

2.2.2 Include knowledge

The main advantage of anomaly detection libraries is that they can be re-trained directly on the edge. By default, if you use the library on a microcontroller, the model must be retrained to better fit the data in its real environment. But it is also possible to add the knowledge acquired during the benchmark (corresponding to the training data of the project). To do so, after the init() function, call this function to add the knowledge to the model:

enum neai_state neai_anomalydetection_knowledge(const float knowledge_buffer[]);
  • Input:
const float knowledge_buffer[], this buffer is defined in the header file knowledge.h provided in the .zip file containing the static NanoEdge AI Library.
  • Output:
the neai_state enum (NEAI_OK == 0, in case of success).

2.2.3 Learning

enum neai_state neai_anomalydetection_learn(float data_input[]);

This function is used to learn patterns in your data. It can be used at any time, in the beginning to build the original knowledge base of the AI model, but also later, as an additional learning phase to complement the existing knowledge.

  • Input:
float data_input[], the length of the data is BUFFER_SIZE * NB_AXES .
  • Output:
the neai_state enum (NEAI_MINIMAL_RECOMMENDED_LEARNING_DONE or NEAI_NOT_ENOUGH_CALL_TO_LEARNING).
Info white.png Information

The learning function can be called:

  1. initially, before any inference, to establish some reference knowledge base
  2. subsequently, whenever needed, to complete the existing knowledge and enrich it (for example, to take into account some new nominal environment conditions)
Warning white.png Warning

NanoEdge AI Library uses float data types instead of int. If you are using int data types, convert (cast) them into float.

2.2.4 Detection

enum neai_state neai_anomalydetection_detect(float data_input[], uint8_t *similarity);

This function returns returns a similarity percentage, measure of the mathematical distance between the incoming signal and the existing knowledge, learned by the library.

  • Input:
float data_input[], the length of the data is BUFFER_SIZE * NB_AXES.
uint8_t *similarity, the variable that contains the similarity score returned by the function.
  • Output:
The percentage of similarity [0-100] between the new signal and learned patterns ("100" means completely similar, and "0" completely dissimilar).
The neai_state enum.
Info white.png Information
  • The uint8_t *similarity variable must be defined prior to calling the detection function, and pointed to using &similarity when passed as an argument (see code example below).
  • The recommended threshold percentage is 90. Values under this threshold reveal a behavior that differs from the usual behavior learned by the AI model. This threshold can be defined by the user, depending on the final application sensitivity requirements.

2.2.5 Setting sensitivity

enum neai_state neai_anomalydetection_set_sensitivity(float sensitivity);

This function sets the sensitivity of the model in detection mode. It can be tuned at any time without having to go through a new learning phase. This sensitivity has no influence on the knowledge acquired during the learning steps. It only plays a role in the detection step, by influencing the similarity percentages that are returned by the detect function (acts as a linear scaler of the preset internal sensitivity).

  • Input: float sensitivity
  • Output: the neai_state enum
Info white.png Information
  • The default sensitivity value is 1. A sensitivity value between 0 and 1 (excluded) decreases the sensitivity of the model, while a value in between 1 and 100 increases it. STMicroelectronics recommend increasing or decreasing sensitivity by steps of 0.1.
  • Sensitivity 1.1 - 100 tends to decrease the percentages of similarity returned (the algorithm is more sensitive to perturbations), while sensitivity 0 - 0.9 tends to increase them (the algorithm is less sensitive to perturbations).

2.2.6 Getting sensitivity

float neai_anomalydetection_get_sensitivity(void);

This function returns the current sensitivity of the model.
The recommended/default sensitivity is 1. However, this value can be defined by the user depending on his application sensitivity requirements.

  • Input: None
  • Output: The sensitivity of model.

2.3 Backing up and restoring the library knowledge

2.3.1 Creating backups

When using NanoEdge AI Library, knowledge is created on the go. It means that after each learning iteration, the Machine Learning model is incrementally getting richer and richer.

For performance reasons, this knowledge is saved into the microcontroller RAM. Since RAM is volatile, better copy this knowledge into non-volatile memory and prevent its loss (for example, in case of power failure).

NanoEdge AI Library attributes a specific memory section called .neai for the knowledge variables (model hyperparameters). To use it, you need to create a memory section in your linker script according to your microcontroller architecture.

Here is a general outline of the procedure:

  1. The idea is to create a section called .neai (for NanoEdge AI) in your RAM, using the linker script that is specific to the board / MCU that you use (it is a .ld file).

    For example, using Arm® Mbed™ for a NUCLEO-L432KC (Cortex®-M4), modify the file STM32L432XX.ld and add the following RAM section:
    .neai 0x0000000020004000  :                                                                            
    {                                                                                                      
       KEEP(*(.neai))          /* keep my variable even if not referenced */
    } >RAM
    

    Make sure that the RAM address (in this example, 0x0000000020004000) corresponds to a valid section according to the board and MCU technical documentation. Here it corresponds to the beginning of the RAM0 (Data Space) section, see below:

    NanoEdgeAI memory size.png
  2. Then, create two new functions in your main code (containing the NanoEdge AI functions and to be compiled and programmed to the MCU): one function to save the knowledge from the RAM to the Flash memory (dump), and another one to restore the knowledge from the Flash memory to the RAM (load).

    How to create these two functions, which write to the RAM or Flash memory, highly depends on the board and MCU used (see their technical documentation).
  3. For the dump function (RAM => Flash memory), indicate the RAM address to start reading from (the .neai section address, such as 0x0000000020004000 in the example above), the Flash memory address to start writing to, and the total size of the knowledge saved in the NanoEdge AI Library (libneai.a).

    You can obtain the size of this knowledge by means of the arm-none-eabi-size tool:
    $ arm-none-eabi-size -A libneai.a | grep neai
    
  4. For the load function (Flash memory => RAM), it is the reverse operation. You need the Flash memory address to start reading from, the RAM address to start writing to (.neai section), and the size of the knowledge to restore.

2.3.2 Restoring backups

To restore a previously saved backup, copy the entire knowledge from the Flash memory back into the RAM .neai section, as explained in step #4 of the section "2.3.1 Creating backups" above.

Then, you are able to use your NanoEdge AI Library functions again.

2.4 Example "Hello World!"

Header file: NanoEdgeAI.h

Example of NanoEdge AI Library header file:

This snippet is provided AS IS, and by taking it, you agree to be bound to the license terms that can be found here for the component: Application.
/* =============
Copyright (c) 2021, STMicroelectronics

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that
the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this list of conditions and the
  following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the
  following disclaimer in the documentation and/or other materials provided with the distribution.

* Neither the name of the copyright holders nor the names of its contributors may be used to endorse or promote
  products derived from this software without specific prior written permission.

*THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER / OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.*
*/

#ifndef __NANOEDGEAI_H__
#define __NANOEDGEAI_H__

/* Includes */
#include <stdint.h>

/* Define */

#define AXIS_NUMBER 3
#define DATA_INPUT_USER 256

#ifndef __NEAI_STATE__
#define __NEAI_STATE__
enum neai_state { 
    NEAI_OK = 0,
    NEAI_INIT_FCT_NOT_CALLED = 123,
    NEAI_BOARD_ERROR,
    NEAI_KNOWLEDGE_BUFFER_ERROR,
    NEAI_NOT_ENOUGH_CALL_TO_LEARNING, //This is a fail-safe to prevent users from learning one or even no signals.
    NEAI_UNKNOWN_ERROR};
#endif

/* Function prototypes */
#ifdef __cplusplus
extern "C" {
#endif
	enum neai_state neai_anomalydetection_init(void);
	enum neai_state neai_anomalydetection_learn(float data_input[]);
	enum neai_state neai_anomalydetection_detect(float data_input[], uint8_t *similarity);
	enum neai_state neai_anomalydetection_set_sensitivity(float sensitivity);
	float neai_anomalydetection_get_sensitivity(void);
#ifdef __cplusplus
}
#endif

#endif

/* =============
Here some sample declaration added in your main program for the use of the NanoEdge AI library.
You can directly copy these declarations or modify the names.
* WARNING: respect the size of the buffer.

uint8_t similarity = 0; // Point to similarity (see argument of neai_anomalydetection_detect fct)
float input_user_buffer[DATA_INPUT_USER * AXIS_NUMBER]; // Buffer of input values
*/

Main program: main.c
This program must be completed by the user (depending for instance on the applications or the desired features).

This snippet is provided AS IS, and by taking it, you agree to be bound to the license terms that can be found here for the component: Application.
/* =============
Copyright (c) 2020, STMicroelectronics

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that
the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this list of conditions and the
  following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the
  following disclaimer in the documentation and/or other materials provided with the distribution.

* Neither the name of the copyright holders nor the names of its contributors may be used to endorse or promote
  products derived from this software without specific prior written permission.

*THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER / OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.*
*/
/**
  **************************************************************************
  * Demo: NanoEdge AI process to include in main program body
  *
  * @note  This program must be completed and customized by the user
  **************************************************************************
  */

/* Includes --------------------------------------------------------------------*/
#include "NanoEdgeAI.h"
/* Number of samples for learning: set by user ---------------------------------*/
#define LEARNING_ITERATIONS 256
float input_user_buffer[DATA_INPUT_USER * AXIS_NUMBER]; // Buffer of input values

/* Private function prototypes defined by user ---------------------------------*/
/*
 * @brief Collect data process
 *
 * This function is defined by user, depends on applications and sensors
 *
 * @param sample_buffer: [in, out] buffer of sample values
 * @retval None
 * @note   If AXIS_NUMBER = 3 (cf NanoEdgeAI.h), the buffer must be
 *         ordered as follow:
 *         [x0 y0 z0 x1 y1 z1 ... xn yn zn], where xi, yi and zi
 *         are the values for x, y and z axes, n is equal to
 *         DATA_INPUT_USER (cf NanoEdgeAI.h)
 */
void fill_buffer(float input_buffer[])
{
	/* USER BEGIN */
	/* USER END */
}

/* -----------------------------------------------------------------------------*/
int main(void)
{
	/* Initialization ------------------------------------------------------------*/
	enum neai_state error_code = neai_anomalydetection_init();
	uint8_t similarity = 0;

	if (error_code != NEAI_OK) {
		/* This happens if the library works into a not supported board. */
	}

	/* Learning process ----------------------------------------------------------*/
	for (uint16_t iteration = 0 ; iteration < LEARNING_ITERATIONS ; iteration++) {
		fill_buffer(input_user_buffer);
		neai_anomalydetection_learn(input_user_buffer);
	}
	
	/* Detection process ---------------------------------------------------------*/
	while (1) {
		fill_buffer(input_user_buffer);
		neai_anomalydetection_detect(input_user_buffer, &similarity);
		/* USER BEGIN */
		/*
		 * e.g.: Trigger functions depending on similarity
		 * (blink LED, ring alarm, etc.).
		 */
		/* USER END */
	}
}

3 Resources

Documentation
All NanoEdge AI Studio documentation is available here.

Tutorials
Step-by-step tutorials to use NanoEdge AI Studio to build a smart device from A to Z.