hand_posture

Hand posture Getting Started Package

Table of Contents

1. Handposture application code

2. Directory contents

3. Before you start

3.1. Hardware and Software environment

3.2. Tools installations

4. Deployment

4.1. Generate C code from tflite file

4.2. Build and deploy

5. Getting started deep dive

5..1 Processing workflow

5.2. Model configuration

6. Limitations

---

1. Handposture application code

The purpose of this package is to enable handposture application on a STM32 board.

This project provides an STM32 microcontroller embedded real time environnement to execute X-CUBE-AI generated model targeting hand posture classification application.

2. Directory contents

This repository is structured as follows:

Directory	Content
Application\<STM32_Board_Name>\STM32CubeIDE	cubeIDE project files; only IDE files related
Application\<STM32_Board_Name>\Inc	Application include files
Application\<STM32_Board_Name>\Src	Application source files
Application\Network\*	Place holder for AI C-model; files generated by STM32Cube.AI
Drivers\CMSIS	CMSIS Drivers
Drivers\BSP	Board Support Package and Drivers
Drivers\STM32XXxx_HAL_Driver	Hardware Abstraction Layer for STM32XXxx family products
Middlewares\ST\STM32_AI_Runtime	Place holder for AI runtime library

3. Before you start

3.1 Hardware and Software environment

In order to run this hand posture classification application examples you need to have the following hardware:

• NUCLEO-F401RE Nucleo board
• X-NUCLEO-53LxA1 Nucleo expansion board

Only this hardware is supported for now

3.2 Tools installations

This getting started needs STM32CubeIDE as well as X-CUBE-AI

You can find the info to install the tools in the parents README of the deployment part and the general README of the model zoo.

4. Deployment

4.1. Generate C code from tflite file

This repo does not provide the AI C-model generated by X-CUBE-AI.

The user needs to generate the AI C-model.

It is directly generated by the deployment script of the model zoo.

4.2. Build and deploy

You should use the deploy.py script to automatically build and deploy the program on the target (if the hardware is connected). Refer to the README of the deployment part, chapter RUN DEPLOYMENT and RUN THE APPLICATION.

Once the deploy.py script has been launched, you can launch the Application\NUCLEO-F401RE\STM32CubeIDE\.project with STM32CubeIDE. With the IDE you can modify, build and deploy on the target.

5. Getting started deep dive

The purpose of this package is to enable hand posture classification application on a STM32 board.

5..1 Processing workflow

The software executes a hand posture classification on each frame captured by the TOF sensor. The framerate depends on the sensor configuration

The processing flow is the following one:

• The sensor frame is a 8 by 8 zones matrix with 2 data by zone:

  • ranging: distance from the sensor
  • peak: number of photons that returned to the sensor

• Network_Preprocess is composed of 2 steps:

  • ValidateFrame: check the subject is in the valid range of distance
  • NormalizeData: normalize the ranging and peak data

• Network_Inference: call AI C-model network

• Network_Postprocess is composed of 2 steps:

  • apply an argmax function to retain the predicted label
  • apply a filter to avoid output pat

5.2. Model configuration

The <getting-start-install-dir>/Application/NUCLEO-F401RE/Inc/ai_model_config.h file contains configuration information.

This file is generated by the deploy.py script.

The number of output class for the model:

#define NB_CLASSES          (8)

The dimension of the model input tensor:

#define INPUT_HEIGHT        (8)
#define INPUT_WIDTH         (8)
#define INPUT_CHANNELS      (2)

A table containing the list of the labels for the output classes:

#define CLASSES_TABLE const char* classes_table[NB_CLASSES] = {\
   "None" ,   "FlatHand" ,   "Like" ,   "Love" ,   "Dislike" ,   "BreakTime" ,\
   "CrossHands" ,   "Fist"};\

A table containing the list of the identifiers for each label to be interpreted by the EVK software:

#define EVK_LABEL_TABLE const int evk_label_table[NB_CLASSES] = {\
   0 ,   20 ,   21 ,   24 ,   25 ,   27 ,\
   28 ,   32};\

The limit used to remove the background and the minimum and maximum limits for the subject to be valid:

#define BACKGROUND_REMOVAL (120)
#define MAX_DISTANCE (350)
#define MIN_DISTANCE (150)

The rest of the model details will be embedded in the .c and .h files generated by the tool X-CUBE-AI.

6. Limitations

• Supports only the NUCLEO-F401RE board with X-NUCLEO-53LxA1 TOF expansion board
• Supports only neural network model whom size fits in SoC internal memory
• Supports only non-quantized neural network models
• Supports only neural network model with a 8x8x2 input shape
• Input layer of the model supports only data in FLOAT32 format
• Output layer of the model provides data in only FLOAT32 format
• Limited to STM32CubeIDE / arm gcc toolchain; IAR and Keil are coming
• Manageable through STM32CubeIDE (open, modification, debug)