The goal of MLPerf™ Tiny is to provide a representative set of deep neural nets and benchmarking code to compare performance between embedded devices. These devices typically run at between 10MHz and 250MHz, and can perform inference using less then 50mW of power.
For this Benchmark we are suggesting 3 Boards:
In this guide, we will learn how to generate the projects for the previous boards.
You need to download and install the following software:
- STM32CubeMX ( v6.10.0 )
- STM32CubeIDE ( v1.14.1 )
- IAR EW for ARM ( v9.50.1 )
- STM32CubeProg ( v2.15.0 )
- STM32Cube.AI ( v9.0.0 )
Also we need to download the following pretrained quantized model from MLCommons™ Github:
In this section we will explain how to generate the different benchmark projects for the NUCLEO-H7A3ZI-Q board. We will take the Image Classification benchmark project as an example and then exactly the same steps should be repeated for the rest of the scenarios (Anomaly Detection, Keyword Spotting and Person Detection)
1. Download the Image Classification model and then save it under
...\stm32ai-perf\STM32_H7A3ZI\image_classificationYou should have something like this to start:
2. Open the .ioc file and follow the next steps to generate your project template:
After generating the project using STM32CubeMX you should have something like this in your workspace:
1. Open the .cproject file and follow the next steps to configure your project:
2. Modify the main.c file:
2.1 Open main.c located under Project Explorer:
2.2 Under Private includes add the following:
/* Private includes ----------------------------------------------------------*/
/* USER CODE BEGIN Includes */
#include "submitter_implemented.h"
/* USER CODE END Includes */2.3 Add the following line to USER CODE Section 2:
/* USER CODE BEGIN 2 */
ee_benchmark_initialize();
/* USER CODE END 2 */2. Modify the stm32h7xx_it.c file:
2.1 Open stm32h7xx_it.c located under Project Explorer:
2.2 Delete any initial code in the file located after the following lines:
/******************************************************************************/
/* STM32H7xx Peripheral Interrupt Handlers */
/* Add here the Interrupt Handlers for the used peripherals. */
/* For the available peripheral interrupt handler names, */
/* please refer to the startup file (startup_stm32h7xx.s). */
/******************************************************************************/2.3 Configure the TCM memory on the STM32H7A3 :
Under the project, open the file called STM32H7A3ZITXQ_FLASH.ld.
Under this file :
- Specify the memory area :
/* Specify the memory areas */
MEMORY
{
ITCMRAM (xrw) : ORIGIN = 0x00000000, LENGTH = 64K
FLASH (rx) : ORIGIN = 0x08000000, LENGTH = 2048K
DTCMRAM1 (xrw) : ORIGIN = 0x20000000, LENGTH = 128K
/*DTCMRAM2 (xrw) : ORIGIN = 0x20010000, LENGTH = 64K*/
RAM (xrw) : ORIGIN = 0x24000000, LENGTH = 1024K
RAM_CD (xrw) : ORIGIN = 0x30000000, LENGTH = 128K
RAM_SRD (xrw) : ORIGIN = 0x38000000, LENGTH = 32K
}
- Set data sections, into DTCMRAM1 :
/* Uninitialized data section */
. = ALIGN(4);
.bss :
{
/* This is used by the startup in order to initialize the .bss section */
_sbss = .; /* define a global symbol at bss start */
__bss_start__ = _sbss;
*(.bss)
*(.bss*)
*(COMMON)
. = ALIGN(4);
_ebss = .; /* define a global symbol at bss end */
__bss_end__ = _ebss;
} >DTCMRAM1
3. Build the project in Release mode:
Set the project to the release mode as the following and then click on build :
Next we will demonstarte how to flash your NUCLEO-H7A3ZI-Q board with the binary file you generated in the previous step
1. Verify the board configuration:
Make sure that your board has the the following jumpers fitted:
2. Flash the board:
Connect the board to the laptop using a usb cable and use STM32CubeProg to flash the board as the following:
For energy mode the EE_CFG_ENERGY_MODE flag should be set to 1.
You need to make sure that the MCU_RST and the MCU_VDD jumpers are not fitted.
The final setup should look like the following:
Make sure to push the Reset Button (the Black Button) on the board before starting the benchmark software.
For more details about the energy benchmark and EEMBCs EnergyRunner™ benchmark framework please refer to this link
For Performance mode the EE_CFG_ENERGY_MODE flag should be set to 0.
The board should be connected to the laptop using a usb cable and you need to make sure that the MCU_RST and the MCU_VDD jumpers are fitted.
Make sure to push the Reset Button (the Black Button) on the board before starting the benchmark software.
For more details about the Performance mode and EEMBCs EnergyRunner™ benchmark framework please refer to this link
In this section we will explain how to generate the different benchmark projects for the NUCLEO-U575ZI-Q board. We will take the Anomaly Detection benchmark project as an example and then exactly the same steps should be repeated for the rest of the scenarios (Image Classification, Keyword Spotting and Person Detection)
1. Download the Anomaly Detection model and then save it under
...\stm32ai-perf\STM32_U575ZI\anomaly_detectionYou should have something like this to start:
2. Open the .ioc file and follow the next steps to generate your project template:
After generating the project using STM32CubeMX you should have something like this in your workspace:
1. Open the .cproject file and follow the next steps to configure your project:
2. Modify the main.c file:
2.1 Open main.c located under Project Explorer:
2.2 Under Private includes add the following:
/* Private includes ----------------------------------------------------------*/
/* USER CODE BEGIN Includes */
#include "submitter_implemented.h"
/* USER CODE END Includes */2.3 Add following code in USER CODE Section 2:
/* USER CODE BEGIN 2 */
ee_benchmark_initialize();
/* USER CODE END 2 */2. Modify the stm32u5xx_it.c file:
2.1 Open stm32u5xx_it.c located under Project Explorer:
2.2 Delete any initial code in the file located after the following lines:
/******************************************************************************/
/* STM32U5xx Peripheral Interrupt Handlers */
/* Add here the Interrupt Handlers for the used peripherals. */
/* For the available peripheral interrupt handler names, */
/* please refer to the startup file (startup_stm32u5xx.s). */
/******************************************************************************/3. Build the project in Release mode:
Set the project to the release mode as the following and then click on build :
Next we will demonstarte how to flash your NUCLEO-U575ZI-Q board with the binary file you generated in the previous step
1. Verify the board configuration:
Make sure that your board has the the following jumpers fitted:
2. Flash the board:
Connect the board to the laptop using a usb cable and use STM32CubeProg to flash the board as the following:
For energy mode the EE_CFG_ENERGY_MODE flag should be set to 1.
You need to make sure that the T_NRST and the IDD jumpers are not fitted.
The final setup should look like the following:
Make sure to push the Reset Button (the Black Button) on the board before starting the benchmark software.
For more details about the energy benchmark and EEMBCs EnergyRunner™ benchmark framework please refer to this link
For Performance mode the EE_CFG_ENERGY_MODE flag should be set to 0.
The board should be connected to the laptop using a usb cable and you need to make sure that the T_NRST and the IDD jumpers are fitted.
Make sure to push the Reset Button (the Black Button) on the board before starting the benchmark software.
For more details about the Performance mode and EEMBCs EnergyRunner™ benchmark framework please refer to this link
In this section we will explain how to generate the different benchmark projects for the NUCLEO-L4R5ZI board. We will take the Person Detection benchmark project as an example and then exactly the same steps should be repeated for the rest of the scenarios (Anomaly Detection, Keyword Spotting and Image Classification)
1. Download the Person Detection model and then save it under
...\stm32ai-perf\STM32_L4R5ZI\person_detectionYou should have something like this to start:
2. Open the .ioc file and follow the next steps to generate your project template:
After generating the project using STM32CubeMX you should have something like this in your workspace:
1. Open the .cproject file and follow the next steps to configure your project:
2. Modify the main.c file:
2.1 Open main.c located under Project Explorer:
2.2 Under Private includes add the following:
/* Private includes ----------------------------------------------------------*/
/* USER CODE BEGIN Includes */
#include "submitter_implemented.h"
/* USER CODE END Includes */2.3 Add the following line to USER CODE Section 2:
/* USER CODE BEGIN 2 */
ee_benchmark_initialize();
/* USER CODE END 2 */2. Modify the stm32l4xx_it.c file:
2.1 Open stm32l4xx_it.c located under Project Explorer:
2.2 Delete any initial code in the file located after the following lines:
/******************************************************************************/
/* STM32L4xx Peripheral Interrupt Handlers */
/* Add here the Interrupt Handlers for the used peripherals. */
/* For the available peripheral interrupt handler names, */
/* please refer to the startup file (startup_stm32l4xx.s). */
/******************************************************************************/3. Build the project in Release mode:
Set the project to the release mode as the following and then click on build :
Next we will demonstarte how to flash your NUCLEO-L4R5ZI board with the binary file you generated in the previous step
1. Verify the board configuration:
Make sure that your board has the the following jumpers fitted:
2. Flash the board:
Connect the board to the laptop using a usb cable and use STM32CubeProg to flash the board as the following and before flashing your board make sure that your board is in single bank, for this you need to go under option bytes then user configuration and let DBANK bit unchecked.
For energy mode the EE_CFG_ENERGY_MODE flag should be set to 1.
You need to make sure that the MCU_RST and the IDD jumpers are not fitted.
The final setup should look like the following:
Make sure to push the Reset Button (the Black Button) on the board before starting the benchmark software.
For more details about the energy benchmark and EEMBCs EnergyRunner™ benchmark framework please refer to this link
For Performance mode the EE_CFG_ENERGY_MODE flag should be set to 0.
The board should be connected to the laptop using a usb cable and you need to make sure that the MCU_RST and the IDD jumpers are fitted.
Make sure to push the Reset Button (the Black Button) on the board before starting the benchmark Software.
For more details about the Performance mode and EEMBCs EnergyRunner™ benchmark framework please refer to this link