# Performing Inference on MCU or Small Systems
[](https://gitee.com/mindspore/docs/blob/r1.10/docs/lite/docs/source_en/use/micro.md)
## Overview
This tutorial describes an ultra-lightweight AI deployment solution for IoT edge devices.
Compared with mobile devices, MCUs are usually used on IoT devices. The ROM resources of the device are limited, and the memory and computing power of the hardware resources are weak.
Therefore, AI applications on IoT devices have strict limits on runtime memory and power consumption of AI model inference.
For MCUs deploying hardware backends, MindSpore Lite provides the ultra-lightweight Micro AI deployment solution. In the offline phase, models are directly generated into lightweight code without online model parsing and graph compilation. The generated Micro inference code is easy to understand, with less memory at runtime and smaller code size.
You can use a MindSpore Lite conversion tool `converter_lite` to easily generate inference code that can be deployed on the x86/ARM64/ARM32/Cortex-M platform.
Deploying a model for inference via the Micro involves the following four steps: model inference code generation, `Micro` lib obtaining, code integration, and compilation and deployment.
## Generating Model Inference Code
### Overview
The Micro configuration item in the parameter configuration file is configured via the MindSpore Lite conversion tool `convert_lite`.
This chapter describes the functions related to code generation in the conversion tool. For details about how to use the conversion tool, see [Converting Models for Inference](https://www.mindspore.cn/lite/docs/en/r1.10/use/converter_tool.html).
### Preparing Environment
The following describes how to prepare the environment for using the conversion tool in the Linux environment.
1. System environment required for running the conversion tool
In this example, the Linux operating system is used. Ubuntu 18.04.02LTS is recommended.
2. Obtain the conversion tool
You can obtain the conversion tool in either of the following ways:
- Download [Release Version](https://www.mindspore.cn/lite/docs/en/r1.10/use/downloads.html) from the MindSpore official website.
Download the release package whose OS is Linux-x86_64 and hardware platform is CPU.
- Start from the source code for [Building MindSpore Lite](https://www.mindspore.cn/lite/docs/en/r1.10/use/build.html).
3. Decompress the downloaded package.
```bash
tar -zxf mindspore-lite-${version}-linux-x64.tar.gz
```
${version} is the version number of the release package.
4. Add the dynamic link library required by the conversion tool to the environment variable LD_LIBRARY_PATH.
```bash
export LD_LIBRARY_PATH=${PACKAGE_ROOT_PATH}/tools/converter/lib:${LD_LIBRARY_PATH}
```
${PACKAGE_ROOT_PATH} is the path of the decompressed folder.
### Generating Inference Code by Running converter_lite
1. Go to the conversion directory
```bash
cd ${PACKAGE_ROOT_PATH}/tools/converter/converter
```
2. Set the Micro configuration item
Create the micro.cfg file in the current directory. The file content is as follows:
```text
[micro_param]
# enable code-generation for MCU HW
enable_micro=true
# specify HW target, support x86,Cortex-M, AMR32A, ARM64 only.
target=x86
# enable parallel inference or not.
support_parallel=false
```
In the configuration file, `[micro_param]` in the first line indicates that the subsequent variable parameters belong to the micro configuration item `micro_param`. These parameters are used to control code generation. Table 1 describes the parameters.
In this example, we will generate inference code for Linux systems with the underlying architecture x86_64, so set `target=x86` to declare that the generated inference code will be used for Linux systems with the underlying architecture x86_64.
3. Prepare the model to generate inference code
Click here to download the [MNIST Handwritten Digit Recognition Model](https://download.mindspore.cn/model_zoo/official/lite/quick_start/micro/mnist.tar.gz) used in this example.
After downloading, decompress the package to obtain `mnist.tflite`. This model is a trained MNIST classification model, that is, a TFLITE model. Copy the `mnist.tflite` model to the current conversion tool directory.
4. Execute converter_lite and generate code
```bash
./converter_lite --fmk=TFLITE --modelFile=mnist.tflite --outputFile=mnist --configFile=micro.cfg
```
The following information is displayed when the code is run successfully:
```text
CONVERTER RESULT SUCCESS:0
```
For details about the parameters related to converter_lite, see [Converter Parameter Description](https://www.mindspore.cn/lite/docs/en/r1.10/use/converter_tool.html#parameter-description).
After the conversion tool is successfully executed, the generated code is saved in the specified `outputFile` directory. In this example, the mnist folder is in the current conversion directory. The content is as follows:
```text
mnist # Specified name of generated code root directory
├── benchmark # Benchmark routines for integrated calls to model inference code
│ ├── benchmark.c
│ ├── calib_output.c
│ ├── calib_output.h
│ ├── load_input.c
│ └── load_input.h
├── CMakeLists.txt # cmake project file of the benchmark routine
└── src # Model inference code directory
├── CMakeLists.txt
├── net.bin # Model weights in binary form
├── net.c
├── net.cmake
├── net.h
├── model.c
├── context.c
├── context.h
├── tensor.c
├── tensor.h
├── weight.c
└── weight.h
```
You can integrate code to implement the application by referring to the interface calling of the inference code in the benchmark routine. For details about the interface, see [API Documentation](https://www.mindspore.cn/lite/api/en/r1.10/index.html).
Table 1: micro_param parameter definition
| parameter | Mandatory or not | Parameter Description | Range | Default value |
| --------------- | -------- | ------------------------------| --------------------------| --------- |
| enable_micro | Yes | The model generates code, otherwise it generates .ms. | true, false | false |
| target | Yes | Platform for which code is generated | x86, Cortex-M, ARM32, ARM64 | x86 |
| support_parallel | No | Generate multi-threaded inference codes or not, which can be set to true only on x86/ARM32/ARM64 platforms | true, false | false |
## Obtaining `Micro` Lib
After generating model inference code, you need to obtain the `Micro` lib on which the generated inference code depends before performing integrated development on the code.
The inference code of different platforms depends on the `Micro` lib of the corresponding platform. You need to specify the platform via the micro configuration item `target` based on the platform in use when generating code, and obtain the `Micro` lib of the platform when obtaining the inference package.
You can download the [Release Version](https://www.mindspore.cn/lite/docs/en/r1.10/use/downloads.html) of the corresponding platform from the MindSpore official website.
In chapter [Generating Model Inference Code](#generating-model-inference-code), we obtain the model inference code of the Linux platform with the x86_64 architecture. The `Micro` lib on which the code depends is the release package used by the conversion tool.
In the release package, the following content depended by the inference code:
```text
mindspore-lite-{version}-linux-x64
├── runtime
│ └── include
│ └── c_api # C API header file integrated with MindSpore Lite
└── tools
└── codegen # The source code generated by code depends on include and lib
├── include # Inference framework header file
│ ├── nnacl # nnacl operator header file
│ └── wrapper # wrapper operator header file
├── lib
│ ├── libwrapper.a # The MindSpore Lite codegen generates some operator static libraries on which the code depends
│ └── libnnacl.a # The MindSpore Lite codegen generates the nnacl operator static library on which the code depends
└── third_party
├── include
│ └── CMSIS # ARM CMSIS NN operator header file
└── lib
└── libcmsis_nn.a # ARM CMSIS NN operator static library
```
## Code Integration and Compilation Deployment
Different platforms have differences in code integration and compilation deployment.
- For the MCU of the cortex-M architecture, see [Performing Inference on the MCU](#performing-inference-on-the-mcu)
- For the Linux platform with the x86_64 architecture, see [Compilation and Deployment on Linux_x86_64 Platform](https://gitee.com/mindspore/mindspore/tree/r1.10/mindspore/lite/examples/quick_start_micro/mnist_x86)
- For details about how to compile and deploy arm32 or arm64 on the Android platform, see [Compilation and Deployment on Android Platform](https://gitee.com/mindspore/mindspore/tree/r1.10/mindspore/lite/examples/quick_start_micro/mobilenetv2_arm64)
- For compilation and deployment on the OpenHarmony platform, see [Executing Inference on Light Harmony Devices](#executing-inference-on-light-harmony-devices)
## Performing Inference on the MCU
### Overview
This tutorial takes the deployment of the [MNIST model](https://download.mindspore.cn/model_zoo/official/lite/quick_start/micro/mnist.tar.gz) on the STM32F767 chip as an example to demonstrate how to deploy the inference model on the MCU of the Cortex-M architecture, including the following steps:
- Use the converter_lite conversion tool to generate model inference code that adapts to the Cortex-M architecture
- Download the `Micro` lib corresponding to the Cortex-M architecture
- Integrate and compile the obtained inference code and `Micro` lib, and deploy verification
On the Windows platform, we demonstrate how to develop inference code through IAR, while on the Linux platform, we demonstrate how to develop inference code through MakeFile cross-compilation.
### Generating MCU Inference Code
Generate inference code for the MCU. For details, see the chapter [Generating Model Inference Code](#generating-model-inference-code). You only need to change `target=x86` in the Micro configuration item to `target=Cortex-M` to generate inference code for the MCU.
After the code is generated, the contents of the folder are as follows:
```text
mnist # Specified name of generated code root directory
├── benchmark # Benchmark routines for integrated calls to model inference code
│ ├── benchmark.c
│ ├── calib_output.c
│ ├── calib_output.h
│ ├── data.c
│ ├── data.h
│ ├── load_input.c
│ └── load_input.h
├── build.sh # One-click compilation script
├── CMakeLists.txt # cmake project file of the benchmark routine
├── cortex-m7.toolchain.cmake # Cross-compilation cmake file for cortex-m7
└── src # Model inference code directory
├── CMakeLists.txt
├── context.c
├── context.h
├── model.c
├── net.c
├── net.cmake
├── net.h
├── tensor.c
├── tensor.h
├── weight.c
└── weight.h
```
### Downloading `Micro` Lib of Cortex-M Architecture
The STM32F767 uses the Cortex-M7 architecture. You can obtain the `Micro` lib of the architecture in either of the following ways:
- Download [Release Version](https://www.mindspore.cn/lite/docs/en/r1.10/use/downloads.html) from the MindSpore official website.
You need to download the release package whose OS is None and hardware platform is Cortex-M7.
- Start from the source code for [Building MindSpore Lite](https://www.mindspore.cn/lite/docs/en/r1.10/use/build.html).
You can run the `MSLITE_MICRO_PLATFORM=cortex-m7 bash build.sh -I x86_64` command to compile the Cortex-M7 release package.
For other Cortex-M architecture platforms that do not provide release packages for download, you can modify MindSpore source code and manually compile the code to obtain the release package by referring to the method of compiling and building from source code.
### Code Integration and Compilation Deployment on Windows: Integrated Development Through IAR
This example shows code integration and burning through IAR and demonstrates how to develop the generated inference code in Windows. The main steps are as follows:
- Download the required software and prepare the integrated environment.
- Generate the required MCU startup code and demonstration project by using the `STM32CubeMX` software.
- Integrate model inference code and `Micro` lib within `IAR`.
- Perform complilation and simulation run
#### Environment Preparation
- [STM32CubeMX Windows Version](https://www.st.com/en/development-tools/stm32cubemx.html) >= 6.0.1
- `STM32CubeMX` is a graphical configuration tool for STM32 chips provided by `STM`. This tool is used to generate the startup code and project of STM chips.
- [IAR EWARM](https://www.iar.com/ewarm) >= 9.1
- `IAR EWARM` is an integrated development environment developed by IAR Systems for ARM microprocessors.
#### Obtaining the MCU Startup Code and Project
If you have an MCU project, skip this chapter.
This chapter uses the STM32F767 startup project as an example to describe how to generate an MCU project for an STM32 chip via `STM32CubeMX`.
- Start `STM32CubeMX` and select `New Project` from `File` to create a project.
- In the `MCU/MPU Selector` window, search for and select `STM32F767IGT6`, and click `Start Project` to create a project for the chip
- On the `Project Manager` page, configure the project name and the path of the generated project, and select `EWARM` in `Toolchain / IDE` to generate the IAR project
- Click `GENERATE CODE` above to generate code
- On the PC where the `IAR` is installed, double-click `Project.eww` in the `EWARM` directory of the generated project to open the IAR project.
#### Integrating Model Inference Code and `Micro` Lib
- Copy the generated inference code to the project, decompress the package obtained in [Downloading `Micro` Lib of Cortex-M Architecture](#downloading-micro-lib-of-cortex-m-architecture), and place it in the generated inference code directory, as shown in the following figure:
```text
test_stm32f767 # MCU project directory
├── Core
│ ├── Inc
│ └── Src
│ ├── main.c
│ └── ...
├── Drivers
├── EWARM # IAR project file directory
└── mnist # Generated Code Root Directory
├── benchmark # Benchmark routines for integrated calls to model inference code
│ ├── benchmark.c
│ ├── data.c
│ ├── data.h
│ └── ...
│── mindspore-lite-1.8.0-none-cortex-m7 # Downloaded Cortex-M7 Architecture `Micro` Lib
├── src # Model inference code directory
└── ...
```
- Import source files to the IAR project
Open the IAR project. On the `Workspace` page, right-click the project, choose `Add > Add Group` and add a `mnist` group. Right-click the group and repeat the operations to create groups `src` and `benchmark`.
Choose `Add -> Add Files` to import the source files in the `src` and `benchmark` directories in the `mnist` folder to the groups.
- Add the dependent header file path and static library
On the `Workspace` page, right-click the project and choose `Options` from the shortcut menu. Select `C/C++ Compiler` in the left pane of the project options window. In the right pane, select `Preprocessor` and add the path of the header file on which the inference code depends to the list. In this example, the path of the header file is as follows:
```text
$PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/runtime
$PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/runtime/include
$PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/include
$PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/third_party/include/CMSIS/Core
$PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/third_party/include/CMSIS/DSP
$PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/third_party/include/CMSIS/NN
$PROJ_DIR$/../mnist
```
In the left pane of the Project Options window, select `Linker`. In the right pane, select `Library` and add the operator static library file on which the inference code depends to the list. The static library file added in this example is as follows:
```text
$PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/lib/libwrapper.a
$PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/lib/libnnacl.a
$PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/third_party/lib/libcmsis_nn.a
```
- Modify the main.c file and invoke the benchmark function
Add a header file reference at the beginning of `main.c` and invoke the `benchmark` function in `benchmark.c` in the main function. The program in the benchmark folder is a sample program that invokes the inference code in the generated `src` and compares the output, which can be modified freely.
```c++
#include "benchmark/benchmark.h"
...
int main(void)
{
...
if (benchmark() == 0) {
printf("\nrun success.\n");
} else {
printf("\nrun failed.\n");
}
...
}
```
- Modify the `mnist/benchmark/data.c` file to store benchmark input and output data in the program for comparison
In the benchmark routine, the input data of the model is set and the inference result is compared with the expected result to obtain the error offset.
In this example, set the input data of the model by modifying the `calib_input0_data` array of `data.c`, and set the expected result by modifying the `calib_output0_data`.
```c++
float calib_input0_data[NET_INPUT0_SIZE] = {0.54881352186203,0.7151893377304077,0.6027633547782898,0.5448831915855408,0.42365479469299316,0.6458941102027893,0.4375872015953064,0.891772985458374,0.9636627435684204,0.3834415078163147,0.7917250394821167,0.5288949012756348,0.5680445432662964,0.9255966544151306,0.07103605568408966,0.08712930232286453,0.020218396559357643,0.832619845867157,0.7781567573547363,0.8700121641159058,0.978618323802948,0.7991585731506348,0.4614793658256531,0.7805292010307312,0.11827442795038223,0.6399210095405579,0.14335328340530396,0.9446688890457153,0.5218483209609985,0.4146619439125061,0.26455560326576233,0.7742336988449097,0.4561503231525421,0.568433940410614,0.018789799883961678,0.6176354885101318,0.6120957136154175,0.6169340014457703,0.9437480568885803,0.681820273399353,0.35950788855552673,0.43703195452690125,0.6976311802864075,0.0602254718542099,0.6667667031288147,0.670637845993042,0.21038256585597992,0.12892629206180573,0.31542834639549255,0.36371076107025146,0.5701967477798462,0.4386015236377716,0.9883738160133362,0.10204481333494186,0.20887675881385803,0.16130951046943665,0.6531082987785339,0.25329160690307617,0.4663107693195343,0.24442559480667114,0.15896958112716675,0.11037514358758926,0.6563295722007751,0.13818295300006866,0.1965823620557785,0.3687251806259155,0.8209932446479797,0.09710127860307693,0.8379449248313904,0.0960984081029892,0.9764594435691833,0.4686512053012848,0.9767611026763916,0.6048455238342285,0.7392635941505432,0.03918779268860817,0.28280696272850037,0.12019655853509903,0.296140193939209,0.11872772127389908,0.3179831802845001,0.414262980222702,0.06414749473333359,0.6924721002578735,0.5666014552116394,0.26538950204849243,0.5232480764389038,0.09394051134586334,0.5759465098381042,0.9292961955070496,0.3185689449310303,0.6674103736877441,0.13179786503314972,0.7163271903991699,0.28940609097480774,0.18319135904312134,0.5865129232406616,0.02010754682123661,0.8289400339126587,0.004695476032793522,0.6778165102005005,0.2700079679489136,0.7351940274238586,0.9621885418891907,0.2487531453371048,0.5761573314666748,0.5920419096946716,0.5722519159317017,0.22308163344860077,0.9527490139007568,0.4471253752708435,0.8464086651802063,0.6994792819023132,0.2974369525909424,0.8137978315353394,0.396505743265152,0.8811032176017761,0.5812729001045227,0.8817353844642639,0.6925315856933594,0.7252542972564697,0.5013243556022644,0.9560836553573608,0.6439902186393738,0.4238550364971161,0.6063932180404663,0.019193198531866074,0.30157482624053955,0.6601735353469849,0.2900775969028473,0.6180154085159302,0.42876869440078735,0.1354740709066391,0.29828232526779175,0.5699648857116699,0.5908727645874023,0.5743252635002136,0.6532008051872253,0.6521032452583313,0.43141844868659973,0.8965466022491455,0.36756187677383423,0.4358649253845215,0.8919233679771423,0.806194007396698,0.7038885951042175,0.10022688657045364,0.9194825887680054,0.7142413258552551,0.9988470077514648,0.14944830536842346,0.8681260347366333,0.16249293088912964,0.6155595779418945,0.1238199844956398,0.8480082154273987,0.8073189854621887,0.5691007375717163,0.40718328952789307,0.06916699558496475,0.6974287629127502,0.45354267954826355,0.7220556139945984,0.8663823008537292,0.9755215048789978,0.855803370475769,0.011714084073901176,0.359978049993515,0.729990541934967,0.17162968218326569,0.5210366249084473,0.054337989538908005,0.19999653100967407,0.01852179504930973,0.793697714805603,0.2239246815443039,0.3453516662120819,0.9280812740325928,0.704414427280426,0.031838931143283844,0.1646941602230072,0.6214783787727356,0.5772286057472229,0.23789282143115997,0.9342139959335327,0.6139659285545349,0.5356327891349792,0.5899099707603455,0.7301220297813416,0.31194499135017395,0.39822107553482056,0.20984375476837158,0.18619300425052643,0.9443724155426025,0.739550769329071,0.49045881628990173,0.22741462290287018,0.2543564736843109,0.058029159903526306,0.43441662192344666,0.3117958903312683,0.6963434815406799,0.37775182723999023,0.1796036809682846,0.024678727611899376,0.06724963337182999,0.6793927550315857,0.4536968469619751,0.5365791916847229,0.8966712951660156,0.990338921546936,0.21689698100090027,0.6630781888961792,0.2633223831653595,0.02065099962055683,0.7583786249160767,0.32001715898513794,0.38346388936042786,0.5883170962333679,0.8310484290122986,0.6289818286895752,0.872650682926178,0.27354204654693604,0.7980468273162842,0.18563593924045563,0.9527916312217712,0.6874882578849792,0.21550767123699188,0.9473705887794495,0.7308558225631714,0.2539416551589966,0.21331197023391724,0.518200695514679,0.02566271834075451,0.20747007429599762,0.4246854782104492,0.3741699755191803,0.46357542276382446,0.27762871980667114,0.5867843627929688,0.8638556003570557,0.11753185838460922,0.517379105091095,0.13206811249256134,0.7168596982955933,0.39605969190597534,0.5654212832450867,0.1832798421382904,0.14484776556491852,0.4880562722682953,0.35561272501945496,0.9404319524765015,0.7653252482414246,0.748663604259491,0.9037197232246399,0.08342243731021881,0.5521924495697021,0.5844760537147522,0.961936354637146,0.29214751720428467,0.24082878232002258,0.10029394179582596,0.016429629176855087,0.9295293092727661,0.669916570186615,0.7851529121398926,0.28173011541366577,0.5864101648330688,0.06395526975393295,0.48562759160995483,0.9774951338768005,0.8765052556991577,0.3381589651107788,0.961570143699646,0.23170162737369537,0.9493188261985779,0.9413776993751526,0.799202561378479,0.6304479241371155,0.8742879629135132,0.2930202782154083,0.8489435315132141,0.6178767085075378,0.013236857950687408,0.34723350405693054,0.14814086258411407,0.9818294048309326,0.4783703088760376,0.49739137291908264,0.6394725441932678,0.36858460307121277,0.13690027594566345,0.8221177458763123,0.1898479163646698,0.5113189816474915,0.2243170291185379,0.09784448146820068,0.8621914982795715,0.9729194641113281,0.9608346819877625,0.9065554738044739,0.774047315120697,0.3331451416015625,0.08110138773918152,0.40724116563796997,0.2322341352701187,0.13248763978481293,0.053427182137966156,0.7255943417549133,0.011427458375692368,0.7705807685852051,0.14694663882255554,0.07952208071947098,0.08960303664207458,0.6720477938652039,0.24536721408367157,0.4205394685268402,0.557368814945221,0.8605511784553528,0.7270442843437195,0.2703278958797455,0.131482794880867,0.05537432059645653,0.3015986382961273,0.2621181607246399,0.45614057779312134,0.6832813620567322,0.6956254243850708,0.28351885080337524,0.3799269497394562,0.18115095794200897,0.7885454893112183,0.05684807524085045,0.6969972252845764,0.7786954045295715,0.7774075865745544,0.25942257046699524,0.3738131523132324,0.5875996351242065,0.27282190322875977,0.3708527982234955,0.19705428183078766,0.4598558843135834,0.044612299650907516,0.7997958660125732,0.07695644348859787,0.5188351273536682,0.3068101108074188,0.5775429606437683,0.9594333171844482,0.6455702185630798,0.03536243736743927,0.4304024279117584,0.5100168585777283,0.5361775159835815,0.6813924908638,0.2775960862636566,0.12886056303977966,0.3926756680011749,0.9564056992530823,0.1871308982372284,0.9039839506149292,0.5438059568405151,0.4569114148616791,0.8820413947105408,0.45860394835472107,0.7241676449775696,0.3990253210067749,0.9040443897247314,0.6900250315666199,0.6996220350265503,0.32772040367126465,0.7567786574363708,0.6360610723495483,0.2400202751159668,0.16053882241249084,0.796391487121582,0.9591665863990784,0.4581388235092163,0.5909841656684875,0.8577226400375366,0.45722344517707825,0.9518744945526123,0.5757511854171753,0.8207671046257019,0.9088436961174011,0.8155238032341003,0.15941447019577026,0.6288984417915344,0.39843425154685974,0.06271295249462128,0.4240322411060333,0.25868406891822815,0.849038302898407,0.03330462798476219,0.9589827060699463,0.35536885261535645,0.3567068874835968,0.01632850244641304,0.18523232638835907,0.40125951170921326,0.9292914271354675,0.0996149331331253,0.9453015327453613,0.869488537311554,0.4541623890399933,0.326700896024704,0.23274412751197815,0.6144647002220154,0.03307459130883217,0.015606064349412918,0.428795725107193,0.06807407736778259,0.2519409954547882,0.2211609184741974,0.253191202878952,0.13105523586273193,0.012036222964525223,0.11548429727554321,0.6184802651405334,0.9742562174797058,0.9903450012207031,0.40905410051345825,0.1629544198513031,0.6387617588043213,0.4903053343296051,0.9894098043441772,0.06530420482158661,0.7832344174385071,0.28839850425720215,0.24141861498355865,0.6625045537948608,0.24606318771839142,0.6658591032028198,0.5173085331916809,0.4240889847278595,0.5546877980232239,0.2870515286922455,0.7065746784210205,0.414856880903244,0.3605455458164215,0.8286569118499756,0.9249669313430786,0.04600730910897255,0.2326269894838333,0.34851935505867004,0.8149664998054504,0.9854914546012878,0.9689717292785645,0.904948353767395,0.2965562641620636,0.9920112490653992,0.24942004680633545,0.10590615123510361,0.9509525895118713,0.2334202527999878,0.6897682547569275,0.05835635960102081,0.7307090759277344,0.8817201852798462,0.27243688702583313,0.3790569007396698,0.3742961883544922,0.7487882375717163,0.2378072440624237,0.17185309529304504,0.4492916464805603,0.30446839332580566,0.8391891121864319,0.23774182796478271,0.5023894309997559,0.9425836205482483,0.6339976787567139,0.8672894239425659,0.940209686756134,0.7507648468017578,0.6995750665664673,0.9679655432701111,0.9944007992744446,0.4518216848373413,0.07086978107690811,0.29279401898384094,0.15235470235347748,0.41748636960983276,0.13128933310508728,0.6041178107261658,0.38280805945396423,0.8953858613967896,0.96779465675354,0.5468848943710327,0.2748235762119293,0.5922304391860962,0.8967611789703369,0.40673333406448364,0.5520782470703125,0.2716527581214905,0.4554441571235657,0.4017135500907898,0.24841345846652985,0.5058664083480835,0.31038081645965576,0.37303486466407776,0.5249704718589783,0.7505950331687927,0.3335074782371521,0.9241587519645691,0.8623185753822327,0.048690296709537506,0.2536425292491913,0.4461355209350586,0.10462789237499237,0.34847599267959595,0.7400975227355957,0.6805144548416138,0.6223844289779663,0.7105283737182617,0.20492368936538696,0.3416981101036072,0.676242470741272,0.879234790802002,0.5436780452728271,0.2826996445655823,0.030235258862376213,0.7103368043899536,0.007884103804826736,0.37267908453941345,0.5305371880531311,0.922111451625824,0.08949454873800278,0.40594232082366943,0.024313200265169144,0.3426109850406647,0.6222310662269592,0.2790679335594177,0.2097499519586563,0.11570323258638382,0.5771402716636658,0.6952700018882751,0.6719571352005005,0.9488610029220581,0.002703213831409812,0.6471966505050659,0.60039222240448,0.5887396335601807,0.9627703428268433,0.016871673986315727,0.6964824199676514,0.8136786222457886,0.5098071694374084,0.33396488428115845,0.7908401489257812,0.09724292904138565,0.44203564524650574,0.5199523568153381,0.6939564347267151,0.09088572859764099,0.2277594953775406,0.4103015661239624,0.6232946515083313,0.8869608044624329,0.618826150894165,0.13346147537231445,0.9805801510810852,0.8717857599258423,0.5027207732200623,0.9223479628562927,0.5413808226585388,0.9233060479164124,0.8298973441123962,0.968286395072937,0.919782817363739,0.03603381663560867,0.1747720092535019,0.3891346752643585,0.9521427154541016,0.300028920173645,0.16046763956546783,0.8863046765327454,0.4463944137096405,0.9078755974769592,0.16023047268390656,0.6611174941062927,0.4402637481689453,0.07648676633834839,0.6964631676673889,0.2473987489938736,0.03961552307009697,0.05994429811835289,0.06107853725552559,0.9077329635620117,0.7398838996887207,0.8980623483657837,0.6725823283195496,0.5289399027824402,0.30444636940956116,0.997962236404419,0.36218905448913574,0.47064894437789917,0.37824517488479614,0.979526937007904,0.1746583878993988,0.32798799872398376,0.6803486943244934,0.06320761889219284,0.60724937915802,0.47764649987220764,0.2839999794960022,0.2384132742881775,0.5145127177238464,0.36792758107185364,0.4565199017524719,0.3374773859977722,0.9704936742782593,0.13343943655490875,0.09680395573377609,0.3433917164802551,0.5910269021987915,0.6591764688491821,0.3972567617893219,0.9992780089378357,0.35189300775527954,0.7214066386222839,0.6375827193260193,0.8130538463592529,0.9762256741523743,0.8897936344146729,0.7645619511604309,0.6982485055923462,0.335498183965683,0.14768557250499725,0.06263600289821625,0.2419017106294632,0.432281494140625,0.521996259689331,0.7730835676193237,0.9587409496307373,0.1173204779624939,0.10700414329767227,0.5896947383880615,0.7453980445861816,0.848150372505188,0.9358320832252502,0.9834262132644653,0.39980170130729675,0.3803351819515228,0.14780867099761963,0.6849344372749329,0.6567619442939758,0.8620625734329224,0.09725799411535263,0.49777689576148987,0.5810819268226624,0.2415570467710495,0.16902540624141693,0.8595808148384094,0.05853492394089699,0.47062090039253235,0.11583399772644043,0.45705875754356384,0.9799623489379883,0.4237063527107239,0.857124924659729,0.11731556057929993,0.2712520658969879,0.40379273891448975,0.39981213212013245,0.6713835000991821,0.3447181284427643,0.713766872882843,0.6391869187355042,0.399161159992218,0.43176013231277466,0.614527702331543,0.0700421929359436,0.8224067091941833,0.65342116355896,0.7263424396514893,0.5369229912757874,0.11047711223363876,0.4050356149673462,0.40537357330322266,0.3210429847240448,0.029950324445962906,0.73725426197052,0.10978446155786514,0.6063081622123718,0.7032175064086914,0.6347863078117371,0.95914226770401,0.10329815745353699,0.8671671748161316,0.02919023483991623,0.534916877746582,0.4042436182498932,0.5241838693618774,0.36509987711906433,0.19056691229343414,0.01912289671599865,0.5181497931480408,0.8427768349647522,0.3732159435749054,0.2228638231754303,0.080532006919384,0.0853109210729599,0.22139644622802734,0.10001406073570251,0.26503971219062805,0.06614946573972702,0.06560486555099487,0.8562761545181274,0.1621202677488327,0.5596824288368225,0.7734555602073669,0.4564095735549927,0.15336887538433075,0.19959613680839539,0.43298420310020447,0.52823406457901,0.3494403064250946,0.7814795970916748,0.7510216236114502,0.9272118210792542,0.028952548280358315,0.8956912755966187,0.39256879687309265,0.8783724904060364,0.690784752368927,0.987348735332489,0.7592824697494507,0.3645446300506592,0.5010631680488586,0.37638914585113525,0.364911824464798,0.2609044909477234,0.49597030878067017,0.6817399263381958,0.27734026312828064,0.5243797898292542,0.117380291223526,0.1598452925682068,0.04680635407567024,0.9707314372062683,0.0038603513967245817,0.17857997119426727,0.6128667593002319,0.08136960119009018,0.8818964958190918,0.7196201682090759,0.9663899540901184,0.5076355338096619,0.3004036843776703,0.549500584602356,0.9308187365531921,0.5207614302635193,0.2672070264816284,0.8773987889289856,0.3719187378883362,0.0013833499979227781,0.2476850152015686,0.31823351979255676,0.8587774634361267,0.4585031569004059,0.4445872902870178,0.33610227704048157,0.880678117275238,0.9450267553329468,0.9918903112411499,0.3767412602901459,0.9661474227905273,0.7918795943260193,0.675689160823822,0.24488948285579681,0.21645726263523102,0.1660478264093399,0.9227566123008728,0.2940766513347626,0.4530942440032959,0.49395784735679626,0.7781715989112854,0.8442349433898926,0.1390727013349533,0.4269043505191803,0.842854917049408,0.8180332779884338};
float calib_output0_data[NET_OUTPUT0_SIZE] = {3.5647096e-05,6.824297e-08,0.009327697,3.2340475e-05,1.1117579e-05,1.5117058e-06,4.6314454e-07,5.161628e-11,0.9905911,3.8835238e-10};
```
#### Compiling and Simulation Run
In this example, software simulation is used to view and analyze the inference result.
On the `Workspace` page, right-click the project and choose `Options` from the shortcut menu. Select the `Debugger` option on the left of the Project Options window. In the `Setup` on the right, set `Driver` as `Simulator` to enable software simulation.
Close the project option window and choose `Project > Download and Debug` on the menu bar to compile and simulate the project. By adding a breakpoint at the `benchmark`, you can observe the inference result of the simulation run and the return value of the benchmark() function.
### Code Integration and Compilation Deployment on Linux: Code Integration via MakeFile
This chapter describes how to integrate and develop the MCU inference code on the Linux platform, by taking the generated model code integration and developing through MakeFile on the Linux platform as an example.
The main steps are as follows:
- Download the required software and prepare the cross compilation and burning environment
- Generate the required MCU startup code and demonstration project using the `STM32CubeMX` software
- Modify the inference code and `Micro` lib of the `MakeFile` integrated model
- Compiling and burning the project
- Read and verify running result of the board
For the complete demo code built in this example, click [Download here](https://download.mindspore.cn/model_zoo/official/lite/quick_start/micro/test_stmf767.tar.gz).
#### Environment Preparation
- [CMake](https://cmake.org/download/) >= 3.18.3
- [GNU Arm Embedded Toolchain](https://developer.arm.com/downloads/-/gnu-rm) >= 10-2020-q4-major-x86_64-linux
- This tool is a cross compilation tool for Cortex-M Linux.
- Download the `gcc-arm-none-eabi` package of the x86_64-Linux version, decompress the package, and add the bin path in the directory to the PATH environment variable: `export PATH=gcc-arm-none-eabi path/bin:$PATH`.
- [STM32CubeMX-Lin](https://www.st.com/en/development-tools/stm32cubemx.html) >= 6.5.0
- `STM32CubeMX` is a graphical configuration tool for STM32 chips provided by `STM`, which is used to generate the startup code and project of STM chips.
- [STM32CubePrg-Lin](https://www.st.com/en/development-tools/stm32cubeprog.html) >= 6.5.0
- This tool is a burning tool provided by `STM` and can be used for program burning and data reading.
#### Obtaining the MCU Startup Code and Project
If you have an MCU project, skip this chapter.
This chapter uses the STM32F767 startup project as an example to describe how to generate an MCU project for an STM32 chip via `STM32CubeMX`.
- Start `STM32CubeMX` and select `New Project` from `File` to create a project
- In the `MCU/MPU Selector` window, search for and select `STM32F767IGT6`, and click `Start Project` to create a project for the chip
- On the `Project Manager` page, configure the project name and the path of the generated project, and select `EWARM` in `Toolchain / IDE` to generate the IAR project
- Click `GENERATE CODE` above to generate code
- On the PC where the `IAR` is installed, double-click `Project.eww` in the `EWARM` directory of the generated project to open the IAR project
#### Integrating Model Inference Code and `Micro` Lib
- Copy the generated inference code to the project, decompress the package obtained in [Downloading `Micro` Lib of Cortex-M Architecture](#downloading-micro-lib-of-cortex-m-architecture), and place it in the generated inference code directory, as shown in the following figure:
```text
stm32f767 # MCU project directory
├── Core
│ ├── Inc
│ └── Src
│ ├── main.c
│ └── ...
├── Drivers
├── mnist # Generate Code Root Directory
│ ├── benchmark # Benchmark routines for integrated calls to model inference code
│ │ ├── benchmark.c
│ │ ├── data.c
│ │ ├── data.h
│ │ └── ...
│ │── mindspore-lite-1.8.0-none-cortex-m7 # Downloaded Cortex-M7 Architecture `Micro` Lib
│ ├── src # Model inference code directory
│ └── ...
├── Makefile
├── startup_stm32f767xx.s
└── STM32F767IGTx_FLASH.ld
```
- Modify `MakeFile` and add the model inference code and dependency library to the project
In this example, the source code to be added to the project includes the model inference code in the `src` directory and the example code called by the model inference in the `benchmark` directory.
Modify the definition of the `C_SOURCES` variable in `MakeFile` and add the source file path.
```bash
C_SOURCES = \
mnist/src/context.c \
mnist/src/model.c \
mnist/src/net.c \
mnist/src/tensor.c \
mnist/src/weight.c \
mnist/benchmark/benchmark.c \
mnist/benchmark/calib_output.c \
mnist/benchmark/load_input.c \
mnist/benchmark/data.c \
...
```
Add the path of the dependent header file: Modify the definition of the `C_INCLUDES` variable in `MakeFile` and add the following path:
```text
LITE_PACK = mindspore-lite-1.8.0-none-cortex-m7
C_INCLUDES = \
-Imnist/$(LITE_PACK)/runtime \
-Imnist/$(LITE_PACK)/runtime/include \
-Imnist/$(LITE_PACK)/tools/codegen/include \
-Imnist/$(LITE_PACK)/tools/codegen/third_party/include/CMSIS/Core \
-Imnist/$(LITE_PACK)/tools/codegen/third_party/include/CMSIS/DSP \
-Imnist/$(LITE_PACK)/tools/codegen/third_party/include/CMSIS/NN \
-Imnist \
...
```
Add the dependent operator library (`-lnnacl -lwrapper -lcmsis_nn`), declare the path of the operator library file, and add the compilation option (`-specs=nosys.specs`).
In this example, the modified variables are defined as follows:
```text
LIBS = -lc -lm -lnosys -lnnacl -lwrapper -lcmsis_nn
LIBDIR = -Lmnist/$(LITE_PACK)/tools/codegen/lib -Lmnist/$(LITE_PACK)/tools/codegen/third_party/lib
LDFLAGS = $(MCU) -specs=nosys.specs -specs=nano.specs -T$(LDSCRIPT) $(LIBDIR) $(LIBS) -Wl,-Map=$(BUILD_DIR)/$(TARGET).map,--cref -Wl,--gc-sections
```
- Modify the main.c file and invoke the benchmark function
Invoke the `benchmark` function in `benchmark.c` in the main function. The program in the benchmark folder is a sample program that invokes the inference code in the generated `src` and compares the output, which can be modified freely. In this example, we call the `benchmark` function directly and assign the `run_dnn_flag` variable based on the returned result.
```c++
run_dnn_flag = '0';
if (benchmark() == 0) {
printf("\nrun success.\n");
run_dnn_flag = '1';
} else {
printf("\nrun failed.\n");
run_dnn_flag = '2';
}
```
Add the header file reference and the definition of the `run_dnn_flag` variable to the beginning of `main.c`.
```c++
#include "benchmark/benchmark.h"
char run_dnn_flag __attribute__((section(".myram"))) ;// Array for testing
```
In this example, to facilitate reading the inference result by using the burner, variables are defined in a customized section (`myram`). You can set the customized section in the following way or ignore the declaration: obtaining the inference result through serial ports or other interactive modes.
To set a customized section, perform the following steps:
Modify the `MEMORY` section in the `STM32F767IGTx_FLASH.ld` file, and add a customized memory segment `MYRAM`. (In this example, add 4 to the `RAM` memory start address to free up memory for `MYRAM`). Then add a customized `myram` segment declaration to the `SectionS` segment.
```text
MEMORY
{
MYRAM (xrw) : ORIGIN = 0x20000000, LENGTH = 1
RAM (xrw) : ORIGIN = 0x20000004, LENGTH = 524284
...
}
...
SECTIONS
{
...
.myram (NOLOAD):
{
. = ALIGN(4);
_smyram = .; /* create a global symbol at data start */
*(.sram) /* .data sections */
*(.sram*) /* .data* sections */
. = ALIGN(4);
_emyram = .; /* define a global symbol at data end */
} >MYRAM AT> FLASH
}
```
- Modify the `mnist/benchmark/data.c` file to store benchmark input and output data in the program for comparison.
In the benchmark routine, the input data of the model is set and the inference result is compared with the expected result to obtain the error offset.
In this example, modify the `calib_input0_data` array of `data.c` to set the input data of the model, and modify the `calib_output0_data` to set the expected result.
```c++
float calib_input0_data[NET_INPUT0_SIZE] = {0.54881352186203,0.7151893377304077,0.6027633547782898,0.5448831915855408,0.42365479469299316,0.6458941102027893,0.4375872015953064,0.891772985458374,0.9636627435684204,0.3834415078163147,0.7917250394821167,0.5288949012756348,0.5680445432662964,0.9255966544151306,0.07103605568408966,0.08712930232286453,0.020218396559357643,0.832619845867157,0.7781567573547363,0.8700121641159058,0.978618323802948,0.7991585731506348,0.4614793658256531,0.7805292010307312,0.11827442795038223,0.6399210095405579,0.14335328340530396,0.9446688890457153,0.5218483209609985,0.4146619439125061,0.26455560326576233,0.7742336988449097,0.4561503231525421,0.568433940410614,0.018789799883961678,0.6176354885101318,0.6120957136154175,0.6169340014457703,0.9437480568885803,0.681820273399353,0.35950788855552673,0.43703195452690125,0.6976311802864075,0.0602254718542099,0.6667667031288147,0.670637845993042,0.21038256585597992,0.12892629206180573,0.31542834639549255,0.36371076107025146,0.5701967477798462,0.4386015236377716,0.9883738160133362,0.10204481333494186,0.20887675881385803,0.16130951046943665,0.6531082987785339,0.25329160690307617,0.4663107693195343,0.24442559480667114,0.15896958112716675,0.11037514358758926,0.6563295722007751,0.13818295300006866,0.1965823620557785,0.3687251806259155,0.8209932446479797,0.09710127860307693,0.8379449248313904,0.0960984081029892,0.9764594435691833,0.4686512053012848,0.9767611026763916,0.6048455238342285,0.7392635941505432,0.03918779268860817,0.28280696272850037,0.12019655853509903,0.296140193939209,0.11872772127389908,0.3179831802845001,0.414262980222702,0.06414749473333359,0.6924721002578735,0.5666014552116394,0.26538950204849243,0.5232480764389038,0.09394051134586334,0.5759465098381042,0.9292961955070496,0.3185689449310303,0.6674103736877441,0.13179786503314972,0.7163271903991699,0.28940609097480774,0.18319135904312134,0.5865129232406616,0.02010754682123661,0.8289400339126587,0.004695476032793522,0.6778165102005005,0.2700079679489136,0.7351940274238586,0.9621885418891907,0.2487531453371048,0.5761573314666748,0.5920419096946716,0.5722519159317017,0.22308163344860077,0.9527490139007568,0.4471253752708435,0.8464086651802063,0.6994792819023132,0.2974369525909424,0.8137978315353394,0.396505743265152,0.8811032176017761,0.5812729001045227,0.8817353844642639,0.6925315856933594,0.7252542972564697,0.5013243556022644,0.9560836553573608,0.6439902186393738,0.4238550364971161,0.6063932180404663,0.019193198531866074,0.30157482624053955,0.6601735353469849,0.2900775969028473,0.6180154085159302,0.42876869440078735,0.1354740709066391,0.29828232526779175,0.5699648857116699,0.5908727645874023,0.5743252635002136,0.6532008051872253,0.6521032452583313,0.43141844868659973,0.8965466022491455,0.36756187677383423,0.4358649253845215,0.8919233679771423,0.806194007396698,0.7038885951042175,0.10022688657045364,0.9194825887680054,0.7142413258552551,0.9988470077514648,0.14944830536842346,0.8681260347366333,0.16249293088912964,0.6155595779418945,0.1238199844956398,0.8480082154273987,0.8073189854621887,0.5691007375717163,0.40718328952789307,0.06916699558496475,0.6974287629127502,0.45354267954826355,0.7220556139945984,0.8663823008537292,0.9755215048789978,0.855803370475769,0.011714084073901176,0.359978049993515,0.729990541934967,0.17162968218326569,0.5210366249084473,0.054337989538908005,0.19999653100967407,0.01852179504930973,0.793697714805603,0.2239246815443039,0.3453516662120819,0.9280812740325928,0.704414427280426,0.031838931143283844,0.1646941602230072,0.6214783787727356,0.5772286057472229,0.23789282143115997,0.9342139959335327,0.6139659285545349,0.5356327891349792,0.5899099707603455,0.7301220297813416,0.31194499135017395,0.39822107553482056,0.20984375476837158,0.18619300425052643,0.9443724155426025,0.739550769329071,0.49045881628990173,0.22741462290287018,0.2543564736843109,0.058029159903526306,0.43441662192344666,0.3117958903312683,0.6963434815406799,0.37775182723999023,0.1796036809682846,0.024678727611899376,0.06724963337182999,0.6793927550315857,0.4536968469619751,0.5365791916847229,0.8966712951660156,0.990338921546936,0.21689698100090027,0.6630781888961792,0.2633223831653595,0.02065099962055683,0.7583786249160767,0.32001715898513794,0.38346388936042786,0.5883170962333679,0.8310484290122986,0.6289818286895752,0.872650682926178,0.27354204654693604,0.7980468273162842,0.18563593924045563,0.9527916312217712,0.6874882578849792,0.21550767123699188,0.9473705887794495,0.7308558225631714,0.2539416551589966,0.21331197023391724,0.518200695514679,0.02566271834075451,0.20747007429599762,0.4246854782104492,0.3741699755191803,0.46357542276382446,0.27762871980667114,0.5867843627929688,0.8638556003570557,0.11753185838460922,0.517379105091095,0.13206811249256134,0.7168596982955933,0.39605969190597534,0.5654212832450867,0.1832798421382904,0.14484776556491852,0.4880562722682953,0.35561272501945496,0.9404319524765015,0.7653252482414246,0.748663604259491,0.9037197232246399,0.08342243731021881,0.5521924495697021,0.5844760537147522,0.961936354637146,0.29214751720428467,0.24082878232002258,0.10029394179582596,0.016429629176855087,0.9295293092727661,0.669916570186615,0.7851529121398926,0.28173011541366577,0.5864101648330688,0.06395526975393295,0.48562759160995483,0.9774951338768005,0.8765052556991577,0.3381589651107788,0.961570143699646,0.23170162737369537,0.9493188261985779,0.9413776993751526,0.799202561378479,0.6304479241371155,0.8742879629135132,0.2930202782154083,0.8489435315132141,0.6178767085075378,0.013236857950687408,0.34723350405693054,0.14814086258411407,0.9818294048309326,0.4783703088760376,0.49739137291908264,0.6394725441932678,0.36858460307121277,0.13690027594566345,0.8221177458763123,0.1898479163646698,0.5113189816474915,0.2243170291185379,0.09784448146820068,0.8621914982795715,0.9729194641113281,0.9608346819877625,0.9065554738044739,0.774047315120697,0.3331451416015625,0.08110138773918152,0.40724116563796997,0.2322341352701187,0.13248763978481293,0.053427182137966156,0.7255943417549133,0.011427458375692368,0.7705807685852051,0.14694663882255554,0.07952208071947098,0.08960303664207458,0.6720477938652039,0.24536721408367157,0.4205394685268402,0.557368814945221,0.8605511784553528,0.7270442843437195,0.2703278958797455,0.131482794880867,0.05537432059645653,0.3015986382961273,0.2621181607246399,0.45614057779312134,0.6832813620567322,0.6956254243850708,0.28351885080337524,0.3799269497394562,0.18115095794200897,0.7885454893112183,0.05684807524085045,0.6969972252845764,0.7786954045295715,0.7774075865745544,0.25942257046699524,0.3738131523132324,0.5875996351242065,0.27282190322875977,0.3708527982234955,0.19705428183078766,0.4598558843135834,0.044612299650907516,0.7997958660125732,0.07695644348859787,0.5188351273536682,0.3068101108074188,0.5775429606437683,0.9594333171844482,0.6455702185630798,0.03536243736743927,0.4304024279117584,0.5100168585777283,0.5361775159835815,0.6813924908638,0.2775960862636566,0.12886056303977966,0.3926756680011749,0.9564056992530823,0.1871308982372284,0.9039839506149292,0.5438059568405151,0.4569114148616791,0.8820413947105408,0.45860394835472107,0.7241676449775696,0.3990253210067749,0.9040443897247314,0.6900250315666199,0.6996220350265503,0.32772040367126465,0.7567786574363708,0.6360610723495483,0.2400202751159668,0.16053882241249084,0.796391487121582,0.9591665863990784,0.4581388235092163,0.5909841656684875,0.8577226400375366,0.45722344517707825,0.9518744945526123,0.5757511854171753,0.8207671046257019,0.9088436961174011,0.8155238032341003,0.15941447019577026,0.6288984417915344,0.39843425154685974,0.06271295249462128,0.4240322411060333,0.25868406891822815,0.849038302898407,0.03330462798476219,0.9589827060699463,0.35536885261535645,0.3567068874835968,0.01632850244641304,0.18523232638835907,0.40125951170921326,0.9292914271354675,0.0996149331331253,0.9453015327453613,0.869488537311554,0.4541623890399933,0.326700896024704,0.23274412751197815,0.6144647002220154,0.03307459130883217,0.015606064349412918,0.428795725107193,0.06807407736778259,0.2519409954547882,0.2211609184741974,0.253191202878952,0.13105523586273193,0.012036222964525223,0.11548429727554321,0.6184802651405334,0.9742562174797058,0.9903450012207031,0.40905410051345825,0.1629544198513031,0.6387617588043213,0.4903053343296051,0.9894098043441772,0.06530420482158661,0.7832344174385071,0.28839850425720215,0.24141861498355865,0.6625045537948608,0.24606318771839142,0.6658591032028198,0.5173085331916809,0.4240889847278595,0.5546877980232239,0.2870515286922455,0.7065746784210205,0.414856880903244,0.3605455458164215,0.8286569118499756,0.9249669313430786,0.04600730910897255,0.2326269894838333,0.34851935505867004,0.8149664998054504,0.9854914546012878,0.9689717292785645,0.904948353767395,0.2965562641620636,0.9920112490653992,0.24942004680633545,0.10590615123510361,0.9509525895118713,0.2334202527999878,0.6897682547569275,0.05835635960102081,0.7307090759277344,0.8817201852798462,0.27243688702583313,0.3790569007396698,0.3742961883544922,0.7487882375717163,0.2378072440624237,0.17185309529304504,0.4492916464805603,0.30446839332580566,0.8391891121864319,0.23774182796478271,0.5023894309997559,0.9425836205482483,0.6339976787567139,0.8672894239425659,0.940209686756134,0.7507648468017578,0.6995750665664673,0.9679655432701111,0.9944007992744446,0.4518216848373413,0.07086978107690811,0.29279401898384094,0.15235470235347748,0.41748636960983276,0.13128933310508728,0.6041178107261658,0.38280805945396423,0.8953858613967896,0.96779465675354,0.5468848943710327,0.2748235762119293,0.5922304391860962,0.8967611789703369,0.40673333406448364,0.5520782470703125,0.2716527581214905,0.4554441571235657,0.4017135500907898,0.24841345846652985,0.5058664083480835,0.31038081645965576,0.37303486466407776,0.5249704718589783,0.7505950331687927,0.3335074782371521,0.9241587519645691,0.8623185753822327,0.048690296709537506,0.2536425292491913,0.4461355209350586,0.10462789237499237,0.34847599267959595,0.7400975227355957,0.6805144548416138,0.6223844289779663,0.7105283737182617,0.20492368936538696,0.3416981101036072,0.676242470741272,0.879234790802002,0.5436780452728271,0.2826996445655823,0.030235258862376213,0.7103368043899536,0.007884103804826736,0.37267908453941345,0.5305371880531311,0.922111451625824,0.08949454873800278,0.40594232082366943,0.024313200265169144,0.3426109850406647,0.6222310662269592,0.2790679335594177,0.2097499519586563,0.11570323258638382,0.5771402716636658,0.6952700018882751,0.6719571352005005,0.9488610029220581,0.002703213831409812,0.6471966505050659,0.60039222240448,0.5887396335601807,0.9627703428268433,0.016871673986315727,0.6964824199676514,0.8136786222457886,0.5098071694374084,0.33396488428115845,0.7908401489257812,0.09724292904138565,0.44203564524650574,0.5199523568153381,0.6939564347267151,0.09088572859764099,0.2277594953775406,0.4103015661239624,0.6232946515083313,0.8869608044624329,0.618826150894165,0.13346147537231445,0.9805801510810852,0.8717857599258423,0.5027207732200623,0.9223479628562927,0.5413808226585388,0.9233060479164124,0.8298973441123962,0.968286395072937,0.919782817363739,0.03603381663560867,0.1747720092535019,0.3891346752643585,0.9521427154541016,0.300028920173645,0.16046763956546783,0.8863046765327454,0.4463944137096405,0.9078755974769592,0.16023047268390656,0.6611174941062927,0.4402637481689453,0.07648676633834839,0.6964631676673889,0.2473987489938736,0.03961552307009697,0.05994429811835289,0.06107853725552559,0.9077329635620117,0.7398838996887207,0.8980623483657837,0.6725823283195496,0.5289399027824402,0.30444636940956116,0.997962236404419,0.36218905448913574,0.47064894437789917,0.37824517488479614,0.979526937007904,0.1746583878993988,0.32798799872398376,0.6803486943244934,0.06320761889219284,0.60724937915802,0.47764649987220764,0.2839999794960022,0.2384132742881775,0.5145127177238464,0.36792758107185364,0.4565199017524719,0.3374773859977722,0.9704936742782593,0.13343943655490875,0.09680395573377609,0.3433917164802551,0.5910269021987915,0.6591764688491821,0.3972567617893219,0.9992780089378357,0.35189300775527954,0.7214066386222839,0.6375827193260193,0.8130538463592529,0.9762256741523743,0.8897936344146729,0.7645619511604309,0.6982485055923462,0.335498183965683,0.14768557250499725,0.06263600289821625,0.2419017106294632,0.432281494140625,0.521996259689331,0.7730835676193237,0.9587409496307373,0.1173204779624939,0.10700414329767227,0.5896947383880615,0.7453980445861816,0.848150372505188,0.9358320832252502,0.9834262132644653,0.39980170130729675,0.3803351819515228,0.14780867099761963,0.6849344372749329,0.6567619442939758,0.8620625734329224,0.09725799411535263,0.49777689576148987,0.5810819268226624,0.2415570467710495,0.16902540624141693,0.8595808148384094,0.05853492394089699,0.47062090039253235,0.11583399772644043,0.45705875754356384,0.9799623489379883,0.4237063527107239,0.857124924659729,0.11731556057929993,0.2712520658969879,0.40379273891448975,0.39981213212013245,0.6713835000991821,0.3447181284427643,0.713766872882843,0.6391869187355042,0.399161159992218,0.43176013231277466,0.614527702331543,0.0700421929359436,0.8224067091941833,0.65342116355896,0.7263424396514893,0.5369229912757874,0.11047711223363876,0.4050356149673462,0.40537357330322266,0.3210429847240448,0.029950324445962906,0.73725426197052,0.10978446155786514,0.6063081622123718,0.7032175064086914,0.6347863078117371,0.95914226770401,0.10329815745353699,0.8671671748161316,0.02919023483991623,0.534916877746582,0.4042436182498932,0.5241838693618774,0.36509987711906433,0.19056691229343414,0.01912289671599865,0.5181497931480408,0.8427768349647522,0.3732159435749054,0.2228638231754303,0.080532006919384,0.0853109210729599,0.22139644622802734,0.10001406073570251,0.26503971219062805,0.06614946573972702,0.06560486555099487,0.8562761545181274,0.1621202677488327,0.5596824288368225,0.7734555602073669,0.4564095735549927,0.15336887538433075,0.19959613680839539,0.43298420310020447,0.52823406457901,0.3494403064250946,0.7814795970916748,0.7510216236114502,0.9272118210792542,0.028952548280358315,0.8956912755966187,0.39256879687309265,0.8783724904060364,0.690784752368927,0.987348735332489,0.7592824697494507,0.3645446300506592,0.5010631680488586,0.37638914585113525,0.364911824464798,0.2609044909477234,0.49597030878067017,0.6817399263381958,0.27734026312828064,0.5243797898292542,0.117380291223526,0.1598452925682068,0.04680635407567024,0.9707314372062683,0.0038603513967245817,0.17857997119426727,0.6128667593002319,0.08136960119009018,0.8818964958190918,0.7196201682090759,0.9663899540901184,0.5076355338096619,0.3004036843776703,0.549500584602356,0.9308187365531921,0.5207614302635193,0.2672070264816284,0.8773987889289856,0.3719187378883362,0.0013833499979227781,0.2476850152015686,0.31823351979255676,0.8587774634361267,0.4585031569004059,0.4445872902870178,0.33610227704048157,0.880678117275238,0.9450267553329468,0.9918903112411499,0.3767412602901459,0.9661474227905273,0.7918795943260193,0.675689160823822,0.24488948285579681,0.21645726263523102,0.1660478264093399,0.9227566123008728,0.2940766513347626,0.4530942440032959,0.49395784735679626,0.7781715989112854,0.8442349433898926,0.1390727013349533,0.4269043505191803,0.842854917049408,0.8180332779884338};
float calib_output0_data[NET_OUTPUT0_SIZE] = {3.5647096e-05,6.824297e-08,0.009327697,3.2340475e-05,1.1117579e-05,1.5117058e-06,4.6314454e-07,5.161628e-11,0.9905911,3.8835238e-10};
```
#### Compiling and Burning the Project
- Compile
In the MCU project directory, run the `make` command to compile the MCU project. After the compilation is successful, the following information is displayed, in which test_stm767 is the MCU project name in this example.
```text
arm-none-eabi-size build/test_stm767.elf
text data bss dec hex filename
120316 3620 87885 211821 33b6d build/test_stm767.elf
arm-none-eabi-objcopy -O ihex build/test_stm767.elf build/test_stm767.hex
arm-none-eabi-objcopy -O binary -S build/test_stm767.elf build/test_stm767.bin
```
- Burn and run
The `STMSTM32CubePrg` tool can be used to burn and run the code. On the PC, use `STLink` to connect to a development board that can be burnt. Then, run the following commands in the current MCU project directory to burn and run the program:
```bash
bash ${STMSTM32CubePrg_PATH}/bin/STM32_Programmer.sh -c port=SWD -w build/test_stm767.bin 0x08000000 -s 0x08000000
```
${STMSTM32CubePrg_PATH is}: installation path of `STMSTM32CubePrg`. For details about the parameters in the command, see the `STMSTM32CubePrg` user manual.
#### Inference Result Verification
In this example, the benchmark running result flag is stored in the memory segment whose start address is `0x20000000` and whose size is 1 byte. Therefore, you can directly obtain the data at this address by using the burner to obtain the result returned by the program.
On the PC, use `STLink` to connect to a development board where programs have been burnt. Run the following command to read the memory data:
```bash
bash ${STMSTM32CubePrg_PATH is }/bin/STM32_Programmer.sh -c port=SWD model=HOTPLUG --upload 0x20000000 0x1 ret.bin
```
${STMSTM32CubePrg_PATH is}: installation path of `STMSTM32CubePrg`. For details about the parameters in the command, see the `STMSTM32CubePrg` user manual.
The read data is saved in the `ret.bin` file and run `cat ret.bin`. If the board inference is successful and `ret.bin` stores `1`, the following information is displayed:
```text
1
```
## Executing Inference on Light Harmony Devices
### Preparing the Light Harmony Compilation Environment
You can learn how to compile and burn in the light Harmony environment at the [OpenHarmony official website](https://www.openharmony.cn).
This tutorial uses the Hi3516 development board as an example to demonstrate how to use Micro to deploy the inference model in the light Harmony environment.
### Compiling Models
Use converter_lite to compile the [lenet model](https://download.mindspore.cn/model_zoo/official/lite/quick_start/micro/mnist.tar.gz) and generate the inference code corresponding to the light Harmony platform. The command is as follows:
```shell
./converter_lite --fmk=TFLITE --modelFile=mnist.tflite --outputFile=${SOURCE_CODE_DIR} --configFile=${COFIG_FILE}
```
Set target to ARM32 in the config configuration file.
### Compiling the Build Script
For details about how to develop light Harmony applications, see [Running Hello OHOS](https://device.harmonyos.com/cn/docs/start/introduce/quickstart-lite-steps-board3516-running-0000001151888681). Copy the mnist directory generated in the previous step to any Harmony source code path, such as, applications/sample/, and create the Build.gn file.
```text
<harmony-source-path>/applications/sample/mnist
├── benchmark
├── CMakeLists.txt
├── BUILD.gn
└── src
```
Download the [precompiled inference runtime package](https://www.mindspore.cn/lite/docs/en/r1.10/use/downloads.html) for OpenHarmony and decompress it to any Harmony source code path. Compile Build.gn file:
```text
import("//build/lite/config/component/lite_component.gni")
import("//build/lite/ndk/ndk.gni")
lite_component("mnist_benchmark") {
target_type = "executable"
sources = [
"benchmark/benchmark.cc",
"benchmark/calib_output.cc",
"benchmark/load_input.c",
"src/net.c",
"src/weight.c",
"src/session.cc",
"src/tensor.cc",
]
features = []
include_dirs = [
"<YOUR MINDSPORE LITE RUNTIME PATH>/runtime",
"<YOUR MINDSPORE LITE RUNTIME PATH>/tools/codegen/include",
"//applications/sample/mnist/benchmark",
"//applications/sample/mnist/src",
]
ldflags = [
"-fno-strict-aliasing",
"-Wall",
"-pedantic",
"-std=gnu99",
]
libs = [
"<YOUR MINDSPORE LITE RUNTIME PATH>/runtime/lib/libmindspore-lite.a",
"<YOUR MINDSPORE LITE RUNTIME PATH>/tools/codegen/lib/libwrapper.a",
]
defines = [
"NOT_USE_STL",
"ENABLE_NEON",
"ENABLE_ARM",
"ENABLE_ARM32"
]
cflags = [
"-fno-strict-aliasing",
"-Wall",
"-pedantic",
"-std=gnu99",
]
cflags_cc = [
"-fno-strict-aliasing",
"-Wall",
"-pedantic",
"-std=c++17",
]
}
```
`<YOUR MINDSPORE LITE RUNTIME PATH>` is the path of the decompressed inference runtime package, such as //applications/sample/mnist/mindspore-lite-1.3.0-ohos-aarch32.
Modify the build/lite/components/applications.json file and add the mnist_benchmark configuration:
```text
{
"component": "mnist_benchmark",
"description": "Communication related samples.",
"optional": "true",
"dirs": [
"applications/sample/mnist"
],
"targets": [
"//applications/sample/mnist:mnist_benchmark"
],
"rom": "",
"ram": "",
"output": [],
"adapted_kernel": [ "liteos_a" ],
"features": [],
"deps": {
"components": [],
"third_party": []
}
},
```
Modify vendor/hisilicon/hispark_taurus/config.json file and add mnist_benchmark component.
```text
{ "component": "mnist_benchmark", "features":[] }
```
### Compiling benchmark
```text
cd <openharmony-source-path>
hb set(Set the compilation path)
.(Select Current Path)
Select ipcamera_hispark_taurus@hisilicon and press Enter.
hb build mnist_benchmark (Perform compilation)
```
Generate the result file out/hispark_taurus/ipcamera_hispark_taurus/bin/mnist_benchmark.
### Performing benchmark
Decompress mnist_benchmark, weight file (mnist/src/net.bin), and [input file](https://download.mindspore.cn/model_zoo/official/lite/quick_start/micro/mnist.tar.gz), copy them to the development board, and run the following commands:
```text
OHOS # ./mnist_benchmark mnist_input.bin net.bin 1
OHOS # =======run benchmark======
input 0: mnist_input.bin
loop count: 1
total time: 10.11800ms, per time: 10.11800ms
outputs:
name: int8toft32_Softmax-7_post0/output-0, DataType: 43, Elements: 10, Shape: [1 10 ], Data:
0.000000, 0.000000, 0.003906, 0.000000, 0.000000, 0.992188, 0.000000, 0.000000, 0.000000, 0.000000,
========run success=======
```