Performing Inference on MCU or Small Systems
Overview
This tutorial describes an ultra-lightweight AI deployment solution for IoT edge devices.
Compared with mobile devices, MCUs are usually used on IoT devices. The ROM resources of the device are limited, and the memory and computing power of the hardware resources are weak.
Therefore, AI applications on IoT devices have strict limits on runtime memory and power consumption of AI model inference.
For MCUs deploying hardware backends, MindSpore Lite provides the ultra-lightweight Micro AI deployment solution. In the offline phase, models are directly generated into lightweight code without online model parsing and graph compilation. The generated Micro inference code is easy to understand, with less memory at runtime and smaller code size.
You can use a MindSpore Lite conversion tool converter_lite
to easily generate inference code that can be deployed on the x86/ARM64/ARM32/Cortex-M platform.
Deploying a model for inference via the Micro involves the following four steps: model inference code generation, Micro
lib obtaining, code integration, and compilation and deployment.
Generating Model Inference Code
Overview
The Micro configuration item in the parameter configuration file is configured via the MindSpore Lite conversion tool convert_lite
.
This chapter describes the functions related to code generation in the conversion tool. For details about how to use the conversion tool, see Converting Models for Inference.
Preparing Environment
The following describes how to prepare the environment for using the conversion tool in the Linux environment.
System environment required for running the conversion tool
In this example, the Linux operating system is used. Ubuntu 18.04.02LTS is recommended.
Obtain the conversion tool
You can obtain the conversion tool in either of the following ways:
Download Release Version from the MindSpore official website.
Download the release package whose OS is Linux-x86_64 and hardware platform is CPU.
Start from the source code for Building MindSpore Lite.
Decompress the downloaded package.
tar -zxf mindspore-lite-${version}-linux-x64.tar.gz
${version} is the version number of the release package.
Add the dynamic link library required by the conversion tool to the environment variable LD_LIBRARY_PATH.
export LD_LIBRARY_PATH=${PACKAGE_ROOT_PATH}/tools/converter/lib:${LD_LIBRARY_PATH}
${PACKAGE_ROOT_PATH} is the path of the decompressed folder.
Generating Inference Code by Running converter_lite
Go to the conversion directory
cd ${PACKAGE_ROOT_PATH}/tools/converter/converter
Set the Micro configuration item
Create the micro.cfg file in the current directory. The file content is as follows:
[micro_param] # enable code-generation for MCU HW enable_micro=true # specify HW target, support x86,Cortex-M, AMR32A, ARM64 only. target=x86 # enable parallel inference or not. support_parallel=false
In the configuration file,
[micro_param]
in the first line indicates that the subsequent variable parameters belong to the micro configuration itemmicro_param
. These parameters are used to control code generation. Table 1 describes the parameters. In this example, we will generate inference code for Linux systems with the underlying architecture x86_64, so settarget=x86
to declare that the generated inference code will be used for Linux systems with the underlying architecture x86_64.Prepare the model to generate inference code
Click here to download the MNIST Handwritten Digit Recognition Model used in this example. After downloading, decompress the package to obtain
mnist.tflite
. This model is a trained MNIST classification model, that is, a TFLITE model. Copy themnist.tflite
model to the current conversion tool directory.Execute converter_lite and generate code
./converter_lite --fmk=TFLITE --modelFile=mnist.tflite --outputFile=mnist --configFile=micro.cfg
The following information is displayed when the code is run successfully:
CONVERTER RESULT SUCCESS:0
For details about the parameters related to converter_lite, see Converter Parameter Description.
After the conversion tool is successfully executed, the generated code is saved in the specified
outputFile
directory. In this example, the mnist folder is in the current conversion directory. The content is as follows:mnist # Specified name of generated code root directory ├── benchmark # Benchmark routines for integrated calls to model inference code │ ├── benchmark.c │ ├── calib_output.c │ ├── calib_output.h │ ├── load_input.c │ └── load_input.h ├── CMakeLists.txt # cmake project file of the benchmark routine └── src # Model inference code directory ├── CMakeLists.txt ├── net.bin # Model weights in binary form ├── net.c ├── net.cmake ├── net.h ├── model.c ├── context.c ├── context.h ├── tensor.c ├── tensor.h ├── weight.c └── weight.h
You can integrate code to implement the application by referring to the interface calling of the inference code in the benchmark routine. For details about the interface, see API Documentation.
Table 1: micro_param parameter definition
parameter |
Mandatory or not |
Parameter Description |
Range |
Default value |
---|---|---|---|---|
enable_micro |
Yes |
The model generates code, otherwise it generates .ms. |
true, false |
false |
target |
Yes |
Platform for which code is generated |
x86, Cortex-M, ARM32, ARM64 |
x86 |
support_parallel |
No |
Generate multi-threaded inference codes or not, which can be set to true only on x86/ARM32/ARM64 platforms |
true, false |
false |
Obtaining Micro
Lib
After generating model inference code, you need to obtain the Micro
lib on which the generated inference code depends before performing integrated development on the code.
The inference code of different platforms depends on the Micro
lib of the corresponding platform. You need to specify the platform via the micro configuration item target
based on the platform in use when generating code, and obtain the Micro
lib of the platform when obtaining the inference package.
You can download the Release Version of the corresponding platform from the MindSpore official website.
In chapter Generating Model Inference Code, we obtain the model inference code of the Linux platform with the x86_64 architecture. The Micro
lib on which the code depends is the release package used by the conversion tool.
In the release package, the following content depended by the inference code:
mindspore-lite-{version}-linux-x64
├── runtime
│ └── include
│ └── c_api # C API header file integrated with MindSpore Lite
└── tools
└── codegen # The source code generated by code depends on include and lib
├── include # Inference framework header file
│ ├── nnacl # nnacl operator header file
│ └── wrapper # wrapper operator header file
├── lib
│ ├── libwrapper.a # The MindSpore Lite codegen generates some operator static libraries on which the code depends
│ └── libnnacl.a # The MindSpore Lite codegen generates the nnacl operator static library on which the code depends
└── third_party
├── include
│ └── CMSIS # ARM CMSIS NN operator header file
└── lib
└── libcmsis_nn.a # ARM CMSIS NN operator static library
Code Integration and Compilation Deployment
Different platforms have differences in code integration and compilation deployment.
For the MCU of the cortex-M architecture, see Performing Inference on the MCU
For the Linux platform with the x86_64 architecture, see Compilation and Deployment on Linux_x86_64 Platform
For details about how to compile and deploy arm32 or arm64 on the Android platform, see Compilation and Deployment on Android Platform
For compilation and deployment on the OpenHarmony platform, see Executing Inference on Light Harmony Devices
Performing Inference on the MCU
Overview
This tutorial takes the deployment of the MNIST model on the STM32F767 chip as an example to demonstrate how to deploy the inference model on the MCU of the Cortex-M architecture, including the following steps:
Use the converter_lite conversion tool to generate model inference code that adapts to the Cortex-M architecture
Download the
Micro
lib corresponding to the Cortex-M architectureIntegrate and compile the obtained inference code and
Micro
lib, and deploy verificationOn the Windows platform, we demonstrate how to develop inference code through IAR, while on the Linux platform, we demonstrate how to develop inference code through MakeFile cross-compilation.
Generating MCU Inference Code
Generate inference code for the MCU. For details, see the chapter Generating Model Inference Code. You only need to change target=x86
in the Micro configuration item to target=Cortex-M
to generate inference code for the MCU.
After the code is generated, the contents of the folder are as follows:
mnist # Specified name of generated code root directory
├── benchmark # Benchmark routines for integrated calls to model inference code
│ ├── benchmark.c
│ ├── calib_output.c
│ ├── calib_output.h
│ ├── data.c
│ ├── data.h
│ ├── load_input.c
│ └── load_input.h
├── build.sh # One-click compilation script
├── CMakeLists.txt # cmake project file of the benchmark routine
├── cortex-m7.toolchain.cmake # Cross-compilation cmake file for cortex-m7
└── src # Model inference code directory
├── CMakeLists.txt
├── context.c
├── context.h
├── model.c
├── net.c
├── net.cmake
├── net.h
├── tensor.c
├── tensor.h
├── weight.c
└── weight.h
Downloading Micro
Lib of Cortex-M Architecture
The STM32F767 uses the Cortex-M7 architecture. You can obtain the Micro
lib of the architecture in either of the following ways:
Download Release Version from the MindSpore official website.
You need to download the release package whose OS is None and hardware platform is Cortex-M7.
Start from the source code for Building MindSpore Lite.
You can run the
MSLITE_MICRO_PLATFORM=cortex-m7 bash build.sh -I x86_64
command to compile the Cortex-M7 release package.
For other Cortex-M architecture platforms that do not provide release packages for download, you can modify MindSpore source code and manually compile the code to obtain the release package by referring to the method of compiling and building from source code.
Code Integration and Compilation Deployment on Windows: Integrated Development Through IAR
This example shows code integration and burning through IAR and demonstrates how to develop the generated inference code in Windows. The main steps are as follows:
Download the required software and prepare the integrated environment.
Generate the required MCU startup code and demonstration project by using the
STM32CubeMX
software.Integrate model inference code and
Micro
lib withinIAR
.Perform complilation and simulation run
Environment Preparation
STM32CubeMX Windows Version >= 6.0.1
STM32CubeMX
is a graphical configuration tool for STM32 chips provided bySTM
. This tool is used to generate the startup code and project of STM chips.
IAR EWARM >= 9.1
IAR EWARM
is an integrated development environment developed by IAR Systems for ARM microprocessors.
Obtaining the MCU Startup Code and Project
If you have an MCU project, skip this chapter.
This chapter uses the STM32F767 startup project as an example to describe how to generate an MCU project for an STM32 chip via STM32CubeMX
.
Start
STM32CubeMX
and selectNew Project
fromFile
to create a project.In the
MCU/MPU Selector
window, search for and selectSTM32F767IGT6
, and clickStart Project
to create a project for the chipOn the
Project Manager
page, configure the project name and the path of the generated project, and selectEWARM
inToolchain / IDE
to generate the IAR projectClick
GENERATE CODE
above to generate codeOn the PC where the
IAR
is installed, double-clickProject.eww
in theEWARM
directory of the generated project to open the IAR project.
Integrating Model Inference Code and Micro
Lib
Copy the generated inference code to the project, decompress the package obtained in Downloading
Micro
Lib of Cortex-M Architecture, and place it in the generated inference code directory, as shown in the following figure:test_stm32f767 # MCU project directory ├── Core │ ├── Inc │ └── Src │ ├── main.c │ └── ... ├── Drivers ├── EWARM # IAR project file directory └── mnist # Generated Code Root Directory ├── benchmark # Benchmark routines for integrated calls to model inference code │ ├── benchmark.c │ ├── data.c │ ├── data.h │ └── ... │── mindspore-lite-1.8.0-none-cortex-m7 # Downloaded Cortex-M7 Architecture `Micro` Lib ├── src # Model inference code directory └── ...
Import source files to the IAR project
Open the IAR project. On the
Workspace
page, right-click the project, chooseAdd > Add Group
and add amnist
group. Right-click the group and repeat the operations to create groupssrc
andbenchmark
. ChooseAdd -> Add Files
to import the source files in thesrc
andbenchmark
directories in themnist
folder to the groups.Add the dependent header file path and static library
On the
Workspace
page, right-click the project and chooseOptions
from the shortcut menu. SelectC/C++ Compiler
in the left pane of the project options window. In the right pane, selectPreprocessor
and add the path of the header file on which the inference code depends to the list. In this example, the path of the header file is as follows:$PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/runtime $PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/runtime/include $PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/include $PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/third_party/include/CMSIS/Core $PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/third_party/include/CMSIS/DSP $PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/third_party/include/CMSIS/NN $PROJ_DIR$/../mnist
In the left pane of the Project Options window, select
Linker
. In the right pane, selectLibrary
and add the operator static library file on which the inference code depends to the list. The static library file added in this example is as follows:$PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/lib/libwrapper.a $PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/lib/libnnacl.a $PROJ_DIR$/../mnist/mindspore-lite-1.8.0-none-cortex-m7/tools/codegen/third_party/lib/libcmsis_nn.a
Modify the main.c file and invoke the benchmark function
Add a header file reference at the beginning of
main.c
and invoke thebenchmark
function inbenchmark.c
in the main function. The program in the benchmark folder is a sample program that invokes the inference code in the generatedsrc
and compares the output, which can be modified freely.#include "benchmark/benchmark.h" ... int main(void) { ... if (benchmark() == 0) { printf("\nrun success.\n"); } else { printf("\nrun failed.\n"); } ... }
Modify the
mnist/benchmark/data.c
file to store benchmark input and output data in the program for comparisonIn the benchmark routine, the input data of the model is set and the inference result is compared with the expected result to obtain the error offset. In this example, set the input data of the model by modifying the
calib_input0_data
array ofdata.c
, and set the expected result by modifying thecalib_output0_data
.float calib_input0_data[NET_INPUT0_SIZE] = {0.54881352186203,0.7151893377304077,0.6027633547782898,0.5448831915855408,0.42365479469299316,0.6458941102027893,0.4375872015953064,0.891772985458374,0.9636627435684204,0.3834415078163147,0.7917250394821167,0.5288949012756348,0.5680445432662964,0.9255966544151306,0.07103605568408966,0.08712930232286453,0.020218396559357643,0.832619845867157,0.7781567573547363,0.8700121641159058,0.978618323802948,0.7991585731506348,0.4614793658256531,0.7805292010307312,0.11827442795038223,0.6399210095405579,0.14335328340530396,0.9446688890457153,0.5218483209609985,0.4146619439125061,0.26455560326576233,0.7742336988449097,0.4561503231525421,0.568433940410614,0.018789799883961678,0.6176354885101318,0.6120957136154175,0.6169340014457703,0.9437480568885803,0.681820273399353,0.35950788855552673,0.43703195452690125,0.6976311802864075,0.0602254718542099,0.6667667031288147,0.670637845993042,0.21038256585597992,0.12892629206180573,0.31542834639549255,0.36371076107025146,0.5701967477798462,0.4386015236377716,0.9883738160133362,0.10204481333494186,0.20887675881385803,0.16130951046943665,0.6531082987785339,0.25329160690307617,0.4663107693195343,0.24442559480667114,0.15896958112716675,0.11037514358758926,0.6563295722007751,0.13818295300006866,0.1965823620557785,0.3687251806259155,0.8209932446479797,0.09710127860307693,0.8379449248313904,0.0960984081029892,0.9764594435691833,0.4686512053012848,0.9767611026763916,0.6048455238342285,0.7392635941505432,0.03918779268860817,0.28280696272850037,0.12019655853509903,0.296140193939209,0.11872772127389908,0.3179831802845001,0.414262980222702,0.06414749473333359,0.6924721002578735,0.5666014552116394,0.26538950204849243,0.5232480764389038,0.09394051134586334,0.5759465098381042,0.9292961955070496,0.3185689449310303,0.6674103736877441,0.13179786503314972,0.7163271903991699,0.28940609097480774,0.18319135904312134,0.5865129232406616,0.02010754682123661,0.8289400339126587,0.004695476032793522,0.6778165102005005,0.2700079679489136,0.7351940274238586,0.9621885418891907,0.2487531453371048,0.5761573314666748,0.5920419096946716,0.5722519159317017,0.22308163344860077,0.9527490139007568,0.4471253752708435,0.8464086651802063,0.6994792819023132,0.2974369525909424,0.8137978315353394,0.396505743265152,0.8811032176017761,0.5812729001045227,0.8817353844642639,0.6925315856933594,0.7252542972564697,0.5013243556022644,0.9560836553573608,0.6439902186393738,0.4238550364971161,0.6063932180404663,0.019193198531866074,0.30157482624053955,0.6601735353469849,0.2900775969028473,0.6180154085159302,0.42876869440078735,0.1354740709066391,0.29828232526779175,0.5699648857116699,0.5908727645874023,0.5743252635002136,0.6532008051872253,0.6521032452583313,0.43141844868659973,0.8965466022491455,0.36756187677383423,0.4358649253845215,0.8919233679771423,0.806194007396698,0.7038885951042175,0.10022688657045364,0.9194825887680054,0.7142413258552551,0.9988470077514648,0.14944830536842346,0.8681260347366333,0.16249293088912964,0.6155595779418945,0.1238199844956398,0.8480082154273987,0.8073189854621887,0.5691007375717163,0.40718328952789307,0.06916699558496475,0.6974287629127502,0.45354267954826355,0.7220556139945984,0.8663823008537292,0.9755215048789978,0.855803370475769,0.011714084073901176,0.359978049993515,0.729990541934967,0.17162968218326569,0.5210366249084473,0.054337989538908005,0.19999653100967407,0.01852179504930973,0.793697714805603,0.2239246815443039,0.3453516662120819,0.9280812740325928,0.704414427280426,0.031838931143283844,0.1646941602230072,0.6214783787727356,0.5772286057472229,0.23789282143115997,0.9342139959335327,0.6139659285545349,0.5356327891349792,0.5899099707603455,0.7301220297813416,0.31194499135017395,0.39822107553482056,0.20984375476837158,0.18619300425052643,0.9443724155426025,0.739550769329071,0.49045881628990173,0.22741462290287018,0.2543564736843109,0.058029159903526306,0.43441662192344666,0.3117958903312683,0.6963434815406799,0.37775182723999023,0.1796036809682846,0.024678727611899376,0.06724963337182999,0.6793927550315857,0.4536968469619751,0.5365791916847229,0.8966712951660156,0.990338921546936,0.21689698100090027,0.6630781888961792,0.2633223831653595,0.02065099962055683,0.7583786249160767,0.32001715898513794,0.38346388936042786,0.5883170962333679,0.8310484290122986,0.6289818286895752,0.872650682926178,0.27354204654693604,0.7980468273162842,0.18563593924045563,0.9527916312217712,0.6874882578849792,0.21550767123699188,0.9473705887794495,0.7308558225631714,0.2539416551589966,0.21331197023391724,0.518200695514679,0.02566271834075451,0.20747007429599762,0.4246854782104492,0.3741699755191803,0.46357542276382446,0.27762871980667114,0.5867843627929688,0.8638556003570557,0.11753185838460922,0.517379105091095,0.13206811249256134,0.7168596982955933,0.39605969190597534,0.5654212832450867,0.1832798421382904,0.14484776556491852,0.4880562722682953,0.35561272501945496,0.9404319524765015,0.7653252482414246,0.748663604259491,0.9037197232246399,0.08342243731021881,0.5521924495697021,0.5844760537147522,0.961936354637146,0.29214751720428467,0.24082878232002258,0.10029394179582596,0.016429629176855087,0.9295293092727661,0.669916570186615,0.7851529121398926,0.28173011541366577,0.5864101648330688,0.06395526975393295,0.48562759160995483,0.9774951338768005,0.8765052556991577,0.3381589651107788,0.961570143699646,0.23170162737369537,0.9493188261985779,0.9413776993751526,0.799202561378479,0.6304479241371155,0.8742879629135132,0.2930202782154083,0.8489435315132141,0.6178767085075378,0.013236857950687408,0.34723350405693054,0.14814086258411407,0.9818294048309326,0.4783703088760376,0.49739137291908264,0.6394725441932678,0.36858460307121277,0.13690027594566345,0.8221177458763123,0.1898479163646698,0.5113189816474915,0.2243170291185379,0.09784448146820068,0.8621914982795715,0.9729194641113281,0.9608346819877625,0.9065554738044739,0.774047315120697,0.3331451416015625,0.08110138773918152,0.40724116563796997,0.2322341352701187,0.13248763978481293,0.053427182137966156,0.7255943417549133,0.011427458375692368,0.7705807685852051,0.14694663882255554,0.07952208071947098,0.08960303664207458,0.6720477938652039,0.24536721408367157,0.4205394685268402,0.557368814945221,0.8605511784553528,0.7270442843437195,0.2703278958797455,0.131482794880867,0.05537432059645653,0.3015986382961273,0.2621181607246399,0.45614057779312134,0.6832813620567322,0.6956254243850708,0.28351885080337524,0.3799269497394562,0.18115095794200897,0.7885454893112183,0.05684807524085045,0.6969972252845764,0.7786954045295715,0.7774075865745544,0.25942257046699524,0.3738131523132324,0.5875996351242065,0.27282190322875977,0.3708527982234955,0.19705428183078766,0.4598558843135834,0.044612299650907516,0.7997958660125732,0.07695644348859787,0.5188351273536682,0.3068101108074188,0.5775429606437683,0.9594333171844482,0.6455702185630798,0.03536243736743927,0.4304024279117584,0.5100168585777283,0.5361775159835815,0.6813924908638,0.2775960862636566,0.12886056303977966,0.3926756680011749,0.9564056992530823,0.1871308982372284,0.9039839506149292,0.5438059568405151,0.4569114148616791,0.8820413947105408,0.45860394835472107,0.7241676449775696,0.3990253210067749,0.9040443897247314,0.6900250315666199,0.6996220350265503,0.32772040367126465,0.7567786574363708,0.6360610723495483,0.2400202751159668,0.16053882241249084,0.796391487121582,0.9591665863990784,0.4581388235092163,0.5909841656684875,0.8577226400375366,0.45722344517707825,0.9518744945526123,0.5757511854171753,0.8207671046257019,0.9088436961174011,0.8155238032341003,0.15941447019577026,0.6288984417915344,0.39843425154685974,0.06271295249462128,0.4240322411060333,0.25868406891822815,0.849038302898407,0.03330462798476219,0.9589827060699463,0.35536885261535645,0.3567068874835968,0.01632850244641304,0.18523232638835907,0.40125951170921326,0.9292914271354675,0.0996149331331253,0.9453015327453613,0.869488537311554,0.4541623890399933,0.326700896024704,0.23274412751197815,0.6144647002220154,0.03307459130883217,0.015606064349412918,0.428795725107193,0.06807407736778259,0.2519409954547882,0.2211609184741974,0.253191202878952,0.13105523586273193,0.012036222964525223,0.11548429727554321,0.6184802651405334,0.9742562174797058,0.9903450012207031,0.40905410051345825,0.1629544198513031,0.6387617588043213,0.4903053343296051,0.9894098043441772,0.06530420482158661,0.7832344174385071,0.28839850425720215,0.24141861498355865,0.6625045537948608,0.24606318771839142,0.6658591032028198,0.5173085331916809,0.4240889847278595,0.5546877980232239,0.2870515286922455,0.7065746784210205,0.414856880903244,0.3605455458164215,0.8286569118499756,0.9249669313430786,0.04600730910897255,0.2326269894838333,0.34851935505867004,0.8149664998054504,0.9854914546012878,0.9689717292785645,0.904948353767395,0.2965562641620636,0.9920112490653992,0.24942004680633545,0.10590615123510361,0.9509525895118713,0.2334202527999878,0.6897682547569275,0.05835635960102081,0.7307090759277344,0.8817201852798462,0.27243688702583313,0.3790569007396698,0.3742961883544922,0.7487882375717163,0.2378072440624237,0.17185309529304504,0.4492916464805603,0.30446839332580566,0.8391891121864319,0.23774182796478271,0.5023894309997559,0.9425836205482483,0.6339976787567139,0.8672894239425659,0.940209686756134,0.7507648468017578,0.6995750665664673,0.9679655432701111,0.9944007992744446,0.4518216848373413,0.07086978107690811,0.29279401898384094,0.15235470235347748,0.41748636960983276,0.13128933310508728,0.6041178107261658,0.38280805945396423,0.8953858613967896,0.96779465675354,0.5468848943710327,0.2748235762119293,0.5922304391860962,0.8967611789703369,0.40673333406448364,0.5520782470703125,0.2716527581214905,0.4554441571235657,0.4017135500907898,0.24841345846652985,0.5058664083480835,0.31038081645965576,0.37303486466407776,0.5249704718589783,0.7505950331687927,0.3335074782371521,0.9241587519645691,0.8623185753822327,0.048690296709537506,0.2536425292491913,0.4461355209350586,0.10462789237499237,0.34847599267959595,0.7400975227355957,0.6805144548416138,0.6223844289779663,0.7105283737182617,0.20492368936538696,0.3416981101036072,0.676242470741272,0.879234790802002,0.5436780452728271,0.2826996445655823,0.030235258862376213,0.7103368043899536,0.007884103804826736,0.37267908453941345,0.5305371880531311,0.922111451625824,0.08949454873800278,0.40594232082366943,0.024313200265169144,0.3426109850406647,0.6222310662269592,0.2790679335594177,0.2097499519586563,0.11570323258638382,0.5771402716636658,0.6952700018882751,0.6719571352005005,0.9488610029220581,0.002703213831409812,0.6471966505050659,0.60039222240448,0.5887396335601807,0.9627703428268433,0.016871673986315727,0.6964824199676514,0.8136786222457886,0.5098071694374084,0.33396488428115845,0.7908401489257812,0.09724292904138565,0.44203564524650574,0.5199523568153381,0.6939564347267151,0.09088572859764099,0.2277594953775406,0.4103015661239624,0.6232946515083313,0.8869608044624329,0.618826150894165,0.13346147537231445,0.9805801510810852,0.8717857599258423,0.5027207732200623,0.9223479628562927,0.5413808226585388,0.9233060479164124,0.8298973441123962,0.968286395072937,0.919782817363739,0.03603381663560867,0.1747720092535019,0.3891346752643585,0.9521427154541016,0.300028920173645,0.16046763956546783,0.8863046765327454,0.4463944137096405,0.9078755974769592,0.16023047268390656,0.6611174941062927,0.4402637481689453,0.07648676633834839,0.6964631676673889,0.2473987489938736,0.03961552307009697,0.05994429811835289,0.06107853725552559,0.9077329635620117,0.7398838996887207,0.8980623483657837,0.6725823283195496,0.5289399027824402,0.30444636940956116,0.997962236404419,0.36218905448913574,0.47064894437789917,0.37824517488479614,0.979526937007904,0.1746583878993988,0.32798799872398376,0.6803486943244934,0.06320761889219284,0.60724937915802,0.47764649987220764,0.2839999794960022,0.2384132742881775,0.5145127177238464,0.36792758107185364,0.4565199017524719,0.3374773859977722,0.9704936742782593,0.13343943655490875,0.09680395573377609,0.3433917164802551,0.5910269021987915,0.6591764688491821,0.3972567617893219,0.9992780089378357,0.35189300775527954,0.7214066386222839,0.6375827193260193,0.8130538463592529,0.9762256741523743,0.8897936344146729,0.7645619511604309,0.6982485055923462,0.335498183965683,0.14768557250499725,0.06263600289821625,0.2419017106294632,0.432281494140625,0.521996259689331,0.7730835676193237,0.9587409496307373,0.1173204779624939,0.10700414329767227,0.5896947383880615,0.7453980445861816,0.848150372505188,0.9358320832252502,0.9834262132644653,0.39980170130729675,0.3803351819515228,0.14780867099761963,0.6849344372749329,0.6567619442939758,0.8620625734329224,0.09725799411535263,0.49777689576148987,0.5810819268226624,0.2415570467710495,0.16902540624141693,0.8595808148384094,0.05853492394089699,0.47062090039253235,0.11583399772644043,0.45705875754356384,0.9799623489379883,0.4237063527107239,0.857124924659729,0.11731556057929993,0.2712520658969879,0.40379273891448975,0.39981213212013245,0.6713835000991821,0.3447181284427643,0.713766872882843,0.6391869187355042,0.399161159992218,0.43176013231277466,0.614527702331543,0.0700421929359436,0.8224067091941833,0.65342116355896,0.7263424396514893,0.5369229912757874,0.11047711223363876,0.4050356149673462,0.40537357330322266,0.3210429847240448,0.029950324445962906,0.73725426197052,0.10978446155786514,0.6063081622123718,0.7032175064086914,0.6347863078117371,0.95914226770401,0.10329815745353699,0.8671671748161316,0.02919023483991623,0.534916877746582,0.4042436182498932,0.5241838693618774,0.36509987711906433,0.19056691229343414,0.01912289671599865,0.5181497931480408,0.8427768349647522,0.3732159435749054,0.2228638231754303,0.080532006919384,0.0853109210729599,0.22139644622802734,0.10001406073570251,0.26503971219062805,0.06614946573972702,0.06560486555099487,0.8562761545181274,0.1621202677488327,0.5596824288368225,0.7734555602073669,0.4564095735549927,0.15336887538433075,0.19959613680839539,0.43298420310020447,0.52823406457901,0.3494403064250946,0.7814795970916748,0.7510216236114502,0.9272118210792542,0.028952548280358315,0.8956912755966187,0.39256879687309265,0.8783724904060364,0.690784752368927,0.987348735332489,0.7592824697494507,0.3645446300506592,0.5010631680488586,0.37638914585113525,0.364911824464798,0.2609044909477234,0.49597030878067017,0.6817399263381958,0.27734026312828064,0.5243797898292542,0.117380291223526,0.1598452925682068,0.04680635407567024,0.9707314372062683,0.0038603513967245817,0.17857997119426727,0.6128667593002319,0.08136960119009018,0.8818964958190918,0.7196201682090759,0.9663899540901184,0.5076355338096619,0.3004036843776703,0.549500584602356,0.9308187365531921,0.5207614302635193,0.2672070264816284,0.8773987889289856,0.3719187378883362,0.0013833499979227781,0.2476850152015686,0.31823351979255676,0.8587774634361267,0.4585031569004059,0.4445872902870178,0.33610227704048157,0.880678117275238,0.9450267553329468,0.9918903112411499,0.3767412602901459,0.9661474227905273,0.7918795943260193,0.675689160823822,0.24488948285579681,0.21645726263523102,0.1660478264093399,0.9227566123008728,0.2940766513347626,0.4530942440032959,0.49395784735679626,0.7781715989112854,0.8442349433898926,0.1390727013349533,0.4269043505191803,0.842854917049408,0.8180332779884338}; float calib_output0_data[NET_OUTPUT0_SIZE] = {3.5647096e-05,6.824297e-08,0.009327697,3.2340475e-05,1.1117579e-05,1.5117058e-06,4.6314454e-07,5.161628e-11,0.9905911,3.8835238e-10};
Compiling and Simulation Run
In this example, software simulation is used to view and analyze the inference result.
On the Workspace
page, right-click the project and choose Options
from the shortcut menu. Select the Debugger
option on the left of the Project Options window. In the Setup
on the right, set Driver
as Simulator
to enable software simulation.
Close the project option window and choose Project > Download and Debug
on the menu bar to compile and simulate the project. By adding a breakpoint at the benchmark
, you can observe the inference result of the simulation run and the return value of the benchmark() function.
Code Integration and Compilation Deployment on Linux: Code Integration via MakeFile
This chapter describes how to integrate and develop the MCU inference code on the Linux platform, by taking the generated model code integration and developing through MakeFile on the Linux platform as an example. The main steps are as follows:
Download the required software and prepare the cross compilation and burning environment
Generate the required MCU startup code and demonstration project using the
STM32CubeMX
softwareModify the inference code and
Micro
lib of theMakeFile
integrated modelCompiling and burning the project
Read and verify running result of the board
For the complete demo code built in this example, click Download here.
Environment Preparation
CMake >= 3.18.3
GNU Arm Embedded Toolchain >= 10-2020-q4-major-x86_64-linux
This tool is a cross compilation tool for Cortex-M Linux.
Download the
gcc-arm-none-eabi
package of the x86_64-Linux version, decompress the package, and add the bin path in the directory to the PATH environment variable:export PATH=gcc-arm-none-eabi path/bin:$PATH
.
STM32CubeMX-Lin >= 6.5.0
STM32CubeMX
is a graphical configuration tool for STM32 chips provided bySTM
, which is used to generate the startup code and project of STM chips.
STM32CubePrg-Lin >= 6.5.0
This tool is a burning tool provided by
STM
and can be used for program burning and data reading.
Obtaining the MCU Startup Code and Project
If you have an MCU project, skip this chapter.
This chapter uses the STM32F767 startup project as an example to describe how to generate an MCU project for an STM32 chip via STM32CubeMX
.
Start
STM32CubeMX
and selectNew Project
fromFile
to create a projectIn the
MCU/MPU Selector
window, search for and selectSTM32F767IGT6
, and clickStart Project
to create a project for the chipOn the
Project Manager
page, configure the project name and the path of the generated project, and selectEWARM
inToolchain / IDE
to generate the IAR projectClick
GENERATE CODE
above to generate codeOn the PC where the
IAR
is installed, double-clickProject.eww
in theEWARM
directory of the generated project to open the IAR project
Integrating Model Inference Code and Micro
Lib
Copy the generated inference code to the project, decompress the package obtained in Downloading
Micro
Lib of Cortex-M Architecture, and place it in the generated inference code directory, as shown in the following figure:stm32f767 # MCU project directory ├── Core │ ├── Inc │ └── Src │ ├── main.c │ └── ... ├── Drivers ├── mnist # Generate Code Root Directory │ ├── benchmark # Benchmark routines for integrated calls to model inference code │ │ ├── benchmark.c │ │ ├── data.c │ │ ├── data.h │ │ └── ... │ │── mindspore-lite-1.8.0-none-cortex-m7 # Downloaded Cortex-M7 Architecture `Micro` Lib │ ├── src # Model inference code directory │ └── ... ├── Makefile ├── startup_stm32f767xx.s └── STM32F767IGTx_FLASH.ld
Modify
MakeFile
and add the model inference code and dependency library to the projectIn this example, the source code to be added to the project includes the model inference code in the
src
directory and the example code called by the model inference in thebenchmark
directory. Modify the definition of theC_SOURCES
variable inMakeFile
and add the source file path.C_SOURCES = \ mnist/src/context.c \ mnist/src/model.c \ mnist/src/net.c \ mnist/src/tensor.c \ mnist/src/weight.c \ mnist/benchmark/benchmark.c \ mnist/benchmark/calib_output.c \ mnist/benchmark/load_input.c \ mnist/benchmark/data.c \ ...
Add the path of the dependent header file: Modify the definition of the
C_INCLUDES
variable inMakeFile
and add the following path:LITE_PACK = mindspore-lite-1.8.0-none-cortex-m7 C_INCLUDES = \ -Imnist/$(LITE_PACK)/runtime \ -Imnist/$(LITE_PACK)/runtime/include \ -Imnist/$(LITE_PACK)/tools/codegen/include \ -Imnist/$(LITE_PACK)/tools/codegen/third_party/include/CMSIS/Core \ -Imnist/$(LITE_PACK)/tools/codegen/third_party/include/CMSIS/DSP \ -Imnist/$(LITE_PACK)/tools/codegen/third_party/include/CMSIS/NN \ -Imnist \ ...
Add the dependent operator library (
-lnnacl -lwrapper -lcmsis_nn
), declare the path of the operator library file, and add the compilation option (-specs=nosys.specs
). In this example, the modified variables are defined as follows:LIBS = -lc -lm -lnosys -lnnacl -lwrapper -lcmsis_nn LIBDIR = -Lmnist/$(LITE_PACK)/tools/codegen/lib -Lmnist/$(LITE_PACK)/tools/codegen/third_party/lib LDFLAGS = $(MCU) -specs=nosys.specs -specs=nano.specs -T$(LDSCRIPT) $(LIBDIR) $(LIBS) -Wl,-Map=$(BUILD_DIR)/$(TARGET).map,--cref -Wl,--gc-sections
Modify the main.c file and invoke the benchmark function
Invoke the
benchmark
function inbenchmark.c
in the main function. The program in the benchmark folder is a sample program that invokes the inference code in the generatedsrc
and compares the output, which can be modified freely. In this example, we call thebenchmark
function directly and assign therun_dnn_flag
variable based on the returned result.run_dnn_flag = '0'; if (benchmark() == 0) { printf("\nrun success.\n"); run_dnn_flag = '1'; } else { printf("\nrun failed.\n"); run_dnn_flag = '2'; }
Add the header file reference and the definition of the
run_dnn_flag
variable to the beginning ofmain.c
.#include "benchmark/benchmark.h" char run_dnn_flag __attribute__((section(".myram"))) ;// Array for testing
In this example, to facilitate reading the inference result by using the burner, variables are defined in a customized section (
myram
). You can set the customized section in the following way or ignore the declaration: obtaining the inference result through serial ports or other interactive modes.To set a customized section, perform the following steps: Modify the
MEMORY
section in theSTM32F767IGTx_FLASH.ld
file, and add a customized memory segmentMYRAM
. (In this example, add 4 to theRAM
memory start address to free up memory forMYRAM
). Then add a customizedmyram
segment declaration to theSectionS
segment.MEMORY { MYRAM (xrw) : ORIGIN = 0x20000000, LENGTH = 1 RAM (xrw) : ORIGIN = 0x20000004, LENGTH = 524284 ... } ... SECTIONS { ... .myram (NOLOAD): { . = ALIGN(4); _smyram = .; /* create a global symbol at data start */ *(.sram) /* .data sections */ *(.sram*) /* .data* sections */ . = ALIGN(4); _emyram = .; /* define a global symbol at data end */ } >MYRAM AT> FLASH }
Modify the
mnist/benchmark/data.c
file to store benchmark input and output data in the program for comparison.In the benchmark routine, the input data of the model is set and the inference result is compared with the expected result to obtain the error offset. In this example, modify the
calib_input0_data
array ofdata.c
to set the input data of the model, and modify thecalib_output0_data
to set the expected result.float calib_input0_data[NET_INPUT0_SIZE] = {0.54881352186203,0.7151893377304077,0.6027633547782898,0.5448831915855408,0.42365479469299316,0.6458941102027893,0.4375872015953064,0.891772985458374,0.9636627435684204,0.3834415078163147,0.7917250394821167,0.5288949012756348,0.5680445432662964,0.9255966544151306,0.07103605568408966,0.08712930232286453,0.020218396559357643,0.832619845867157,0.7781567573547363,0.8700121641159058,0.978618323802948,0.7991585731506348,0.4614793658256531,0.7805292010307312,0.11827442795038223,0.6399210095405579,0.14335328340530396,0.9446688890457153,0.5218483209609985,0.4146619439125061,0.26455560326576233,0.7742336988449097,0.4561503231525421,0.568433940410614,0.018789799883961678,0.6176354885101318,0.6120957136154175,0.6169340014457703,0.9437480568885803,0.681820273399353,0.35950788855552673,0.43703195452690125,0.6976311802864075,0.0602254718542099,0.6667667031288147,0.670637845993042,0.21038256585597992,0.12892629206180573,0.31542834639549255,0.36371076107025146,0.5701967477798462,0.4386015236377716,0.9883738160133362,0.10204481333494186,0.20887675881385803,0.16130951046943665,0.6531082987785339,0.25329160690307617,0.4663107693195343,0.24442559480667114,0.15896958112716675,0.11037514358758926,0.6563295722007751,0.13818295300006866,0.1965823620557785,0.3687251806259155,0.8209932446479797,0.09710127860307693,0.8379449248313904,0.0960984081029892,0.9764594435691833,0.4686512053012848,0.9767611026763916,0.6048455238342285,0.7392635941505432,0.03918779268860817,0.28280696272850037,0.12019655853509903,0.296140193939209,0.11872772127389908,0.3179831802845001,0.414262980222702,0.06414749473333359,0.6924721002578735,0.5666014552116394,0.26538950204849243,0.5232480764389038,0.09394051134586334,0.5759465098381042,0.9292961955070496,0.3185689449310303,0.6674103736877441,0.13179786503314972,0.7163271903991699,0.28940609097480774,0.18319135904312134,0.5865129232406616,0.02010754682123661,0.8289400339126587,0.004695476032793522,0.6778165102005005,0.2700079679489136,0.7351940274238586,0.9621885418891907,0.2487531453371048,0.5761573314666748,0.5920419096946716,0.5722519159317017,0.22308163344860077,0.9527490139007568,0.4471253752708435,0.8464086651802063,0.6994792819023132,0.2974369525909424,0.8137978315353394,0.396505743265152,0.8811032176017761,0.5812729001045227,0.8817353844642639,0.6925315856933594,0.7252542972564697,0.5013243556022644,0.9560836553573608,0.6439902186393738,0.4238550364971161,0.6063932180404663,0.019193198531866074,0.30157482624053955,0.6601735353469849,0.2900775969028473,0.6180154085159302,0.42876869440078735,0.1354740709066391,0.29828232526779175,0.5699648857116699,0.5908727645874023,0.5743252635002136,0.6532008051872253,0.6521032452583313,0.43141844868659973,0.8965466022491455,0.36756187677383423,0.4358649253845215,0.8919233679771423,0.806194007396698,0.7038885951042175,0.10022688657045364,0.9194825887680054,0.7142413258552551,0.9988470077514648,0.14944830536842346,0.8681260347366333,0.16249293088912964,0.6155595779418945,0.1238199844956398,0.8480082154273987,0.8073189854621887,0.5691007375717163,0.40718328952789307,0.06916699558496475,0.6974287629127502,0.45354267954826355,0.7220556139945984,0.8663823008537292,0.9755215048789978,0.855803370475769,0.011714084073901176,0.359978049993515,0.729990541934967,0.17162968218326569,0.5210366249084473,0.054337989538908005,0.19999653100967407,0.01852179504930973,0.793697714805603,0.2239246815443039,0.3453516662120819,0.9280812740325928,0.704414427280426,0.031838931143283844,0.1646941602230072,0.6214783787727356,0.5772286057472229,0.23789282143115997,0.9342139959335327,0.6139659285545349,0.5356327891349792,0.5899099707603455,0.7301220297813416,0.31194499135017395,0.39822107553482056,0.20984375476837158,0.18619300425052643,0.9443724155426025,0.739550769329071,0.49045881628990173,0.22741462290287018,0.2543564736843109,0.058029159903526306,0.43441662192344666,0.3117958903312683,0.6963434815406799,0.37775182723999023,0.1796036809682846,0.024678727611899376,0.06724963337182999,0.6793927550315857,0.4536968469619751,0.5365791916847229,0.8966712951660156,0.990338921546936,0.21689698100090027,0.6630781888961792,0.2633223831653595,0.02065099962055683,0.7583786249160767,0.32001715898513794,0.38346388936042786,0.5883170962333679,0.8310484290122986,0.6289818286895752,0.872650682926178,0.27354204654693604,0.7980468273162842,0.18563593924045563,0.9527916312217712,0.6874882578849792,0.21550767123699188,0.9473705887794495,0.7308558225631714,0.2539416551589966,0.21331197023391724,0.518200695514679,0.02566271834075451,0.20747007429599762,0.4246854782104492,0.3741699755191803,0.46357542276382446,0.27762871980667114,0.5867843627929688,0.8638556003570557,0.11753185838460922,0.517379105091095,0.13206811249256134,0.7168596982955933,0.39605969190597534,0.5654212832450867,0.1832798421382904,0.14484776556491852,0.4880562722682953,0.35561272501945496,0.9404319524765015,0.7653252482414246,0.748663604259491,0.9037197232246399,0.08342243731021881,0.5521924495697021,0.5844760537147522,0.961936354637146,0.29214751720428467,0.24082878232002258,0.10029394179582596,0.016429629176855087,0.9295293092727661,0.669916570186615,0.7851529121398926,0.28173011541366577,0.5864101648330688,0.06395526975393295,0.48562759160995483,0.9774951338768005,0.8765052556991577,0.3381589651107788,0.961570143699646,0.23170162737369537,0.9493188261985779,0.9413776993751526,0.799202561378479,0.6304479241371155,0.8742879629135132,0.2930202782154083,0.8489435315132141,0.6178767085075378,0.013236857950687408,0.34723350405693054,0.14814086258411407,0.9818294048309326,0.4783703088760376,0.49739137291908264,0.6394725441932678,0.36858460307121277,0.13690027594566345,0.8221177458763123,0.1898479163646698,0.5113189816474915,0.2243170291185379,0.09784448146820068,0.8621914982795715,0.9729194641113281,0.9608346819877625,0.9065554738044739,0.774047315120697,0.3331451416015625,0.08110138773918152,0.40724116563796997,0.2322341352701187,0.13248763978481293,0.053427182137966156,0.7255943417549133,0.011427458375692368,0.7705807685852051,0.14694663882255554,0.07952208071947098,0.08960303664207458,0.6720477938652039,0.24536721408367157,0.4205394685268402,0.557368814945221,0.8605511784553528,0.7270442843437195,0.2703278958797455,0.131482794880867,0.05537432059645653,0.3015986382961273,0.2621181607246399,0.45614057779312134,0.6832813620567322,0.6956254243850708,0.28351885080337524,0.3799269497394562,0.18115095794200897,0.7885454893112183,0.05684807524085045,0.6969972252845764,0.7786954045295715,0.7774075865745544,0.25942257046699524,0.3738131523132324,0.5875996351242065,0.27282190322875977,0.3708527982234955,0.19705428183078766,0.4598558843135834,0.044612299650907516,0.7997958660125732,0.07695644348859787,0.5188351273536682,0.3068101108074188,0.5775429606437683,0.9594333171844482,0.6455702185630798,0.03536243736743927,0.4304024279117584,0.5100168585777283,0.5361775159835815,0.6813924908638,0.2775960862636566,0.12886056303977966,0.3926756680011749,0.9564056992530823,0.1871308982372284,0.9039839506149292,0.5438059568405151,0.4569114148616791,0.8820413947105408,0.45860394835472107,0.7241676449775696,0.3990253210067749,0.9040443897247314,0.6900250315666199,0.6996220350265503,0.32772040367126465,0.7567786574363708,0.6360610723495483,0.2400202751159668,0.16053882241249084,0.796391487121582,0.9591665863990784,0.4581388235092163,0.5909841656684875,0.8577226400375366,0.45722344517707825,0.9518744945526123,0.5757511854171753,0.8207671046257019,0.9088436961174011,0.8155238032341003,0.15941447019577026,0.6288984417915344,0.39843425154685974,0.06271295249462128,0.4240322411060333,0.25868406891822815,0.849038302898407,0.03330462798476219,0.9589827060699463,0.35536885261535645,0.3567068874835968,0.01632850244641304,0.18523232638835907,0.40125951170921326,0.9292914271354675,0.0996149331331253,0.9453015327453613,0.869488537311554,0.4541623890399933,0.326700896024704,0.23274412751197815,0.6144647002220154,0.03307459130883217,0.015606064349412918,0.428795725107193,0.06807407736778259,0.2519409954547882,0.2211609184741974,0.253191202878952,0.13105523586273193,0.012036222964525223,0.11548429727554321,0.6184802651405334,0.9742562174797058,0.9903450012207031,0.40905410051345825,0.1629544198513031,0.6387617588043213,0.4903053343296051,0.9894098043441772,0.06530420482158661,0.7832344174385071,0.28839850425720215,0.24141861498355865,0.6625045537948608,0.24606318771839142,0.6658591032028198,0.5173085331916809,0.4240889847278595,0.5546877980232239,0.2870515286922455,0.7065746784210205,0.414856880903244,0.3605455458164215,0.8286569118499756,0.9249669313430786,0.04600730910897255,0.2326269894838333,0.34851935505867004,0.8149664998054504,0.9854914546012878,0.9689717292785645,0.904948353767395,0.2965562641620636,0.9920112490653992,0.24942004680633545,0.10590615123510361,0.9509525895118713,0.2334202527999878,0.6897682547569275,0.05835635960102081,0.7307090759277344,0.8817201852798462,0.27243688702583313,0.3790569007396698,0.3742961883544922,0.7487882375717163,0.2378072440624237,0.17185309529304504,0.4492916464805603,0.30446839332580566,0.8391891121864319,0.23774182796478271,0.5023894309997559,0.9425836205482483,0.6339976787567139,0.8672894239425659,0.940209686756134,0.7507648468017578,0.6995750665664673,0.9679655432701111,0.9944007992744446,0.4518216848373413,0.07086978107690811,0.29279401898384094,0.15235470235347748,0.41748636960983276,0.13128933310508728,0.6041178107261658,0.38280805945396423,0.8953858613967896,0.96779465675354,0.5468848943710327,0.2748235762119293,0.5922304391860962,0.8967611789703369,0.40673333406448364,0.5520782470703125,0.2716527581214905,0.4554441571235657,0.4017135500907898,0.24841345846652985,0.5058664083480835,0.31038081645965576,0.37303486466407776,0.5249704718589783,0.7505950331687927,0.3335074782371521,0.9241587519645691,0.8623185753822327,0.048690296709537506,0.2536425292491913,0.4461355209350586,0.10462789237499237,0.34847599267959595,0.7400975227355957,0.6805144548416138,0.6223844289779663,0.7105283737182617,0.20492368936538696,0.3416981101036072,0.676242470741272,0.879234790802002,0.5436780452728271,0.2826996445655823,0.030235258862376213,0.7103368043899536,0.007884103804826736,0.37267908453941345,0.5305371880531311,0.922111451625824,0.08949454873800278,0.40594232082366943,0.024313200265169144,0.3426109850406647,0.6222310662269592,0.2790679335594177,0.2097499519586563,0.11570323258638382,0.5771402716636658,0.6952700018882751,0.6719571352005005,0.9488610029220581,0.002703213831409812,0.6471966505050659,0.60039222240448,0.5887396335601807,0.9627703428268433,0.016871673986315727,0.6964824199676514,0.8136786222457886,0.5098071694374084,0.33396488428115845,0.7908401489257812,0.09724292904138565,0.44203564524650574,0.5199523568153381,0.6939564347267151,0.09088572859764099,0.2277594953775406,0.4103015661239624,0.6232946515083313,0.8869608044624329,0.618826150894165,0.13346147537231445,0.9805801510810852,0.8717857599258423,0.5027207732200623,0.9223479628562927,0.5413808226585388,0.9233060479164124,0.8298973441123962,0.968286395072937,0.919782817363739,0.03603381663560867,0.1747720092535019,0.3891346752643585,0.9521427154541016,0.300028920173645,0.16046763956546783,0.8863046765327454,0.4463944137096405,0.9078755974769592,0.16023047268390656,0.6611174941062927,0.4402637481689453,0.07648676633834839,0.6964631676673889,0.2473987489938736,0.03961552307009697,0.05994429811835289,0.06107853725552559,0.9077329635620117,0.7398838996887207,0.8980623483657837,0.6725823283195496,0.5289399027824402,0.30444636940956116,0.997962236404419,0.36218905448913574,0.47064894437789917,0.37824517488479614,0.979526937007904,0.1746583878993988,0.32798799872398376,0.6803486943244934,0.06320761889219284,0.60724937915802,0.47764649987220764,0.2839999794960022,0.2384132742881775,0.5145127177238464,0.36792758107185364,0.4565199017524719,0.3374773859977722,0.9704936742782593,0.13343943655490875,0.09680395573377609,0.3433917164802551,0.5910269021987915,0.6591764688491821,0.3972567617893219,0.9992780089378357,0.35189300775527954,0.7214066386222839,0.6375827193260193,0.8130538463592529,0.9762256741523743,0.8897936344146729,0.7645619511604309,0.6982485055923462,0.335498183965683,0.14768557250499725,0.06263600289821625,0.2419017106294632,0.432281494140625,0.521996259689331,0.7730835676193237,0.9587409496307373,0.1173204779624939,0.10700414329767227,0.5896947383880615,0.7453980445861816,0.848150372505188,0.9358320832252502,0.9834262132644653,0.39980170130729675,0.3803351819515228,0.14780867099761963,0.6849344372749329,0.6567619442939758,0.8620625734329224,0.09725799411535263,0.49777689576148987,0.5810819268226624,0.2415570467710495,0.16902540624141693,0.8595808148384094,0.05853492394089699,0.47062090039253235,0.11583399772644043,0.45705875754356384,0.9799623489379883,0.4237063527107239,0.857124924659729,0.11731556057929993,0.2712520658969879,0.40379273891448975,0.39981213212013245,0.6713835000991821,0.3447181284427643,0.713766872882843,0.6391869187355042,0.399161159992218,0.43176013231277466,0.614527702331543,0.0700421929359436,0.8224067091941833,0.65342116355896,0.7263424396514893,0.5369229912757874,0.11047711223363876,0.4050356149673462,0.40537357330322266,0.3210429847240448,0.029950324445962906,0.73725426197052,0.10978446155786514,0.6063081622123718,0.7032175064086914,0.6347863078117371,0.95914226770401,0.10329815745353699,0.8671671748161316,0.02919023483991623,0.534916877746582,0.4042436182498932,0.5241838693618774,0.36509987711906433,0.19056691229343414,0.01912289671599865,0.5181497931480408,0.8427768349647522,0.3732159435749054,0.2228638231754303,0.080532006919384,0.0853109210729599,0.22139644622802734,0.10001406073570251,0.26503971219062805,0.06614946573972702,0.06560486555099487,0.8562761545181274,0.1621202677488327,0.5596824288368225,0.7734555602073669,0.4564095735549927,0.15336887538433075,0.19959613680839539,0.43298420310020447,0.52823406457901,0.3494403064250946,0.7814795970916748,0.7510216236114502,0.9272118210792542,0.028952548280358315,0.8956912755966187,0.39256879687309265,0.8783724904060364,0.690784752368927,0.987348735332489,0.7592824697494507,0.3645446300506592,0.5010631680488586,0.37638914585113525,0.364911824464798,0.2609044909477234,0.49597030878067017,0.6817399263381958,0.27734026312828064,0.5243797898292542,0.117380291223526,0.1598452925682068,0.04680635407567024,0.9707314372062683,0.0038603513967245817,0.17857997119426727,0.6128667593002319,0.08136960119009018,0.8818964958190918,0.7196201682090759,0.9663899540901184,0.5076355338096619,0.3004036843776703,0.549500584602356,0.9308187365531921,0.5207614302635193,0.2672070264816284,0.8773987889289856,0.3719187378883362,0.0013833499979227781,0.2476850152015686,0.31823351979255676,0.8587774634361267,0.4585031569004059,0.4445872902870178,0.33610227704048157,0.880678117275238,0.9450267553329468,0.9918903112411499,0.3767412602901459,0.9661474227905273,0.7918795943260193,0.675689160823822,0.24488948285579681,0.21645726263523102,0.1660478264093399,0.9227566123008728,0.2940766513347626,0.4530942440032959,0.49395784735679626,0.7781715989112854,0.8442349433898926,0.1390727013349533,0.4269043505191803,0.842854917049408,0.8180332779884338}; float calib_output0_data[NET_OUTPUT0_SIZE] = {3.5647096e-05,6.824297e-08,0.009327697,3.2340475e-05,1.1117579e-05,1.5117058e-06,4.6314454e-07,5.161628e-11,0.9905911,3.8835238e-10};
Compiling and Burning the Project
Compile
In the MCU project directory, run the
make
command to compile the MCU project. After the compilation is successful, the following information is displayed, in which test_stm767 is the MCU project name in this example.arm-none-eabi-size build/test_stm767.elf text data bss dec hex filename 120316 3620 87885 211821 33b6d build/test_stm767.elf arm-none-eabi-objcopy -O ihex build/test_stm767.elf build/test_stm767.hex arm-none-eabi-objcopy -O binary -S build/test_stm767.elf build/test_stm767.bin
Burn and run
The
STMSTM32CubePrg
tool can be used to burn and run the code. On the PC, useSTLink
to connect to a development board that can be burnt. Then, run the following commands in the current MCU project directory to burn and run the program:bash ${STMSTM32CubePrg_PATH}/bin/STM32_Programmer.sh -c port=SWD -w build/test_stm767.bin 0x08000000 -s 0x08000000
${STMSTM32CubePrg_PATH is}: installation path of
STMSTM32CubePrg
. For details about the parameters in the command, see theSTMSTM32CubePrg
user manual.
Inference Result Verification
In this example, the benchmark running result flag is stored in the memory segment whose start address is 0x20000000
and whose size is 1 byte. Therefore, you can directly obtain the data at this address by using the burner to obtain the result returned by the program.
On the PC, use STLink
to connect to a development board where programs have been burnt. Run the following command to read the memory data:
bash ${STMSTM32CubePrg_PATH is }/bin/STM32_Programmer.sh -c port=SWD model=HOTPLUG --upload 0x20000000 0x1 ret.bin
${STMSTM32CubePrg_PATH is}: installation path of STMSTM32CubePrg
. For details about the parameters in the command, see the STMSTM32CubePrg
user manual.
The read data is saved in the ret.bin
file and run cat ret.bin
. If the board inference is successful and ret.bin
stores 1
, the following information is displayed:
1
Executing Inference on Light Harmony Devices
Preparing the Light Harmony Compilation Environment
You can learn how to compile and burn in the light Harmony environment at the OpenHarmony official website. This tutorial uses the Hi3516 development board as an example to demonstrate how to use Micro to deploy the inference model in the light Harmony environment.
Compiling Models
Use converter_lite to compile the lenet model and generate the inference code corresponding to the light Harmony platform. The command is as follows:
./converter_lite --fmk=TFLITE --modelFile=mnist.tflite --outputFile=${SOURCE_CODE_DIR} --configFile=${COFIG_FILE}
Set target to ARM32 in the config configuration file.
Compiling the Build Script
For details about how to develop light Harmony applications, see Running Hello OHOS. Copy the mnist directory generated in the previous step to any Harmony source code path, such as, applications/sample/, and create the Build.gn file.
<harmony-source-path>/applications/sample/mnist
├── benchmark
├── CMakeLists.txt
├── BUILD.gn
└── src
Download the precompiled inference runtime package for OpenHarmony and decompress it to any Harmony source code path. Compile Build.gn file:
import("//build/lite/config/component/lite_component.gni")
import("//build/lite/ndk/ndk.gni")
lite_component("mnist_benchmark") {
target_type = "executable"
sources = [
"benchmark/benchmark.cc",
"benchmark/calib_output.cc",
"benchmark/load_input.c",
"src/net.c",
"src/weight.c",
"src/session.cc",
"src/tensor.cc",
]
features = []
include_dirs = [
"<YOUR MINDSPORE LITE RUNTIME PATH>/runtime",
"<YOUR MINDSPORE LITE RUNTIME PATH>/tools/codegen/include",
"//applications/sample/mnist/benchmark",
"//applications/sample/mnist/src",
]
ldflags = [
"-fno-strict-aliasing",
"-Wall",
"-pedantic",
"-std=gnu99",
]
libs = [
"<YOUR MINDSPORE LITE RUNTIME PATH>/runtime/lib/libmindspore-lite.a",
"<YOUR MINDSPORE LITE RUNTIME PATH>/tools/codegen/lib/libwrapper.a",
]
defines = [
"NOT_USE_STL",
"ENABLE_NEON",
"ENABLE_ARM",
"ENABLE_ARM32"
]
cflags = [
"-fno-strict-aliasing",
"-Wall",
"-pedantic",
"-std=gnu99",
]
cflags_cc = [
"-fno-strict-aliasing",
"-Wall",
"-pedantic",
"-std=c++17",
]
}
<YOUR MINDSPORE LITE RUNTIME PATH>
is the path of the decompressed inference runtime package, such as //applications/sample/mnist/mindspore-lite-1.3.0-ohos-aarch32.
Modify the build/lite/components/applications.json file and add the mnist_benchmark configuration:
{
"component": "mnist_benchmark",
"description": "Communication related samples.",
"optional": "true",
"dirs": [
"applications/sample/mnist"
],
"targets": [
"//applications/sample/mnist:mnist_benchmark"
],
"rom": "",
"ram": "",
"output": [],
"adapted_kernel": [ "liteos_a" ],
"features": [],
"deps": {
"components": [],
"third_party": []
}
},
Modify vendor/hisilicon/hispark_taurus/config.json file and add mnist_benchmark component.
{ "component": "mnist_benchmark", "features":[] }
Compiling benchmark
cd <openharmony-source-path>
hb set(Set the compilation path)
.(Select Current Path)
Select ipcamera_hispark_taurus@hisilicon and press Enter.
hb build mnist_benchmark (Perform compilation)
Generate the result file out/hispark_taurus/ipcamera_hispark_taurus/bin/mnist_benchmark.
Performing benchmark
Decompress mnist_benchmark, weight file (mnist/src/net.bin), and input file, copy them to the development board, and run the following commands:
OHOS # ./mnist_benchmark mnist_input.bin net.bin 1
OHOS # =======run benchmark======
input 0: mnist_input.bin
loop count: 1
total time: 10.11800ms, per time: 10.11800ms
outputs:
name: int8toft32_Softmax-7_post0/output-0, DataType: 43, Elements: 10, Shape: [1 10 ], Data:
0.000000, 0.000000, 0.003906, 0.000000, 0.000000, 0.992188, 0.000000, 0.000000, 0.000000, 0.000000,
========run success=======