Inference on the Ascend 310 AI Processor

Linux Ascend Inference Application Beginner Intermediate Expert

Overview

Ascend 310 is a highly efficient and integrated AI processor oriented to edge scenarios. The Atlas 200 Developer Kit (Atlas 200 DK) is a developer board that uses the Atlas 200 AI accelerator module. Integrated with the HiSilicon Ascend 310 AI processor, the Atlas 200 allows data analysis, inference, and computing for various data such as images and videos, and can be widely used in scenarios such as intelligent surveillance, robots, drones, and video servers.

This tutorial describes how to use MindSpore to perform inference on the Atlas 200 DK. The process is as follows:

Prepare the development environment, including creating an SD card for the Atlas 200 DK, configuring the Python environment, and updating the development software package.
Export the AIR model file. The ResNet-50 model is used as an example.
Use the ATC tool to convert the AIR model file into an OM model.
Build the inference code to generate an executable main file.
Load the saved OM model, perform inference, and view the result.

You can obtain the complete executable sample code at https://gitee.com/mindspore/docs/tree/r1.1/tutorials/tutorial_code/acl_resnet50_sample.

Preparing the Development Environment

Hardware Preparation

A server with the Ubuntu ARM OS is used to prepare a bootable SD card for the Atlas 200 DK and deploy the development environment.
An SD card with a capacity of at least 16 GB.

Software Package Preparation

The following five types of scripts and software packages are required for configuring the development environment:

Entry script for SD card preparation: make_sd_card.py
Script for preparing a bootable SD card: make_ubuntu_sd.sh
Ubuntu OS image package: ubuntu-18.04.xx-server-arm64.iso
Driver package and running package of Atlas 200 DK:
- Ascend310-driver-*{software version}*-ubuntu18.04.aarch64-minirc.tar.gz
- Ascend310-aicpu_kernels-*{software version}*-minirc.tar.gz
- Ascend-acllib-*{software version}*-ubuntu18.04.aarch64-minirc.run
Package for installing the development kit: Ascend-Toolkit-*{version}*-arm64-linux_gcc7.3.0.run

In the preceding information:

For details about the first three items, see Creating an SD Card with a Card Reader.
You are advised to obtain other software packages from Firmware and Driver. On this page, select Atlas 200 DK from the product series and product model and select the required files to download.

Preparing the SD Card

A card reader is connected to the Ubuntu server through a USB port, and the SD card is prepared using the script for SD card preparation. For details, see Procedure.

Connecting the Atlas 200 DK to the Ubuntu Server

The Atlas 200 DK can be connected to the Ubuntu server through a USB port or network cable. For details, see Connecting the Atlas 200 DK to the Ubuntu Server.

Configuring the Python Environment

Install Python and GCC software. For details, see Installing Dependencies.

Installing the Development Kit

Install the development kit software package Ascend-Toolkit-*{version}*-arm64-linux_gcc7.3.0.run. For details, see Installing the Development Kit.

Inference Directory Structure

Create a directory to store the inference code project, for example, /home/HwHiAiUser/Ascend/ascend-toolkit/20.0.RC1/acllib_linux.arm64/sample/acl_execute_model/acl_resnet50_sample. The inc, src, and test_data sample code can be obtained from the official website, and the model directory stores the exported AIR model file and the converted OM model file. The out directory stores the executable file generated after building and the output result directory. The directory structure of the inference code project is as follows:

└─acl_resnet50_sample
    ├── inc
    │   ├── model_process.h                   // Header file that declares functions related to resource initialization/destruction
    │   ├── sample_process.h                  // Header file that declares functions related to model processing
    │   ├── utils.h                           // Header file that declares common functions (such as the file reading function)
    ├── model
    │   ├── resnet50_export.air               // AIR model file
    │   ├── resnet50_export.om                // Converted OM model file
    ├── src
    │   ├── acl.json                          // Configuration file for system initialization
    │   ├── CMakeLists.txt                    // Build script
    │   ├── main.cpp                          // /Main function, which is the implementation file of image classification
    │   ├── model_process.cpp                 // Implementation file of model processing functions
    │   ├── sample_process.cpp                // Implementation file of functions related to resource initialization and destruction
    │   ├── utils.cpp                         // Implementation file of common functions (such as the file reading function)
    ├── test_data
    │   ├── test_data_1x3x224x224_1.bin       // Input sample data 1
    │   ├── test_data_1x3x224x224_2.bin       // input sample data 2
    ├── out
    │   ├── main                              // Executable file generated during building
    │   ├── result                            // Directory for storing the output result

The output result directory acl_resnet50_sample/out/result must be created before inference.

Exporting the AIR Model

Train the target network on the Ascend 910 AI Processor, save it as a checkpoint file, and export the model file in AIR format through the network and checkpoint file. For details about the export process, see Export AIR Model.

The resnet50_export.air is a sample AIR file exported using the ResNet-50 model.

Converting the AIR Model File into an OM Model

Log in to the Atlas 200 DK environment, create the model directory for storing the AIR file resnet50_export.air, for example, /home/HwHiAiUser/Ascend/ascend-toolkit/20.0.RC1/acllib_linux.arm64/sample/acl_execute_model/acl_resnet50_sample/model, go to the directory, and set the following environment variables where install_path specifies the actual installation path:

export install_path=/home/HwHiAiUser/Ascend/ascend-toolkit/20.0.RC1
export PATH=/usr/local/python3.7.5/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH
export PYTHONPATH=${install_path}/atc/python/site-packages/te:${install_path}/atc/python/site-packages/topi:$PYTHONPATH
export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH
export ASCEND_OPP_PATH=${install_path}/opp

Take resnet50_export.air as an example. Run the following command to convert the model and generate the resnet50_export.om file in the current directory.

/home/HwHiAiUser/Ascend/ascend-toolkit/20.0.RC1/atc/bin/atc --framework=1 --model=./resnet50_export.air --output=./resnet50_export --input_format=NCHW --soc_version=Ascend310

In the preceding information:

--model: path of the original model file
--output: path of the converted OM model file
--input_format: input image format

For detailed information about ATC tools, please select the corresponding CANN version in the Developer Documentation(Community Edition), and then search for the chapter of “ATC Tool Instructions”.

Building Inference Code

Go to the project directory acl_resnet50_sample and set the following environment variables:

export DDK_PATH=/home/HwHiAiUser/Ascend/ascend-toolkit/20.0.RC1
export NPU_HOST_LIB=/home/HwHiAiUser/Ascend/ascend-toolkit/20.0.RC1/acllib_linux.arm64/lib64/stub/

The include directory of the acllib package in the CMakeLists.txt file must be correctly specified. Otherwise, an error indicating that acl/acl.h cannot be found is reported. The code location of the include directory is as follows. If the location is inconsistent with the actual installation directory, modify it.

...
#Header path

 include_directories(

  ${INC_PATH}/acllib_linux.arm64/include/

  ../

 )
...

Run the following command to create a build directory:

mkdir -p build/intermediates/minirc

Run the following command to switch to the build directory:

cd build/intermediates/minirc

Run the cmake command:

cmake ../../../src -DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++ -DCMAKE_SKIP_RPATH=TRUE

Run the make command for building:

make

After building, the executable main file is generated in acl_resnet50_sample/out.

Performing Inference and Viewing the Result

Copy the generated OM model file resnet50_export.om to the acl_resnet50_sample/out directory (the same path as the executable main file) and ensure that the input data sample is ready in the acl_resnet50_sample/test_data directory. Then, you can perform inference.

Note that the following environment variables must be set. Otherwise, the inference fails.

export LD_LIBRARY_PATH=/home/HwHiAiUser/Ascend/acllib/lib64/

Go to the acl_resnet50_sample/out directory. If the result directory does not exist in the current directory, run the mkdir result command to create one and run the following command to perform inference:

./main  ./resnet50_export.om  ../test_data

After the execution is successful, the following inference result is displayed. The top5 probability label is displayed, and the output result is saved in the .bin file format in the acl_resnet50_sample/out/result directory.

[INFO]  acl init success
[INFO]  open device 0 success
[INFO]  create context success
[INFO]  create stream success
[INFO]  get run mode success
[INFO]  load model ./resnet50_export.om success
[INFO]  create model description success
[INFO]  create model output success
[INFO]  start to process file:../test_data/test_data_1x3x224x224_1.bin
[INFO]  model execute success
[INFO]  top 1: index[2] value[0.941406]
[INFO]  top 2: index[3] value[0.291992]
[INFO]  top 3: index[1] value[0.067139]
[INFO]  top 4: index[0] value[0.013519]
[INFO]  top 5: index[4] value[-0.226685]
[INFO]  output data success
[INFO]  dump data success
[INFO]  start to process file:../test_data/test_data_1x3x224x224_2.bin
[INFO]  model execute success
[INFO]  top 1: index[2] value[0.946289]
[INFO]  top 2: index[3] value[0.296143]
[INFO]  top 3: index[1] value[0.072083]
[INFO]  top 4: index[0] value[0.014549]
[INFO]  top 5: index[4] value[-0.225098]
[INFO]  output data success
[INFO]  dump data success
[INFO]  unload model success, modelId is 1
[INFO]  execute sample success
[INFO]  end to destroy stream
[INFO]  end to destroy context
[INFO]  end to reset device is 0
[INFO]  end to finalize acl