Building Cloud-side MindSpore Lite

View Source On Gitee

This section describes how to quickly compile MindSpore Lite.

Cloud-side MindSpore Lite contains modules:

Modules

Supported Platforms

Description

converter

Linux

Model Converter

runtime(cpp, java)

Linux

Model Inference Framework

benchmark

Linux

Benchmarking Tool

minddata

Linux

Image Processing Library

Environment Requirements

  • System Environment: Linux x86_64 or arm64, Ubuntu 18.04.02LTS recommended

  • C++ compilation dependencies

  • Compilation dependency of the Java API module (optional), which is not compiled if the JAVA_HOME environment variable is not set.

    • Gradle >= 6.6.1

      • Configure environment variables: export GRADLE_HOME=GRADLE path, and export GRADLE_USER_HOME=GRADLE path

      • Add the bin directory to the PATH: export PATH=${GRADLE_HOME}/bin:$PATH

    • Maven >= 3.3.1

      • Configure environment variables: export MAVEN_HOME=MAVEN path

      • Add the bin directory to the PATH: export PATH=${MAVEN_HOME}/bin:$PATH

    • OpenJDK between 1.8 and 1.15

      • Configure environment variables: export JAVA_HOME=JDK path

      • Add the bin directory to the PATH: export PATH=${JAVA_HOME}/bin:$PATH

  • Compilation dependency for the Python API module (optional), which is not compiled if Python3 or NumPy is not installed.

    • Python >= 3.7.0

    • NumPy >= 1.17.0 (If installation with pip fails, please upgrade the pip version first: python -m pip install -U pip)

    • wheel >= 0.32.0 (If installation with pip fails, please upgrade the pip version first: python -m pip install -U pip)

Gradle recommends using gradle-6.6.1-complete, and configuring other versions of gradle will use the gradle wrapper mechanism to automatically download gradle-6.6.1-complete.

Compilation Options

The build.sh script in the MindSpore Lite root directory can be used to compile cloud-side MindSpore Lite.

Instructions for Using the Parameters of build.sh

Parameters

Description of the parameters

Range of values

Default values

-I

Select target architecture

arm64, x86_64

None

-d

Set this parameter to compile the Debug version, otherwise compile the Release version

None

None

-i

Set this parameter for incremental compilation, otherwise for full compilation

None

None

-j[n]

Set the number of threads used at compile time, otherwise the default setting is 8 threads

Integer

8

  • If the JAVA_HOME environment variable is configured and Gradle is installed, the JAR package is compiled at the same time.

  • Add the -i parameter for incremental compilation does not take effect when the -I parameter changes, e.g. -I x86_64 becomes -I arm64.

  • Cross-compilation is not supported, i.e. the arm64 version needs to be compiled in the arm environment.

Module Build Compilation Options

The building of modules is controlled through environment variables, and the users can control the build compilation modules by declaring the relevant environment variables. After modifying the compilation options, the -i parameter can not be added for incremental compilation when compiling with the build.sh script in order for the options to take effect.

General module compilation options:

Options

Description of the parameters

Range of values

Default values

MSLITE_ENABLE_TOOLS

Whether to compile the accompanying benchmarking tool

on, off

on

MSLITE_ENABLE_TESTCASES

Whether to compile test cases

on, off

off

MSLITE_ENABLE_ACL

Whether to enable Ascend ACL

on, off

off

MSLITE_ENABLE_CLOUD_INFERENCE

Whether to enable cloud-side inference

on, off

off

MSLITE_ENABLE_SSE

Whether to enable SSE instruction set, only valid for -I x86_64

on, off

off

MSLITE_ENABLE_AVX512

Whether to enable AVX512 instruction set, only valid for -I x86_64

on, off

off

  • The cloud-side inference version relies on the model converter, so when MSLITE_ENABLE_CLOUD_INFERENCE is configured to on, it will compile converter at the same time.

  • If the environment only supports the SSE instruction set, the AVX512 instruction set needs to be configured as off.

Compilation Examples

First, you need to download the source code from the MindSpore Lite code repository before compiling.

git clone https://gitee.com/mindspore/mindspore-lite.git

Environment Preparation

Ascend

  • Verify the installation of the Ascend AI processor package.

    • The Ascend package is available in both commercial and community versions.

      1. Commercial edition needs approval from Ascend to download, release date is TBD.

      2. There is no restriction on downloading the Community Edition. Please go to CANN Community Edition, select 8.2.RC1 version, and get the corresponding firmware and driver installation packages from the Firmware and Driver. For package selection and installation, please refer to the commercial version installation guide document above.

    • The default installation path for the installation package is /usr/local/Ascend. After installation, make sure the current user has permission to access the installation path of the Ascend AI processor companion package. If you don't have permission, the root user needs to add the current user to the user group where /usr/local/Ascend is located.

    • Install the whl package included with the Ascend AI processor companion software. If you have previously installed the package included with the Ascend AI processor, you need to uninstall the corresponding whl package first by using the following command.

    pip uninstall te topi -y
    

    The default installation path is installed via the following command. If the installation path is not the default path, you need to replace the path in the command with the installation path.

    pip install /usr/local/Ascend/ascend-toolkit/latest/lib64/topi-{version}-py3-none-any.whl
    pip install /usr/local/Ascend/ascend-toolkit/latest/lib64/te-{version}-py3-none-any.whl
    
  • Configure environment variables

    • After installing Ascend package, you need to export Runtime-related environment variables. /usr/local/Ascend of the /LOCAL_ASCEND=/usr/local/Ascend in the following command indicates the installation path of the package, so you need to change it to the actual installation path of the package.

      # control log level. 0-EBUG, 1-INFO, 2-WARNING, 3-ERROR, 4-CRITICAL, default level is WARNING.
      export GLOG_v=2
      
      # Conda environmental options
      LOCAL_ASCEND=/usr/local/Ascend # the root directory of run package
      
      # lib libraries that the run package depends on
      export LD_LIBRARY_PATH=${LOCAL_ASCEND}/ascend-toolkit/latest/lib64:${LOCAL_ASCEND}/driver/lib64:${LOCAL_ASCEND}/ascend-toolkit/latest/opp/op_impl/built-in/ai_core/tbe/op_tiling:${LD_LIBRARY_PATH}
      
      # Environment variables that must be configured
      export TBE_IMPL_PATH=${LOCAL_ASCEND}/ascend-toolkit/latest/opp/op_impl/built-in/ai_core/tbe            # TBE operator implementation tool path
      export ASCEND_OPP_PATH=${LOCAL_ASCEND}/ascend-toolkit/latest/opp                                       # OPP path
      export PATH=${LOCAL_ASCEND}/ascend-toolkit/latest/compiler/ccec_compiler/bin/:${PATH}                  # TBE operator compilation tool path
      export PYTHONPATH=${TBE_IMPL_PATH}:${PYTHONPATH}                                                       # Python library that TBE implementation depends on
      

CPU

Use x86_64 or ARM64 environment.

Installing LLVM-optional

The CPU backend of the graph kernel fusion in the converter needs to rely on LLVM-12. Run the following commands to install LLVM to enable CPU backend. If LLVM-12 is not installed, the graph kernel fusion can only support Ascend backend.

wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
sudo add-apt-repository "deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-12 main"
sudo apt-get update
sudo apt-get install llvm-12-dev -y

Executing Compilation

Double-backend-unification packages need to configure the following environment variables:

export MSLITE_ENABLE_CLOUD_INFERENCE=on
export MSLITE_ENABLE_ACL=on
  • If you don't need Ascend backend, you can configure export MSLITE_ENABLE_ACL=off.

Execute the following command in the source root directory to compile different versions of MindSpore Lite.

  • Compile the x86_64 architecture version while set the number of threads.

    bash build.sh -I x86_64 -j32
    
  • Compile the arm64 architecture version while set the number of threads.

    bash build.sh -I arm64 -j32
    

Finally, the following file will be generated in the output/ directory:

  • mindspore-lite-{version}-{os}-{arch}.tar.gz: contains runtime and companion tools.

  • mindspore-lite-{version}-{python}-{os}-{arch}.whl: contains the Whl package for runtime (Python).

  • version: The version number of the output, which corresponds to the version of the branch code compiled.

  • python: The output Python version, e.g. Python 3.7 for cp37-cp37m.

  • os: The operating system to which the output piece should be deployed

  • arch: The system architecture in which the output pieces should be deployed.

To experience the Python interface, you need to move to the output/ directory and use the following command to install the Whl installer.

pip install mindspore-lite-{version}-{python}-{os}-{arch}.whl

After installation, you can use the following command to check whether the installation is successful: if no error is reported, the installation is successful.

python -c "import mindspore_lite"

After successful installation, you can use the pip show mindspore_lite command to see where the Python modules for MindSpore Lite are installed.

Directory Structure

mindspore-lite-{version}-linux-{arch}
├── runtime
│   ├── include
│   ├── lib
│   │   ├── libascend_kernel_plugin.so # Ascend Kernel Plugin Dynamic Library
│   │   ├── libdvpp_utils.so           # DVPP Image Preprocessing Tools Dynamic Library
│   │   ├── libminddata-lite.a         # Image Processing Static Library
│   │   ├── libminddata-lite.so        # Image Processing Dynamic Library
│   │   ├── libmindspore-core.so       # MindSpore Core Dynamic Library
│   │   ├── libmindspore-glog.so.0     # glog Dynamic Library
│   │   ├── libmindspore-lite-jni.so   # jni dynamic library of MindSpore Lite inference framework
│   │   ├── libmindspore-lite.so       # MindSpore Lite Inference Framework Dynamic Library
│   │   ├── libmsplugin-ge-litert.so   # GE LiteRT Plugin Dynamic Library
│   │   └── mindspore-lite-java.jar    # MindSpore Lite Inference framework jar package
│   └── third_party
│       ├── glog
│       ├── libjpeg-turbo
│       └── securec
└── tools
    ├── benchmark              # Benchmarking Tools
    │   └── benchmark          # Benchmarking tool executable file
    └── converter              # Model converter
        ├── converter
        │   └── converter_lite # Converter executable file
        ├── include
        └── lib
            ├── libascend_pass_plugin.so
            ├── libmindspore_converter.so
            ├── libmindspore_core.so
            ├── libmindspore_glog.so.0
            ├── libmslite_shared_lib.so
            ├── libmslite_converter_plugin.so
            ├── libopencv_core.so.4.5
            ├── libopencv_imgcodecs.so.4.5
            └── libopencv_imgproc.so.4.5