Model Analysis and Preparation

View Source On Gitee

Reproducing Algorithm Implementation

  1. Obtain the PyTorch reference code.

  2. Analyze the algorithm, network structure, and tricks in the original code, including the method of data augmentation, learning rate attenuation policy, optimizer parameters, and the initialization method of training parameters, etc.

  3. Reproduce the accuracy of the reference implementation, obtain the performance data of the reference implementation, and identify some issues in advance.

Please refer to Details of Reproducing Algorithm Implementation.

Analyzing API Compliance

Before practicing migration, it is recommended to analyze the API compliance in MindSpore's migration code to avoid affecting code implementation due to the lack of API support.

The API missing analysis here refers to APIs in the network execution diagram, including MindSpore operators and advanced encapsulated APIs, and excluding the APIs used in data processing. You are advised to use third-party APIs, such as NumPy, OpenCV, Pandas, and PIL, to replace APIs used in data processing.

There are two methods to analyze API compliance:

  1. Scanning API by MindSpore Dev Toolkit (recommended).

  2. Querying the API Mapping Table.

Scanning API by Toolkit

MindSpore Dev Toolkit is a development kit supporting PyCharm and Visual Studio Code plug-in developed by MindSpore, which can scan API based on file-level or project-level.

Refer to PyCharm API Scanning for the tutorials of Dev Toolkit in PyCharm.

api_scan_pycharm

Refer to Visual Studio Code API Scanning for the tutorials of Dev Toolkit in Visual Studio Code.

api_scan_pycharm

Querying the API Mapping Table

Take the PyTorch code migration as an example. After obtaining the reference code implementation, you can filter keywords such as torch, nn, and ops to obtain the used APIs. If the method of another repository is invoked, you need to manually analyze the API. Then, check the PyTorch and MindSpore API Mapping Table. Alternatively, the API searches for the corresponding API implementation.

For details about the mapping of other framework APIs, see the API naming and function description. For APIs with the same function, the names of MindSpore may be different from those of other frameworks. The parameters and functions of APIs with the same name may also be different from those of other frameworks. For details, see the official description.

Processing Missing API

You can use the following methods to process the missing API:

  1. Use equivalent replacement

  2. Use existing APIs to package equivalent function logic

  3. Customize operators

  4. Seek help from the community

Refer to Missing API Processing Policy for details.

Analyzing Function Compliance

During continuous delivery of MindSpore, some functions are restricted. If restricted functions are involved during network migration, before migration, functional compliance needs to be analyzed. It can be analyzed from the following points:

  1. Dynamic shape.

  2. Sparse.

Dynamic Shape

Currently MindSpore dynamic shape feature is under iterative development, and the dynamic shape functionality is not well supported.

During the migration process, if you encounter issues related to dynamic shape, you can refer to Strategies for Migrating Dynamic Shape.

Sparse

MindSpore now supports the two most commonly used sparse data formats, CSR and COO, but due to the limited support for sparse operators at the moment, most of the sparse features are still limited. In this case, it is recommended to find whether the corresponding operator supports sparse computation first, and if not it needs to be converted to a normal operator. For details, see Sparse.