Train with MindSpore

Filter condition Clear All Conditions

Operating System

Windows Linux

Hardware

Ascend GPU CPU

User

Beginner Intermediate Expert Enterprise

Stage

Whole Process Data preparation Model Development Model Training Model Optimization Model Export Model Loading Inference Application Model Publishing Model Evaluation

---

Implementing an Image Classification Application

This tutorial uses a simple image classification example to demonstrate the basic functions of MindSpore.

Implementing Simple Linear Function Fitting

Regression algorithms usually use a series of properties to predict a value, and the predicted values are consecutive. This example describes the linear regression algorithms and how to use MindSpore to perform linear regression AI training.

Loading Image Dataset

This tutorial uses the MNIST dataset as an example to demonstrate how to load and process image data using MindSpore.

Loading Text Dataset

This tutorial briefly demonstrates how to load and process text data using MindSpore.

Defining the Network

A neural network model composed of multiple layers is an important part of the training process. This tutorial introduces several ways to define the network model.

Saving Models

This tutorial describes how to save MindSpore CheckPoint files, and how to export MindIR, AIR, ONNX files.

Loading a Model for Inference and Transfer Learning

This tutorial uses examples to describe how to load models from local and MindSpore Hub.

Publishing Models

This tutorial uses GoogleNet as an example to describe how to submit models for model developers who are interested in publishing models into MindSpore Hub.

Converting Dataset to MindRecord

Users can convert non-standard datasets and common datasets into the MindSpore data format, MindRecord, so that they can be easily loaded to MindSpore for training. In addition, the performance of MindSpore in some scenarios is optimized, which delivers better user experience when you use datasets in the MindSpore data format.

Optimize the Data Processing

MindSpore provides data processing and data augmentation functions for users. In the pipeline process, if each step can be properly used, the data performance will be greatly improved. This section describes how to optimize performance during data loading, data processing, and data augmentation based on the CIFAR-10 dataset.

Custom Operators (Ascend)

When built-in operators cannot meet requirements during network development, you can call the Python API of MindSpore to quickly extend custom operators of the Ascend AI processor.

Custom Operators (GPU)

When built-in operators cannot meet requirements during network development, you can utilize MindSpore to quickly extend custom operators of the Graphics Processing Unit.

Custom Operators (CPU)

When the built-in operators are not enough for developing the network, you can extend your custom CPU operators fast and conveniently using MindSpore’s Python API and C++ API.

Migrating from Third Party Frameworks with Tools

MindConverter is a migration tool to transform the model scripts from PyTorch to Mindspore. Users can migrate their PyTorch models to Mindspore rapidly with minor changes according to the conversion report.

Migrating Training Scripts from Third Party Frameworks

This tutorial describes how to migrate existing TensorFlow and PyTorch networks to MindSpore, including key steps and operation recommendations which help you quickly migrate your network.

Deep Probabilistic Programming

MindSpore Deep Probabilistic Programming (MDP) combines deep learning and Bayesian learning. By setting a network weight to distribution and introducing latent space distribution, MDP can sample the distribution and forward propagation, which introduces uncertainty and enhances the robustness and explainability of a model. This chapter will introduce in detail the application of deep probabilistic programming on MindSpore.

Implementing High-order Automatic Differentiation

This tutorial introduces the higher-order derivatives in MindSpore graph mode.

Debugging in PyNative Mode

In PyNative mode, single operators, common functions, network inference, and separated gradient calculation can be executed. This tutorial describes the usage and precautions in detail.

Using Dump in the Graph Mode

The input and output of the operator can be saved for debugging through the data dump when the training result deviates from the expectation.

Customized Debugging Information

This tutorial describes how to use the customized capabilities provided by MindSpore, such as callback, metrics, and log printing, to help you quickly debug the training network.

Collecting Summary Data

Scalars, images, computational graphs, and model hyperparameters during training are recorded in files and can be viewed on the web page.

Viewing Dashboard

Training dashboard is an important part of mindinsight’s visualization component, and its tags include scalar visualization, parameter distribution visualization, computational graph visualization, data graph visualization, image visualization and tensor visualization.

Viewing Lineage and Scalars Comparison

Model lineage, data lineage and comparison Kanban in mindinsight are the same as training dashboard. In the visualization of training data, different scalar trend charts are observed by comparison dashboard to find problems, and then the lineage function is used to locate the problem causes, so as to give users the ability of efficient tuning in data enhancement and deep neural network.

Use Mindoptimizer to Tune Hyperparameters

Because different hyperparameters impact the performance of model, hyperparameters are highly important in training tasks. Traditional methods require manual analysis of hyperparameters, manual debugging, and configuration, which consumes time and effort. MindInsight parameter tuning command can be used for automatic parameter tuning.

Performance Profiling (Ascend)

Performance data like operators’ execution time is recorded in files and can be viewed on the web page, this can help the user optimize the performance of neural networks. This tutorial is applicable for the hardware platform of Ascend AI processors.

Performance Profiling (GPU)

Performance data like operators’ execution time is recorded in files and can be viewed on the web page, this can help the user optimize the performance of neural networks. This tutorial is applicable for the hardware platform of GPU.

Cluster Performance Profiling (Ascend)

This article describes how to use MindSpore Profiler for cluster performance debugging on Ascend AI processors.

Use Debugger

MindSpore Debugger is a debugging tool for training in `Graph Mode`. It can be applied to visualize and analyze the intermediate computation results of the computational graph.

Explain Models

Currently, most deep learning models are black-box models with good performance but poor explainability. The model explanation module aims to provide users with explanation of the model decision basis, help users better understand the model, trust the model, and improve the model when an error occurs in the model. This tutorial introduces how to use MindSpore to explain models.

MindInsight Commands

This tutorial introduces the MindInsight commands, including starting a service, stopping a service, exporting Summary, etc.

Data Augmentation

Auto Augmentation finds a suitable image augmentation scheme for a specific dataset by searching through a series of image augmentation sub-policies. This tutorial introduces how to apply auto augmentation to the ImageNet dataset.

Evaluating the Model during Training

This tutorial applies a neural network, LeNet5, to the MINIST dataset and introduces how to evaluate the model during training, save models in corresponding epochs, and select the optimal model from the saved ones.

Incremental Operator Build

When a network model is executed, MindSpore builds the used operators. The time consumed in this stage increases with the scale of the network model. To improve the performance of secondary model execution, an incremental operator build mechanism is provided. Currently, the incremental operator build function can be used only on the Ascend AI chips.

Parallel Distributed Training (Ascend)

This tutorial describes how to train the ResNet-50 network in data parallel and automatic parallel modes on MindSpore based on the Ascend 910 AI processor.

Parallel Distributed Training (GPU)

This tutorial describes how to train the ResNet-50 network in data parallel and automatic parallel modes on MindSpore based on GPU.

Adopt Host&Device for Hybrid Training

In Deep Learning, researchers have difficulty on training large models, because the requirement of model parameters may sometimes exceed the memory limitation of the equipment. For efficiently training large models, one possible option is to adopt a hybrid training method that combines Host and Device. This hybrid method takes advantage of both sides: large memory of Host and great computation capability of Device. This tutorial takes Wide&Deep as an example to introduce how to combine Host and Ascend 910 AI processor for hybrid training in MindSpore.

Training with Parameter Server

A parameter server is a widely used architecture in distributed training. Compared with the synchronous AllReduce training method, a parameter server has better flexibility, scalability, and node failover capabilities. This tutorial describes how to use parameter server to train LeNet on Ascend 910.

Saving and Loading Models in Hybrid Parallel Mode

In the hybrid parallel scenario, the dividing strategy is implemented by users. MindSpore saves the slice strategy of model, which is the same on each node, and the data corresponding to each node is stored respectively. Users need to integrate, save, and load the checkpoint files by themselves. This tutorial describes how to integrate, save, and load checkpoint files in the hybrid parallel scenario.

Enabling Mixed Precision

The mixed precision training method accelerates the deep learning neural network training process by using both the single-precision and half-precision data formats, and maintains the network precision achieved by the single-precision training at the same time. Mixed precision training can accelerate the computation process, reduce memory usage, and enable a larger model or batch size to be trained on specific hardware.

Enabling Graph Kernel Fusion

The graph kernel fusion is used to analyze and optimize the computational graph logic of the existing network, as well as split, reconstruct, and fuse the original computing logic to reduce the overhead of operator execution gaps and improve the computing resource utilization of devices, thereby optimizing the overall execution time of the network.

Applying Gradient Accumulation Algorithm

This tutorial describes the gradient accumulation training method to solve the problem that some large-scale networks cannot train large batch_size due to insufficient memory.

Application of Single-Node Tensor Cache

This tutorial demonstrates how to use the single-node cache service to cache data that has been processed with data augmentation.

Applying Quantization Aware Training

MindSpore quantization aware training is to replace high-accuracy data with low-accuracy data to simplify the model training process. In this process, the accuracy loss is inevitable. Therefore, a fake quantization node is used to simulate the accuracy loss, and backward propagation learning is used to reduce the accuracy loss.

Applying NAD algorithm to Improve the Security of a Model

This tutorial introduces security measures provided by MindArmour, leading you to a quick start of MindArmour. MindArmour is able to provide security for your AI models.

Protecting User Privacy with Differential Privacy Mechanism

Machine learning algorithms usually update model parameters and learn data features based on a large amount of data. Ideally, these models can learn the common features of a class of entities and achieve good generalization, such as “smoking patients are more likely to get lung cancer” rather than models with individual features, such as “Zhang San is a smoker who gets lung cancer.” However, machine learning algorithms do not distinguish between general and individual features. The published machine learning models, especially the deep neural networks, may unintentionally memorize and expose the features of individual entities in training data. This can be exploited by malicious attackers to reveal Zhang San’s privacy information from the published model. Therefore, it is necessary to use differential privacy to protect machine learning models from privacy leakage.

Applying the fuzz_testing Module to Test Security of a Model

The fuzz_testing module in MindArmour treats neuron coverage as the test criterion. We use neuron coverage to guide input mutation, so that the input can activate more neurons and the distribution of neuron values is wider, so as to explore different types of model output results and wrong behaviors.

Using Membership Inference to Test Model Security

In machine learning and deep learning, if an attacker has some access permissions (black box, gray box, or white box) of a model to obtain some or all information about the model output, structure, or parameters, they can determine whether a sample belongs to a training set of a model. In this case, we can use membership inference to evaluate the privacy data security of machine learning and deep learning models. If more than 60% samples can be correctly inferred using membership inference, the model has privacy data leakage risks.

Image Classification Using ResNet-50 Network

Computer vision is one of the most widely researched and mature technology fields of deep learning, and is widely applied to scenarios such as mobile phone photographing, intelligent security protection, and automated driving. This tutorial describes how to apply MindSpore to computer vision scenarios based on image classification tasks.

ResNet-50 Second-Order Optimization Practice

This tutorial describes how to use the second-order optimizer THOR provided by MindSpore to train the ResNet-50 v1.5 network and ImageNet dataset on Ascend 910 and GPU.

Using MobileNetV2 to Implement Fine-Tuning

This tutorial describes how to perform fine-tuning training and validation in the MindSpore frameworks of different systems and processors.

Realizing Sentiment Classification with SentimentNet

Sentiment classification is a subset of text classification in NLP, and is one of the most basic applications of NLP. It is a process of analyzing and inferencing affective states and subjective information, that is, analyzing whether a person’s sentiment is positive or negative.

Using the BERT Network to Implement Intelligent Poem Writing

This tutorial introduces how to use MindSpore to train an intelligent poem writing model and deploy the prediction service.

Implementing Regional Ocean Model GOMO

This tutorial describes how to use MindSpore to build and run the 3D ocean model GOMO on GPU.

Use MindSpore on the Cloud

ModelArts is a one-stop AI development platform provided by HUAWEI CLOUD. It integrates the Ascend AI Processor resource pool. Developers can experience MindSpore on this platform. ResNet-50 is used as an example to describe how to use MindSpore to complete a training task on ModelArts.