Distributed Parallel

Distributed Parallelism Overview
Distributed Parallel Startup Methods
Data Parallel
Semi-automatic Parallel
Automatic Parallel
Manually Parallelism
Parameter Server
Model Saving and Loading
Fault Recovery
Optimization Techniques
Experimental Characteristics
Distributed High-Level Configuration Case

Custom Operator

Custom Operators (Custom-based)
MindSpore Hybrid Syntax Specification
Custom Operator Registration
Advanced Usage of aot-type Custom Operators

Performance Optimization

Profiling↗
Sinking Mode
Enabling Graph Kernel Fusion
Memory Reuse

Algorithm Optimization

Gradient Accumulation Algorithm
Second-order Optimization

High-level Functional Programming

Automatic Vectorization (Vmap)
Computing Jacobian and Hessian Matrices Using Functional Transformations
Per-sample-gradients

Data Processing

Auto Augmentation
Single-Node Data Cache
Optimizing the Data Processing

Model Inference

Inference Model Overview
Model Compression

Complex Problem Debugging

Using Dump in the Graph Mode
Ascend Optimization Engine (AOE)
Running Data Recorder
Fault Recovery
Feature Value Detection

MindSpore

»
Distributed High-Level Configuration Case
View page source

Distributed High-Level Configuration Case

PengCheng·PanGu Model Network Multi-dimension Hybrid Parallel Analysis
Multi-dimensional Hybrid Parallel Case Based on Double Recursive Search
Performing Distributed Training on K8S Clusters

Previous Next

© Copyright MindSpore.

Built with Sphinx using a theme provided by Read the Docs.