MindSpore Federated Documents

MindSpore Federated is an open source federated learning tool for MindSpore, and it enables all-scenario intelligent applications when user data is stored locally.

Federated learning is a cryptographically distributed machine learning technique for solving data silos and performing efficient, secure and reliable machine learning across multiple parties or multiple resource computing nodes. Support the various participants of machine learning to build AI models together without directly sharing local data, including but not limited to mainstream deep learning models such as ad recommendation, classification, and detection, mainly applied in finance, medical, recommendation and other fields.

MindSpore Federated provides a horizontal federated model with sample federation and a vertical federation model with feature federation. Support commercial deployment for millions of stateless terminal devices, as well as cloud federated between data centers across trusted zones.

Code repository address: <https://gitee.com/mindspore/federated>

Advantages of the MindSpore Federated Horizontal Framework

Horizontal Federated Architecture:

1. Privacy Protection

   It supports accuracy-lossless security aggregation solution based on secure multi-party computation (MPC) to prevent model theft.

   It supports performance-lossless encryption based on local differential privacy to prevent private data leakage from models.

   It supports a gradient protection scheme based on Symbolic Dimensional Selection (SignDS), which prevents model privacy data leakage while reducing communication overhead by 99%.

2. Distributed Federated Aggregation

   The loosely coupled cluster processing mode on the cloud and distributed gradient quadratic aggregation paradigms support the deployment of tens of millions of heterogeneous devices, implements high-performance and high-availability federated aggregation computing, and can cope with network instability and sudden load changes.

3. Federated Learning Efficiency Improvement

   The adaptive frequency modulation strategy and gradient compression algorithm are supported to improve the federated learning efficiency and saving bandwidth resources.

   Multiple federated aggregation policies are supported to improve the smoothness of federated learning convergence and optimize both global and local accuracies.

4. Easy to Use

   Only one line of code is required to switch between the standalone training and federated learning modes.

   The network models, aggregation algorithms, and security algorithms are programmable, and the security level can be customized.

   It supports the effectiveness evaluation of federated training models and provides monitoring capabilities for federated tasks.

Advantages of the MindSpore Federated Vertical Framework

Vertical Federated Architecture:

1. Privacy Protection

   Support high-performance Privacy Set Intersection Protocol (PSI), which prevents federated participants from obtaining ID information outside the intersection and can cope with data imbalance scenarios.

   Support feature protection software solution that combines quantization and differential privacy, to prevent attackers from reconstructing original privacy data from intermediate features.

   Support feature protection hardware solution which is based on trusted execution environment, to provide high-strength and efficient feature protection capabilities.

   Support label protection solution which is based on differential privacy, to prevent the leakage of user label data.

2. Federated training

   Support multiple types of split learning network structures.

   Cross-domain training for large models with pipelined parallel optimization.

MindSpore Federated Working Process

1. Scenario Identification and Data Accumulation

   Identify scenarios where federated learning is used and accumulate local data for federated tasks on the client.

2. Model Selection and Framework Deployment

   Select or develop a model prototype and use a tool to generate a federated learning model that is easy to deploy.

3. Application Deployment

   Deploy the corresponding components to the business application and set up federated configuration tasks and deployment scripts on the server.

Common Application Scenarios

1. Image Classification

   Use the federated learning to implement image classification applications.

2. Text Classification

   Use the federated learning to implement text classification applications.

Deployment

• Obtaining MindSpore Federated
• Horizontal Federated Cloud-based Deployment
• Horizontal Federated Device-side Deployment
• Vertical Federated Deployment

Horizontal Application

• Federated Learning Image Classification Dataset Process
• Implementing an Image Classification Application of Cross-device Federated Learning (x86)
• Implementing a Sentiment Classification Application (Android)
• Implementing a Cloud-Slio Federated Image Classification Application (x86)
• Implementing a Cross-Silo Federated Target Detection Application (x86)

Vertical Application

• Vertical Federated Learning Data Access
• Vertical Federated Learning Model Training - Wide&Deep Recommendation Application
• Vertical Federated Learning Model Training - Pangu Alpha Large Model Cross-Domain Training

Security and Privacy

• Horizontal FL-Local Differential Privacy Perturbation Training
• Horizontal FL-Local Differential Privacy SignDS training
• Horizontal Federated-Local Differential Privacy Inference Result Protection
• Horizontal FL-Pairwise Encryption Training
• Vertical Federated-Privacy Set Intersection
• Vertical Federated-Feature Protection Based on Information Obfuscation
• Vertical Federated - Feature Protection Based on Trusted Execution Environment
• Vertical Federated - Label Protection Based on Differential Privacy

Communication Compression

• Device-Cloud Federated Learning Communication Compression
• Vertical Federated Learning Communication Compression

Horizontal Federated API Reference

• Federated Server
• Device-side Client
• Cross-Silo

Vertical Federated API Reference

• Data Join
• Vertical Federated Learning Communicator
• Vertical Federated Trainer

References

• FAQ

RELEASE NOTES

• Release Notes