【Federated Learning】【Federated Learning】Definition of federated learning and common federated learning framework--Introduction (1)

Definition of federated learning and common federated learning frameworks

1. Joint learning

1. What is federated learning

Federated learning is a distributed learning method that uses multiple devices or computers to process data models locally, and uploads some model parameters to the cloud for aggregation using encryption methods to build a global model. Different from traditional centralized machine learning, federated learning can enable each endpoint (device or computer) to share a small amount of information to efficiently train a global model while protecting data privacy. Federated learning is widely used in fields such as healthcare, smart cities, Internet of Things, etc., because these fields need to deal with privacy and security issues, and often have large amounts of decentralized data.

To put it simply, each customer uses his own data to train the model on his own device, and then transmits the model parameters to the server through the network. After the server obtains the model parameters of the client, it will calculate the global model parameters and transmit these parameters to each client, and the network model will be optimized in a cycle.

Image quoted from the Flower frame website

2. The role of joint learning

Federated learning has the following functions:

1. Protect data privacy: Federated learning allows model training without uploading sensitive data to the cloud, effectively protecting the data privacy of individuals or organizations.
2. Improve model performance: Joint learning enables the model to use multi-source heterogeneous data for training, improve model generalization and prediction performance, and reduce over-fitting and under-fitting problems.
3. Saving computing resources: Federated learning enables model training to be performed locally, requiring only a small amount of information transmission and cloud aggregation, thereby reducing computing resource overhead and network latency.
4. Improve data security: Federated learning can ensure the security of data transmission and processing through means such as encryption and security protocols, and prevent malicious attacks and data leakage.
5. Applicable to complex environments: Federated learning is suitable for training multi-source heterogeneous data, especially in complex environments such as healthcare, smart cities, and the Internet of Things, and can better meet the challenges of data processing and privacy protection.

3. Steps of joint learning

1. Train client-side models: Each client uses its own data to train a local model. Each client independently updates its own model without sharing any information with other clients.
2. Aggregate model parameters: After each client completes local model training, it needs to aggregate the information of all local models to obtain a global model. Methods for model aggregation include Simple Weighted Average (SWA), FedAvg, etc.
3. Model update: Send the global model back to each client and have each client update the global model with its own data. Note that each client does not receive the global model directly, but computes gradients based on the difference between its local and global models, and uses these gradients to update the local model. Through multiple rounds of iterations, the global model can be gradually optimized to improve the generalization performance of the model.

2. Common joint learning framework

1、TensorFlow Federated (TFF)

TFF is a federated learning framework developed by Google that focuses on distributed computing between mobile devices and servers. TFF allows machine learning developers to easily combine their deep learning models on multiple devices and protect the privacy and security of user data.

2、PySyft

PySyft is a Python library dedicated to supporting secure federated learning and privacy protection, which allows researchers and developers to use federated learning in various deep learning frameworks such as PyTorch, TensorFlow, and Keras. PySyft mainly provides important security technologies such as homomorphic encryption and differential privacy, which can ensure the security of federated learning data.

3、LEAF

LEAF is a federated learning framework that can simultaneously support multiple machine learning frameworks, such as TensorFlow, PyTorch, and Keras. It provides environment-based remote execution and local-based simulation environments that developers can leverage to debug their federated learning models.

4、Fate

FATE is an open source federated learning framework, jointly developed by Chinese Academy of Sciences and Harbin Institute of Technology and other universities and research institutions. It supports multiple federated learning scenarios such as vertical federated learning, horizontal federated learning, and federated overloaded learning, and can quickly establish a safe and efficient federated learning platform.

5、Flower

Flower is a lightweight Python library for fast federated learning. It allows the use of different types of machine learning frameworks, such as TensorFlow, PyTorch, and Keras, and provides some integrated optimizers that can greatly improve the convergence speed and accuracy of federated learning.

Next, an article on training a deep learning model based on the Flower framework will be updated.