In-depth understanding of federated learning - the definition of federated learning

Category: General Catalog of "In-depth Understanding of Federated Learning"

---

Suppose there are two different enterprise $A$ and$B$ , they have different data. For example, enterprise$A$ has user characteristic data, while enterprise$B$ has product feature data and annotation data. According to the GDPR guidelines, the two companies cannot roughly combine the data of the two parties, because the original provider of the data, that is, their respective users, has no opportunity to agree to do so. Assuming that both parties establish a task model, each task can be classification or prediction, and these tasks have already been approved by their respective users when obtaining data. The question now is how$A$ and$B$ Each end builds a high-quality model. However, due to incomplete data (e.g. corporate$A$ lacks label data, enterprise$B$ lack of feature data), or insufficient data (the amount of data is not enough to build a good model), then the model at each end may not be established or the effect is not ideal. Federated learning is to solve this problem: each enterprise's own data does not go out of the local area, and the federated system can establish a virtual shared model through the parameter exchange method under the encryption mechanism, that is, without violating data privacy regulations. This virtual model is like the optimal model built by aggregating data together. But when building a virtual model, the data itself does not move, nor does it leak privacy or affect data compliance. In this way, the built models only serve local targets in their respective regions. Under such a federal mechanism, all participants have the same identity and status, and the federal system helps everyone establish a "common prosperity" strategy. That's why this system is called "federated learning".

The above examples illustrate the basic idea of ​​federated learning. The following will standardize the definition of federated learning, introduce the public value and commercial value of federated learning, and clarify the relationship between federated learning and existing research. In order to further accurately explain the idea of ​​federated learning, we define it as follows: In the process of machine learning, each participant can conduct joint modeling with the help of other parties' data. All parties do not need to share data resources, that is, when the data does not come out of the local area, data joint training is carried out to establish a shared machine learning model.

The constraints of the federated learning system are:
$$|{\text{V}}_{\text{FED}}-{\text{V}}_{\text{SUM}}|\le d$$

where ${\text{V}}_{\text{FED}}$is the effect of federated learning model, ${\text{V}}_{\text{SUM}}$For the traditional method (data aggregation method) model effect, $\delta$ is a bounded positive number.

References:
[1] Yang Qiang, Liu Yang, Cheng Yong, Kang Yan, Chen Tianjian, Yu Han. Federated Learning [M]. Electronic Industry Press, 2020 [2] WeBank, FedAI.
Federated Learning White Paper V2.0. Tencent Research Institute, etc., 2021