0. Preface

1. Motivation for Federal Learning

(1) Protect user privacy and data security.

(2) Maximize the use of the computing power of terminal equipment under the cloud system.

2.FL mode

(1) B2C. Such as Google's Gboard system. It can also support edge computing. The terminal (edge) devices of the cloud system can handle many computing tasks, thereby reducing the need to communicate with the central server through raw data.

(2) B2B. Multiple organizations unite to build a shared machine learning model.

3. FL requires cooperation in multiple disciplines.

Machine learning algorithms, distributed machine learning, cryptography and security, privacy protection data mining, game theory and economics principles, incentive mechanism design, legal and regulatory requirements.

ch1, introduction

1 Overview

In FL, all encrypted models are aggregated into an encrypted global model, so the cloud server cannot learn the data or model of each device. The model aggregated in the cloud is still encrypted .

2.FL

When satisfied $V_ {SUM} -V_ {FED} <\ delta$ , the federated learning model $M_{FED}$ has a $\delta$ (non-negative real number) performance loss.

Among them, $V_ {SUM}$ and $V_{FED}$ are $M {sum}$ the performance measures of the centralized model and the federated model (such as accuracy, recall, F1 score, etc.).

Federated learning training end conditions : model convergence / reaching the maximum number of iterations / reaching the maximum training time.

3. The challenge of FL

(1) The communication link between the participants and the central server aggregation server may be slow and unstable.

(2) Data from different parties may lead to non-independent and identical distribution.

(3) Different participants may have an uneven number of training data samples, which may lead to model deviations, or even failure of federated learning model training.

(4) Since the participating parties are usually very geographically dispersed, it is difficult to be authenticated, which makes the federated learning model vulnerable to malicious attacks.

4. Classification of FL

Let the matrix $D_{i}$ represent $i$ the data of the first participant; let $D_{i}$ each row of the matrix represent a data sample, and each column represents a specific data feature.

The feature space is set $x$ , the data label (label) is set to a space $Y$ , the space is set to the sample ID $with$ , the three components a training data set ( $x$ , , ). $Y$ $with$

(1) Horizontal federal learning . The data of participants suitable for federated learning has overlapping data features (column overlap is much) , that is, the data features are aligned between the participants, but the data samples owned by the participants are different. Example: The participants are two banks serving different regional markets. Although they may have few overlapping customers, their customer data may have very similar feature spaces due to similar business models.

(2) Vertical federated learning . The training data applicable to federated learning participants has overlapping data samples (multiple rows overlap) , that is, the data samples between the participants are aligned, but the data characteristics are different. Example: A bank and an e-commerce company in the same area provide different services, but when there is a very large intersection in customer groups, they can collaborate on their different feature spaces to get a better machine learning for each model.

(3) Federal transfer learning . It is suitable for situations where the data samples and data features of the participants have little overlap .

(4) Recently, there are literatures that propose segmentation learning . It can be regarded as a special form of longitudinal federated learning. It uses a deep neural network (DNN) on top of longitudinal federated learning. In other words, segmentation learning mainly uses the setting of federated learning, and trains the DNN on the longitudinally divided data set.

[Federal Study] "Federal Study" reading notes (1)