A low-cost method to recover from attacks in federated learning - FedRecover: Background and related work, problem definition, FEDRECOVER, evaluation and study report of Recovering from Poisoning Attacks

II. Background and related work

A. Federated Learning (FL) Background

Composition of federated learning system:

• Existence $n$ clients, each with its local training dataset $D_i$。
• Use $D$ to represent the union of all clients’ local training data sets: $D={\bigcup }_{i=1}^n{D}_i$。

Joint title:

• The goal of the clients is to cooperatively train a shared machine learning model, called the global model, based on the overall training data set.

Functions:

• The clients jointly optimize a loss function, expressed as:
  $$\underset{In}{\min }L(D;w)=\underset{In}{\min }\sum _{i=1}^{n}L(D_i;w)$$
  inside $w$ is the global model parameter, and is simplified as$L_i(w)=L(D_i;w)$。

Server’s role:

• Servers provided by service providers (e.g. Google, Facebook, Apple) maintain global models.

The process of federated learning:

1. Step I: The server broadcasts the current global model to all clients ${In}_t$。
2. Step II: Each client uses gradient descent to calculate model updates based on the received global model and its local training data$g_t^i$, and then report it to the server.
3. Step III: The server aggregates the client's model updates according to an aggregation rule A.

B. Poisoning attack in federated learning

• The fragility of federated learning:

  • Federated learning is vulnerable to poisoning attacks, where malicious clients contaminate the global model by sending malicious model updates in the second step of FL.

• Ralicious client operations:

  • They can construct malicious model updates by polluting their local training data and/or directly manipulating model updates.

• Classification of poisoning attacks:

  1. Non-targeted poisoning attack: Tainted global models have higher test error rates for a large number of test inputs.
  2. Targeted Poisoning Attack: The tainted global model predicts an attacker-selected target label for the attacker-selected test input, but the test error rate for other test inputs is not Affected.
     • Example: A backdoor attack is a popular type of targeted poisoning attack in which the test input chosen by the attacker is any input embedded in a trigger.

• Introduction to the main attack methods:

  • Trim attack: Fang et al. proposed a non-targeted poisoning attack framework, which aims to formulate malicious attacks that maximize the difference between aggregated model updates before and after the attack. Model updated. This attack method is built on the Trimmed-mean aggregation rule, but is also applicable to other aggregation rules such as FedAvg and Median.
  • Backdoor attack: In a backdoor attack, the attacker contaminates the local training data of a malicious client by adding duplicates with triggers. To increase the impact of model updates, malicious clients amplify them before reporting them to the server. Example: Imagine an image classification system. An attacker could add an inconspicuous marker (such as a small red dot) to certain training images and assign those images a specific, possibly incorrect, label. In normal use, this little red dot may not appear, so the model's behavior appears to be normal. However, when the input image contains this little red dot, the model may incorrectly classify it as that specific label assigned by the attacker, even though this label otherwise does not match the actual content of the image.

• Existing defects and problems:

  • Although there are some methods that attempt to detect and remove backdoors in neural networks, they are not sufficient for federated learning. For example, some methods assume a clean training data set, but this is often not true for FL servers.

C. Detect malicious clients

• Purpose and basic issues:

  • The purpose of malicious client detection is to distinguish malicious clients from benign clients, which is essentially a two-classification problem.

• Guankey thought:

  • The main idea is to exploit certain statistical differences between the characteristics (e.g., model updates) of malicious and benign clients.

• How to play:

  • Different detection methods use different features and binary classifiers for detection.
  • For each client, these detection methods first extract features from its model updates in one or more rounds and then use a classifier to predict whether it is malicious.

• Specific example:

  • Zhang et al. proposed to detect malicious clients by checking the client’s model update consistency. Specifically, the server predicts the client's model updates based on historical model updates in each round. If model updates received over multiple rounds are inconsistent with predictions, the server marks the client as malicious.
  • Zhang et al. also utilized Cauchy's mean theorem and L-BFGS algorithm to predict model updates for clients, but they used the same approximate Hessian matrix for all clients, which was experimentally proven to be ineffective in model recovery.

• Relationship with Sybil detection:

  • Detecting malicious clients is also relevant to Sybil detection in distributed systems.
  • Traditional Sybil detection methods can also be used to detect malicious clients, where malicious clients are considered Sybil. In particular, these Sybil detection methods leverage the client's IP, network behavior, and (if available) social graphs.

D. Machine forgets

• Definition: Machine forgetting aims to make the machine learning model "forget" certain training samples. For example, users may want the model to forget their data due to privacy concerns.

• Method:

  • Cao et al.'s method: Propose to convert the learning algorithm used to train the machine learning model into a summation form, so that only a small number of summations need to be updated to forget a training sample .
  • Bourtoule et al.'s method: Decompose model training into an aggregation of multiple component models, with each training sample contributing to only one component model. Therefore, only one component model needs to be retrained to forget one training sample.
  • Wu et al.’s DeltaGrad method: Use the gradient of the training samples to be forgotten to estimate the gradient of the loss function on the remaining training samples.

• Relationship with recovery from poisoning attacks in FL: Model recovery from poisoning attacks in FL can be viewed as forgetting the detected malicious clients, even if the global model forgets Model updates obtained from detected malicious clients.

• Limitations of existing forgetting methods: Existing machine forgetting methods are insufficient for FL because:

  1. They require changes to the FL algorithm to train multiple constituent models, and are inefficient when multiple constituent models involve detected malicious customers and therefore need to be retrained.
  2. They require access to the customer's private local training data.

III.Problem Definition

A. Threat Model

• Attacker’s goal:

  • Non-targeted poisoning attack: The attacker hopes to increase the test error rate of the global model for a large number of test inputs.
  • Targeted poisoning attack: The attacker’s purpose is to make the global model predict the target label selected by the attacker for the target test input selected by the attacker, but other test inputs Forecasts are not affected.
  • Backdoor Attack: This is a subcategory of targeted poisoning attacks where the target test input includes any input embedded with a trigger of the attacker's choice, such as a specific signature pattern.

• Attacker capabilities:

  • Suppose an attacker controls some malicious clients but does not compromise the server.
  • The malicious client may be a fake client injected into the FL system by the attacker, or a real client in the FL system that has been compromised by the attacker.
  • A malicious client can send arbitrary model updates to the server.

• Background knowledge of the attacker:

  • Partial knowledge setting: The attacker knows the global model, loss function, local training data on the malicious client, and model updates.
  • Full knowledge setting: In addition to what is in the partial knowledge setting, the attacker also knows the local training data on all clients, model updates, and the server's aggregation rules. Poisoning attacks tend to be more powerful in full knowledge settings than in partial knowledge settings.
  • This paper considers strong poisoning attacks in a complete knowledge setting.

B. Design goals

• Purpose: Our goal is to design an accurate and efficient model recovery method for FL. We use training from scratch as a benchmark to measure the accuracy and efficiency of our recovery methods.

• quasi-certain:

  • For non-targeted poisoning attacks, the test error rate of the restored global model should be close to the test error rate of the restored global model trained from scratch.
  • For targeted poisoning attacks, the attack success rate of the global model recovered by our method should be as low as the attack success rate of the global model recovered by training from scratch.

• Efficiency:

  • Our recovery method should reduce the computational and communication costs on the client side.
  • Since clients are typically resource-constrained devices, we focus on client efficiency.
  • Model recovery imposes a communication and computational cost on the client when asking it to compute its exact model updates at each round.
  • Therefore, we measure the efficiency of the recovery method by how many rounds the client is required to compute its exact model updates. Our goal is to design an efficient recovery method that requires clients to compute their exact model updates in only a few epochs.

• Independent of detection method:

  • Different detection methods have been proposed to detect malicious clients.
  • Our goal is to design a general recovery method that is compatible with any detection method.
  • All detection methods predict a list of malicious clients, and our recovery method should be able to recover a global model using this list without requiring any additional information about the detection process.
  • In practice, the detector may miss some malicious clients or mistakenly detect some benign clients as malicious. Even if the false negative and false positive rates of the detector are non-zero, our recovery method should still be as accurate as training from scratch and more efficient.

• Independent of aggregation rules:

  • Various aggregation rules are proposed in FL, and the poisoned global model may be trained using different aggregation rules.
  • Our goal is to design a general recovery method that is compatible with any aggregation rule.
  • Our recovery method should not rely on FL's aggregation rules.

C. Server Requirements

• We assume that the server has storage capacity to hold the global model and client-side model updates collected while training the poisoned global model before the malicious client is detected.
• We also assume that the server has the computational power to estimate the client's model updates during recovery.
• These requirements are reasonable because servers (e.g. data centers) are often powerful.

IV. FEDRECOVER

A. Overview

• After the detected malicious client is removed, FedRecover initializes a new global model and iteratively trains it in multiple epochs.

• At each round, FedRecover simulates the three steps of FL we discussed in Section II-A on the server.

• The server no longer requires the remaining client to compute and communicate model updates, but instead uses stored historical information (including the original global model and original model updates) to estimate model updates.

• Estimation errors from client model updates over multiple rounds may accumulate, eventually leading to an inaccurate recovered global model.

• To optimize FedRecover, we further propose several strategies, including warm-up, periodic correction, anomaly repair, and final adjustment.

  • In these strategies, the server requires clients to compute their exact model updates in the early rounds of the recovery process, every certain number of rounds, when the estimated model updates are anomalous, and in the last rounds, respectively. Estimate them.

• In theory, we can bound the difference between the global model recovered by FedRecover and the global model recovered by training from scratch, under some assumptions; we show that this difference increases exponentially as FedRecover increases the computational/communication cost of the client decline.

B. Estimating client-side model updates

• Symbol definitions: We first define some symbols that help describe our method (given in Appendix Table I). The global model and the client's model updates collected by the server during the original training (i.e. before the malicious client was detected) are called the original global model and the original model update.

  • 使用${\overline{In}}_t$ represents the original global model, using ${\bar{g}}_{i_t}$Represents the original model update reported by the i-th client at round t.

  • 使用${\widehat{In}}_t$Represents the global model of FedRecover recovery in round t.

  • If the client computes it, use$g_{i_t}$represents the exact model update of the i-th client at the t-th round of the recovery process.

• Calculation of model update: Based on the integral version of Cauchy’s mean theorem, we can calculate the exact model update as follows$g_{i_t}$:

  • $g_{i_t}={\bar{g}}_{i_t}+H_{i_t}({\widehat{In}}_t-{\overline{In}}_t)$

  • Among$H_{i_t}$is an integrated Hessian matrix of the i-th client in round t. Intuitively, the gradient g is a function of the model parameters w.

  • 函数值$g_{i_t}-{\bar{g}}_{i_t}$The difference between can be passed through the variable ${\widehat{In}}_t-{\overline{In}}_t$The difference between and the integrated gradient along the function g, that is, $H_{i_t}$to describe.

• Computational Problem: The above equation involves an integrated Hessian matrix, which is challenging. To address this challenge, we use the efficient L-BFGS algorithm to compute an approximate Hessian matrix.

Approximate integrated Hessian matrix using L-BFGS algorithm

• L-BFGS algorithm: In optimization, the L-BFGS algorithm is a popular tool for approximating the Hessian matrix or its inverse. The algorithm requires global model differences and model update differences in past rounds to approximate at the current round.

• defined difference：

  • The global model difference in round t is defined as:$\Delta w_t={\widehat{In}}_t-{\overline{In}}_t$。

  • The model update difference of the i-th client in round t is defined as:$\Delta g_{i_t}=g_{i_t}-{\bar{g}}_{i_t}$。

  • The global model difference measures the difference between the recovered global model in a round and the original global model, while the model update difference measures the difference between the client's exact model update in a round and the original model update.

• L-BFGS buffer: The L-BFGS algorithm maintains a buffer of global model differences in round t$\Delta W_t$, and a buffer of model update differences for each client$\Delta G_{i_t}$。

• The size problem of the Hessian matrix: The size of the Hessian matrix is ​​the square of the number of global model parameters. Therefore, when the global model is a deep neural network, the Hessian matrix may be too large to stored in memory.

• Hessian-vector product: In practice, the product of the Hessian matrix and a vector v is usually required, which is called the Hessian-vector product. For example, in FedRecover, our goal is to find $H_{i_t}v$, also$in={\widehat{In}}_t-{\overline{In}}_t$. Therefore, modern implementations of the L-BFGS algorithm accept the vector v as an additional input and directly approximate the Hessian-vector product in an efficient manner.

• Challenges of L-BFGS: The standard L-BFGS algorithm faces a key challenge, which requires exact model updates in each round$g_{i_t}$to compute a buffer of model update differences, but our goal is to avoid asking clients to compute their exact model updates in most rounds. Next, we propose several optimization strategies to address this challenge.

D. Complete algorithm

• Arithmetic overview:

  • Algorithm 1 in the appendix shows the complete algorithm of our FedRecover.
  • Without loss of generality, we assume that before $m$ clients are malicious.

• Warm-up phase:

  • Previously$In\; the\; Tw$ round of warm-up phase, the server updates the restored global model according to the three steps of the FL framework discussed in Section II-A.

• Update after warm-up:

  • Each round after warm-up$In\; t$, if the server was in the previous round$t-1$ requires the client to compute the exact model update and the server first updates the buffer of the L-BFGS algorithm, as discussed in Section IV-C.
  • The server then updates the recovered global model using periodic revisions or estimated model updates.
  • If at least one coordinate in the estimated model update is greater than the anomaly threshold$\tau$, the client is required to calculate the exact model update.

• Final adjustments before training ends:

  • Before the server terminates the training process, it asks the client to calculate the exact model updates for the final adjustments.

Algorithm 1: FedRecover

Import:

• $n-m$ The remaining clients to be recovered C r = { C i ∣ m + 1 ≤ i ≤ n } C_r = \ {C_i | m + 1 \leq i \leq n\} Cr​={ Ci​∣m+1≤i≤n}
• Original global model${\overline{In}}_0,{\overline{In}}_1,\dots ,{\overline{In}}_T$ and original model update${\bar{g}}_0^i,{\bar{g}}_1^i,\dots ,{\bar{g}}_{T-1}^i$ $(m+1\le i\le n)$
• Academic rate $the$
• Number of preheating rounds $T_w$
• Periodic correction parameter $T_c$
• Number of rounds of final adjustment $T_f$
• Buffer size of L-BFGS algorithm $s$
• 异次阈值$t$
• 聚合规则 $A$

Exit:

• Recovered global model ${\widehat{In}}_T$

Calculation process:

1. ${\widehat{In}}_0\leftarrow {\overline{In}}_0$//Initialize the restored global model
2. 对于 $t=0,1,\dots ,Tw-1$ // 预热
   1. ${\widehat{In}}_{t+1}\leftarrow$ ExactTraining($Cr,{\widehat{In}}_t,\eta ,A$)
3. 对于 $t=Tw,Tw+1,\dots ,T-T_f-1$
   1. Update buffer if necessary $\Delta W_t$ 和 $\Delta G_t^i$
   2. 如果 $(t-Tw+1)againstTc==0$ // Periodicity correction
      1. ${\widehat{In}}_{t+1}\leftarrow$ ExactTraining($Cr,{\widehat{In}}_t,\eta ,A$)
   3. otherwise
      1. 对于 $i=m+1,m+2,\dots ,n$
         1. ${\tilde{H}}_t^i({\widehat{In}}_t-{\overline{In}}_t)\leftarrow$ L-BFGS($\Delta W_t,\Delta G_t^i,{\widehat{In}}_t-{\overline{In}}_t$)
         2. ${\hat{g}}_t^i={\bar{g}}_t^i+{\tilde{H}}_t^i({\widehat{In}}_t-{\overline{In}}_t)$
         3. 如果 $\Vert {\hat{g}}_t^i{\Vert }_{\infty }>\tau$ // different regular repair
            1. The server will ${\widehat{In}}_t$Sent to i-th client
            2. 第i个客户端计算 $g_t^i=\frac{\partial L_i({\widehat{In}}_t)}{\partial {\widehat{In}}_t}$
            3. The i-th client reports to the server $g_t^i$
            4. ${\hat{g}}_t^i\leftarrow g_t^i$
         4. ${\widehat{In}}_{t+1}\leftarrow {\widehat{In}}_t-the\cdot A({\hat{g}}_t^{m+1},{\hat{g}}_t^{m+2},\dots ,{\hat{g}}_t^n)$
4. 对于 $t=T-T_f,T-T_f+1,\dots ,T-1$ // Final adjustment
   1. ${\widehat{In}}_{t+1}\leftarrow$ ExactTraining($Cr,{\widehat{In}}_t,\eta ,A$)
5. Definition ${\widehat{In}}_T$

FEDRECOVER explained in detail

Explanation of warm-up phase

2. 对于 $t=0,1,\dots ,Tw-1$ // Warm-up
This line starts a loop that lasts $Tw$ rounds of the warm-up phase . The warm-up phase is a step before the algorithm starts and aims to provide a good starting point for subsequent calculations.

2.1. ${\widehat{In}}_{t+1}\leftarrow$ ExactTraining($Cr,{\widehat{In}}_t,\eta ,A$)
This line indicates that in each round of warm-up, the server will use all honest clients (defined by$Cr$ means) provides model updates for accurate training. "Exact training" here means that the server will rely entirely on model updates received from honest clients without making any approximations or guesses.

• ${\widehat{In}}_t$: Toumae rotation$Global\; model\; of\; recovery\; of\; t$.
• $Cr$: The set of all honest clients.
• $\eta$: Learning rate, which controls the step size of each model update.
• $A$: Aggregation rules that define how model updates from multiple clients are aggregated together.

The warm-up phase ensures that the algorithm starts from a suitable starting point, thereby improving the efficiency and accuracy of model recovery in subsequent steps.

Post-warmup update phase

3. 对于 $t=Tw,Tw+1,\dots ,T-T_f-1$
This is the main part of the algorithm. After the warm-up phase, the post-warm-up update of the model begins. This cycle begins with the rounds at the end of the warm-up and continues through the final rounds before the end of the workout.

3.1. Update buffer if necessary $\Delta W_t$ 和 $\Delta G_t^i$
Depending on possible changes in the context or new information on the data, the buffers of the L-BFGS algorithm may need to be updated.

3.2. 如果 $(t-Tw+1)againstTc==0$ // Cyclic Correction
This step checks whether the current round is a scheduled cyclic correction round. If so, do precise training.

• 3.2.1. ${\widehat{In}}_{t+1}\leftarrow$ ExactTraining($Cr,{\widehat{In}}_t,\eta ,A$)
  In this step, exact model training is performed without any approximation or estimation.

3.3. Otherwise
If it is not a periodic correction round, perform the following steps:

• 3.3.1. 对于 $i=m+1,m+2,\dots ,n$
  Perform the following operations for each honest client:
  • 3.3.1.1. Use the L-BFGS algorithm to estimate model updates.
  • 3.3.1.2. Computes model updated estimates.
  • 3.3.1.3. 如果 $\Vert {\hat{g}}_t^i{\Vert }_{\infty }>\tau$ // Anomaly repair
    If the model's estimated update exceeds a predetermined anomaly threshold, then:
    • 3.3.1.3.1. The server sends the current model to the i-th client.
    • 3.3.1.3.2. This client computes an exact update of the model.
    • 3.3.1.3.3. The client sends precise updates back to the server.
    • 3.3.1.3.4. Update the model with the exact updates sent by the client.
  • 3.3.1.4. Update the model using aggregation rules and learning rates.

This part is the core of the FedRecover algorithm, ensuring that the recovered model gradually improves with each round of training, and that any abnormal model updates can be identified and corrected in a timely manner.

final adjustment stage

4. 对于 $t=T-T_f,T-T_f+1,\dots ,T-1$ // Final adjustments
In the last few rounds of model training, the purpose of this part is to ensure that the model is Sufficient precision training and correction has been done.

4.1. ${\widehat{In}}_{t+1}\leftarrow$ ExactTraining($Cr,{\widehat{In}}_t,\eta ,A$)
In this step, similar to the previous periodic correction, use the ExactTraining function for accurate model training without any An approximation or estimate. This ensures that at the end of training, the model is optimal and has as few errors as possible.

Return the final recovery model

5 Definition ${\widehat{In}}_T$
After completing all warm-up, subject training and final adjustments, the algorithm returns the final recovery model ${\widehat{In}}_T$. This is the product of the purpose of the FedRecover algorithm, a model that minimizes anomalies and errors in a federated learning environment.

Algorithm 2: L-BFGS

Import:

• A global model difference buffer$\Delta W=[\Delta w_{b1},\Delta w_{b2},\dots ,\Delta w_{bs}]$
• A model update difference buffer$\Delta G=[\Delta g_{b1},\Delta g_{b2},\dots ,\Delta g_{bs}]$
• 向量 $in$

Exit:

• Approximate Hessian-vector product $\tilde{H}v$

Calculation process:

1. $A=\Delta W^T\Delta G$
2. $D=\text{diag}(A)$ // $A$的对angledsquare
3. $L=\text{tril}(A)$ // $Lower\; triangular\; matrix\; of\; A$
4. $p=\frac{\Delta g_{bs-1}^T\Delta w_{bs-1}}{\Delta w_{bs-1}^T\Delta w_{bs-1}}$
5. $p={\left[\begin{array}{c}-D\\ L^T\\ L\\ \sigma \Delta W^T\Delta W\end{array}\right]}^{-1}\left[\begin{array}{c}\Delta G^{T}v\\ \sigma \Delta W^{T}v\end{array}\right]$
6. $\tilde{H}v=\sigma v-\left[\begin{array}{c}\Delta G\\ \sigma \Delta W\end{array}\right]p$
7. Reply $\tilde{H}v$

Algorithm 3: ExactTraining

Import:

• Client $C$
• Current global model ${\widehat{In}}_t$
• Academic rate $the$
• 聚合规则 $A$

Exit:

• Updated global model ${\widehat{In}}_{t+1}$

Calculation process:

1. The server broadcasts to the client ${\widehat{In}}_t$
2. for $i=1,2,\dots ,|C|$ do
   3. 第$i$ clients compute accurate model updates$g_{i,t}=\frac{\partial L_i({\widehat{In}}_t)}{\partial {\widehat{In}}_t}$
   4. No. $i$ clients report to the server $g_{i,t}$
3. end for
4. ${\widehat{In}}_{t+1}={\widehat{In}}_t-the\cdot A(g_{t,1},g_{t,2},\dots ,g_{t,|C|})$
5. Definition ${\widehat{In}}_{t+1}$

E. Theoretical Analysis

First, we analyze the computational and communication costs of the client introduced by training from scratch and FedRecover. Next, we show that under some assumptions, the difference between the global model recovered by FedRecover at each round and the global model recovered by training from scratch is bounded. Finally, we show how this difference relates to the client's computational/communication costs, i.e., the trade-off between the accuracy of the global model recovered in FedRecover and the client's computational/communication costs. We note that our theoretical boundary analysis is based on some assumptions that may not be applicable to complex models such as neural networks. Therefore, we conduct an empirical evaluation of FedRecover with neural networks in the next section.

Impact of computation and communication costs on the client:

When requiring clients to compute model updates, we introduce some computation and communication costs to the client. Furthermore, this computation/communication cost is roughly independent of which round the client is asked to compute the model update. Therefore, we can think of this cost as a cost unit. Training from scratch requires each client to compute model updates at every epoch. Therefore, the average client computation/communication cost of training from scratch is $O\left(T\right)$, inside$T$ is the total number of rounds. In FedRecover, the cost depends on the number of warm-up rounds $Tw$, periodic correction parameter$Tc$, the number of rounds that triggered exception repair, and the number of final tuning rounds$T_f$. The number of rounds of anomaly repair depends on the data set, FL method and threshold $\tau$, which makes it difficult to theoretically analyze the cost of FedRecover. However, when exception repair is not used, i.e. $t=\infty$, we can show that the average client computation/communication cost of FedRecover is$O(T_w+T_f+\lfloor (T-T_w-T_f)/Tc\rfloor )$。

Bounds on the difference between global models recovered via FedRecover and trained from scratch:

We first describe the assumptions on which our theoretical analysis is based. We then show bounds on the differences between global models recovered by FedRecover and trained from scratch.

Hypothesis 1

A lapsed function is$\mu$-强凸和$L$-Smooth.对于每个客户$i$, we have the following two inequalities, for any $w$sum${In}^{\prime }$：
$$⟨w-{In}^{\prime },\nabla L_i(w)-\nabla L_i(w^{\prime })⟩\ge \mu \Vert w-{In}^{\prime }{\Vert }^2(3)$$
$$⟨w-{In}^{\prime },\nabla L_i\left(w\right)-\nabla L_i(w^{\prime })⟩\ge \frac{1}{L}\Vert \nabla L_i(w)-\nabla L_i(w^{\prime }){\Vert }^2(4)$$
inside$L_i$This is a guest$The\; loss\; function\; of\; i$, $⟨\cdot ,\backslash cdot\; \backslash \; |$$\Vert \cdot \Vert$display orientation${\ell }_2$norm.

Hypothesis 2

The error of approximating the Hessian-vector product in the L-BFGS algorithm is bounded. Formally, every approximate Hessian-vector product satisfies the following conditions:
$$\forall i,\forall t,\Vert {\tilde{H}}_i^t({\widehat{In}}_t-{\overline{In}}_t)+g{\bar{i}}_t-g_i^t\Vert \le M\left(5\right)$$
inside$M$ is a finite positive value.

Theorem 1

Assuming 1-2 holds, FedAvg is used as the aggregation rule, threshold$t=\infty$ (i.e., no exception repair is used), learning rate$\eta$ 意方$the\le \min (\frac{1}{m},\frac{1}{L})$, and all malicious clients are detected. Then, each round $t>0$The difference between the global model recovered by FedRecover and the model recovered by training from scratch can be defined as follows:
$$\Vert {\widehat{In}}_t-{In}_t\Vert \le {\left(\sqrt{1-nm}\right)}^t\Vert {\widehat{In}}_0-{In}_0\Vert +\frac{1-{\left(\sqrt{1-nm}\right)}^t}{1-\sqrt{1-nm}}\eta M(6)$$
其中，${\widehat{In}}_t$和${In}_t$In rounds $t$Global model recovered by FedRecover and trained from scratch.

Corollary 1

When the L-BFGS algorithm can accurately calculate the integral Hessian-vector product (i.e., $M=0$), the difference between the global model recovered by FedRecover and the model recovered from scratch is defined as
$$\Vert {\widehat{In}}_t-{In}_t\Vert \le {\left(\sqrt{1-nm}\right)}^t\Vert {\widehat{In}}_0-{In}_0\Vert$$
Therefore, the global model recovered by FedRecover converges to the global model recovered by training from scratch, i.e., we have ${\lim }_{t\to \infty }{\widehat{In}}_t={\lim }_{t\to \infty }{In}_t$。

The trade-off between accuracy and cost

Given Corollary 1, when FedRecover is run$When\; T$ round, we have the difference boundary as
$$\Vert {\widehat{In}}_T-{In}_T\Vert \le {\left(\sqrt{1-nm}\right)}^T\Vert {\widehat{In}}_0-{In}_0\Vert$$
Different limit adjunction$The\; increase\; in\; T$ decreases exponentially. In addition, when $t=When\; \infty$, the calculation/communication cost of FedRecover is the same as $T$ forms a linear relationship. Therefore, as costs increase, the difference bound decreases exponentially. In other words, we observe an accuracy-cost trade-off with FedRecover, i.e., when more costs are introduced for the client, the global model recovered by FedRecover is more accurate (i.e., closer to the global model trained from scratch) .

V. Assessment

A. Experimental setup

1. Dataset: We considered several different learning task datasets for evaluation. Specifically, we used two image classification datasets (MNIST and Fashion-MNIST), a purchasing style prediction dataset (Purchase), and a human activity recognition dataset (HAR). Unless otherwise stated, for simplicity, the experimental results we show are based on MNIST.

   • MNIST: MNIST [22] is a 10-category digital image classification dataset containing 60,000 training images and 10,000 test images. The height and width of each image are 28. We adopt the convolutional neural network (CNN) in [18] as the global model architecture. This CNN consists of two convolutional layers, each followed by a pooling layer and then two fully connected layers. We assume there are 100 customers and assign training images to them using the method in [18], which has a parameter called non-IID degree, ranging from 0.1 to 1. When the degree of non-IID is greater than 0.1, the customer's local training data is non-IID, and the greater the degree of non-IID, the greater the degree of non-IID. By default we set the degree of non-iID to 0.5 when assigning training images to clients, but we explore its impact on FedRecover.

   • Fashion-MNIST: Fashion-MNIST [35] is another 10-class image classification dataset. Unlike MNIST, which contains digital images, Fashion-MNIST contains 70,000 fashion images. The dataset is divided into 60,000 training images and 10,000 testing images, each of which is 28×28 in size. We adopt the same CNN as MNIST. In addition, we also assume that there are 100 clients, and when distributing training images to them, the default non-IID degree is 0.5.

   • Purchase: Purchase is a retail data set released by [1]. The task is to predict which buying style a customer belongs to. The dataset contains a total of 197,324 purchase records, each with 600 binary features and belonging to one of 100 imbalanced categories. The dataset is divided into 180,000 training records and 17,324 test records. Following the method of [29], we adopt a fully connected neural network with only one hidden layer as the global model architecture. The number of neurons in the hidden layer is 1,024 and the activation function is Tanh. Let's also assume there are 100 customers in total. Following the method of [29], we distribute the training records to them evenly.

   • Human Activity Recognition (HAR): HAR [4] is a 6-category human activity recognition data set. The dataset is collected from the smartphones of 30 real-world users. Each data sample contains 561 features, represents signals collected by multiple sensors of the user's smartphone, and belongs to one of 6 possible activities (e.g., walking, sitting, and standing). We consider every user in the dataset to be a customer. Furthermore, following the recommendation of [11], we use 75% of each customer's data as local training data and the remaining 25% as testing data. We adopt a fully connected neural network containing two hidden layers as the global model architecture, where each hidden layer consists of 256 neurons and uses ReLU as the activation function.

2. FL setup: Recall that the original FL training had three steps in each round. We consider a client using stochastic gradient descent to compute model updates. Considering the different characteristics of the datasets, we adopt the following parameter settings for original FL training: For MNIST and Fashion-MNIST, we perform 2,000 epochs of training with a learning rate of 3 × 1 0 − 4 3 \times 10^{-4} $3\times 10^{-4}$, the batch size is 32; for Purchase, we perform 1,000 epochs of training, and the learning rate is $1\times 10^{-4}$, the batch size is 2,000; for HAR, we perform 1,000 epochs of training, and the learning rate is $3\times 10^{-4}$, the batch size is 32. We consider three aggregation rules: FedAvg [24], Median [36] and Trimmed-mean [36]. We do not consider Krum [8] because it is neither accurate nor robust [18], [5], nor do we consider FLTrust [11] because it requires the server to have an additional clean data set. In all datasets, we set trim parameters for Trimmed-mean$k=n\times 20\%$. Specifically, for the MNIST, Fashion-MNIST, Purchase and HAR datasets, k is 20, 20, 20 and 6 respectively.

3. Attack settings: By default, we randomly sample 20% of customers as malicious customers. Specifically, for the MNIST, Fashion-MNIST, Purchase and HAR datasets, the number of malicious clients are 20, 20, 20 and 6 respectively. Furthermore, we assume that the attacker performs a full-knowledge attack. We consider trim attacks (a non-directed poisoning attack) [18] and backdoor attacks (a directed poisoning attack) [5]. We adopt the default parameter settings of the Trim attack in [18]. We follow the method of [11] to design triggers in backdoor attacks. Specifically, for MNIST and Fashion-MNIST, we use the same white pixel located in the lower right corner as the trigger. For Purchase and HAR, we set every 20 feature values ​​to 0 as a trigger. We choose 0 as the target label for all datasets. In a backdoor attack, each malicious client amplifies its malicious model updates. We set the amplification factor to 10 for MNIST and 5 for Fashion-MNIST and HAR because the success rate of backdoor attacks is high under these settings. We note that Purchase's attack success rate is similar when the amplification factor is varied from 1 to 100. Therefore, in order to be more invisible, we set the amplification factor of Purchase to 1. In each round of the original FL training, the malicious client performs a trim attack or a backdoor attack. Additionally, during the recovery process, when some malicious clients go undetected, they attack in every warm-up, periodic correction, anomaly fixing, and final tuning round.

B. Experimental results

1. FedRecover accuracy and efficiency:

   • According to Figure 1, for four datasets, three aggregation rules and two attacks, TER, ASR and ACP of training from scratch, using only historical information and FedRecover are shown.
   • FedRecover is observed to be both accurate and efficient in recovering global models from contaminated models.
   • FedRecover can achieve similar TER and ASR as training from scratch, and can obtain a large number of ACPs, that is, FedRecover can significantly reduce the computing/communication costs of the client.
   • Using only historical information introduces no cost to the client (i.e. ACP of 100), but the global model it recovers has a larger TER (approximately a random guess).
     

2. Impact of the number of malicious clients:

   • Figure 2 shows the impact of the number of malicious clients on recovery from a Trim attack.
   • It is observed that when a different number of clients are malicious, FedRecover can recover a global model that is as accurate as training from scratch, i.e., FedRecover’s TER (and ASR) is close to that of training from scratch.
   • In addition, FedRecover can save the client most of the cost compared to training from scratch.
     

3. Impact of non-iid degree:

   • Figure 3 shows the impact of the degree of non-IID of the client’s local training data on recovery from a Trim attack.
   • It is observed that for a wide range of non-IIDs, FedRecover can recover a global model that is as accurate as training it from scratch.
     

4. Number of preheating rounds$Tw$effect:

   • 图4显显了$The\; impact\; of\; Tw$ on FedRecover’s recovery from two attacks.
   • TER and ASR remain stable, while ACP decreases as the number of warm-up rounds increases.
     

5. Correction Period$Tc$'s influence:

   • Figure 5 shows $The\; impact\; of\; Tc$ on FedRecover’s recovery from two attacks.
   • Watch it$Tc$tradeoff between accuracy and efficiency.
     

6. Tolerance rate$\alpha$effect：

   • Figure 6 shows $The\; impact\; of\; \alpha$ on FedRecover.
     

7. Final adjustment round number$T_f$Impact:

   • Figure 7 shows $T_f$Impact on FedRecover when recovering from both attacks.
   • TER and ASR remain stable, while ACP decreases slightly as the number of final adjustment rounds increases.
     

8. The impact of false negative rate (FNR) and false positive rate (FPR) when detecting malicious clients:

   • Figure 8 shows the impact of FNR and FPR on recovery from Trim attacks.
   • It is observed that even if FNR or FPR is non-zero, FedRecover can still recover a global model that is as accurate as training it from scratch.
     

VI. Discussion and Limitations

A. Security/Privacy Concerns about Storing Historical Information

• Basic point of view: In FedRecover, the server stores the client's historical information, including each round of model updates.
• Security/Privacy Issues: Does stored historical information create additional security/privacy issues for the client?
  • In our threat model, it is assumed that the server has not been compromised by an attacker, so that the stored historical information does not introduce additional security/privacy issues.
  • If the server may have been compromised by an attacker, the security/privacy issues FedRecover poses to the client depend on when the server was compromised.
• Future Work Directions: Investigating possible additional security/privacy risks in such scenarios is an interesting future work direction.

B. Client exit

• Core idea: This article mainly studies how to restore the global model after some malicious clients are detected and removed by the server.
  • In practice, benign clients may also exit the FL system after global model training due to various reasons such as privacy.
  • An exiting client may want the global model to forget what it learned from its private local training data, or even its existence.
• Using FedRecover: The global model can be restored using FedRecover by treating exiting benign clients as detected "malicious" clients.
• Future Work Directions: Investigating the privacy guarantees provided by the restored global model for exiting benign clients is an interesting future work direction.

C. Server storage and computing costs

• Extra storage and computing costs: FedRecover brings additional storage and computing costs to the server.
  • Assume that the local/global model has$M$ parameters. The server requires $O\left(nMT\right)$ Preserved original model updated and complete model, part 1$n$ is the number of clients, $T$ is the number of global rounds.
  • The server needs to be estimated to be approximately$O((n-m\left)T\right)$ model update, part $m$ is the number of malicious clients. The estimated complexity of a model update is $O(M^{2}s)$，其中$s<M$ is the size of the buffer.
• Storage and compute costs: Storage and compute costs are acceptable for powerful servers such as modern data centers.

VII. Conclusion and future work

• FedRecover Introduction: We propose a model recovery method called FedRecover, which aims to eliminate the impact of backdoor attacks on global models in FL.
• The value of historical information: Our theoretical and empirical results show that the historical information collected by the server during the training of the poisoned global model before the malicious client is detected is useful There is value in efficiently recovering an accurate global model after detecting a malicious client.
• Future work direction:
  • Explore the accuracy and efficiency of FedRecover under adaptive backdoor attacks.
  • Extended FedRecover to other areas of FL.