0. Summary

Federated learning is vulnerable to poisoning attacks, where a malicious client poisons the global model by sending malicious model updates to the server. Existing defenses mainly focus on preventing a small number of malicious clients from poisoning the global model through robust federated learning methods, and detecting malicious clients when they exist in large numbers. However, after detecting malicious clients, how to recover the global model from poisoning attacks remains an open challenge. A simple solution is to delete detected malicious clients and train a new global model from scratch using the remaining clients. However, this ab-scratch recovery method incurs huge computational and communication costs, which may be intolerable for resource-constrained clients such as smartphones and IoT devices.

In this work, we propose a method: FedRecover, which recovers an accurate global model from poisoning attacks with minimal computational and communication costs on the client side. Our key idea is that the server evaluates the client's model updates, rather than asking the client to compute and communicate them during recovery. In particular, before detecting a malicious client, while training a poisoned global model, the server stores historical information, including updates of the global model and the client model for each round. During recovery, the server estimates the client's model updates in each round using the client's stored historical information. Furthermore, we further optimize FedRecover to recover a more accurate global model using warm-up, periodic correction, anomaly fix and final tuning strategies, where the server asks the clients to compute and deliver their accurate model updates. Theoretically, we demonstrate that under certain assumptions, the global model recovered by FedRecover is close to or identical to that recovered by train-from-scratch. Empirically, our evaluation on four datasets, three federated learning methods, and off-target and on-target poisoning attacks (e.g., backdoor attacks) shows that Federcover is both accurate and effective.

1 Introduction

Federated learning (FL) is an emerging machine learning paradigm that enables many clients (such as smartphones, IoT devices, and edge devices) to collaboratively learn a shared machine learning model (called the global model). Specifically, the training data is scattered across clients in FL, and the server maintains the global model. Roughly speaking, FL performs the following three steps in each round: the server broadcasts the current global model to (a subset of) clients; each client fine-tunes the global model using its local training data and reports its model updates to the server; the server The client's model updates are aggregated according to some aggregation rule, and the aggregated model updates are used to update the global model. Different FL methods essentially use different aggregation rules. FL is already used by tech giants. For example, Google uses FL on a virtual keyboard app called Gboard to predict the next word; WeBank uses FL for credit risk prediction.

However, due to its distributed setting, FL is vulnerable to poisoning attacks. Specifically, the attacker can access some malicious clients, which may be fake clients injected into the system by the attacker, or real clients compromised by the attacker.
Malicious clients poison the global model by sending carefully crafted malicious model updates to the server. Malicious clients can perform malicious model updates by poisoning their local training data and/or directly constructing non-following FL protocols. In the untargeted poisoning attack, the poisoned global model indiscriminately misclassifies many test inputs, i.e., the poisoned global model has a large test error rate. In a targeted poisoning attack, the poisoned global model predicts the attacker-chosen target label for the attacker-chosen target, but its predictions for other test inputs are unaffected. For example, in a backdoor attack (a type of targeted poisoning attack), the target input could be any trigger embedded in the attacker's choice.

Existing defenses against FL poisoning attacks can prevent a small number of malicious clients from poisoning the global model and/or detect malicious clients .
Specifically , some studies have proposed Byzantine robust or provably robust FL methods that can prevent a small number of malicious clients from poisoning the global model, i.e., they can guarantee that the global model learned with malicious clients is close to that learned without malicious clients , or a guaranteed lower bound on test accuracy under a finite number of malicious clients. However, these FL methods are still vulnerable to poisoning attacks by a large number of malicious clients. Therefore, some studies further propose detecting malicious clients during or after training, which can be used together with prevention methods to form a defense-in-depth strategy. For example, a server can distinguish malicious clients from benign clients by some statistical difference in the model updates sent to the server. Since this detection method requires sufficient model updates to make confident decisions, malicious clients often poison the global model before being detected. Therefore, the server needs to recover an accurate global model from the poisoned global model after detecting malicious clients.

[Malicious clients cannot be completely isolated, so they must face the problem of dealing with the poisoned global model]

However, efficient model restoration of FL has largely remained unexplored. Since the server does not know in which round the attack occurred, the server may not be able to simply roll back to a clean global model in the previous round. A naive approach to recovery (which we call train-from-scratch) is to remove detected malicious clients and train a new global model from scratch using the remaining clients. train-from-scratch can recover an accurate global model. But this requires customers to re-engage in the entire training process, which brings a lot of calculation and communication costs to customers. Such computational and communication costs may be intolerable for resource-constrained clients such as smartphones and IoT devices.

[The accuracy can be restored from the beginning, but the client needs to participate from the beginning, which is too expensive]

Our work: In this work, we propose the FedRecover method, which can recover an accurate model from a poisoned global model, while introducing a small amount of computation and communication costs for clients. Like train-from-scratch, FedRecover removes detected malicious clients, reinitializes the global model, and iteratively trains it in multiple epochs. However, unlike train-from-scratch, FedRecover reduces costs for clients by changing the way to obtain model updates. Our intuition is that historical information, including the global model and the client's model updates, which were collected when the server trained the poisoned global model before the malicious client was detected, still carries valuable information for model recovery. Based on this intuition, our key idea is that during recovery, the server uses such historical information to evaluate model updates for remaining clients, rather than requiring clients to perform computation and communication. FedRecover is not affected by detection methods and FL aggregation rules of malicious clients, that is, in a defense-in-depth strategy, FedRecover can be used with any detection method and FL aggregation rules.

[The server retains the information communicated before, and when rebuilding, the server uses the old information to calculate itself. So it can be combined with any method]

The key to FedRecover is that the server itself estimates the client's model updates during the recovery process. Specifically, the server stores historical information when training a poisoned global model before malicious clients are detected. During recovery, the server uses the well-known Cauchy median value theorem to estimate the client's model update for each round. However, Cauchy's median value theorem requires each client to have an integrated Hessian matrix, the exact value of which is difficult to compute. To address this challenge, we further utilize an L-BFGS-based algorithm to efficiently approximate the integral Hessian matrix. FedRecover imposes some storage and computation costs on the server due to storing historical information and estimating the client's model updates. However, such a cost is acceptable due to the powerful server.

[The real innovation point: use an algorithm to approximate the Hessian matrix, and prove his error]

Since FedRecover estimates the client's model updates, during the recovery process, estimation errors may accumulate over multiple rounds, which may eventually lead to a reduction in the accuracy of the recovered global model. We propose multiple strategies to address this challenge. Specifically, the L-BFGS algorithm requires the global model recovered from previous rounds to estimate the client’s model update for the current round. A global model that is accurately recovered during previous rounds of recovery will help reduce estimation errors in future rounds. Therefore, we propose a warm-up strategy , where the server asks clients to compute and deliver their precise model updates in the first Tw rounds of the recovery process. Furthermore, we also propose a periodic correction strategy , where the server asks the clients to compute and communicate their precise model updates every Tc round. When the client's estimated model updates are large, it has a large impact on the recovered global model. To reduce the impact of potentially mis-estimated large model updates, we propose an exception-fixing strategy , where the server asks the client to compute its exact model update when at least one coordinate of the estimated model update is larger than a threshold τ. Furthermore, we also propose a final tuning strategy to reduce the estimation error before the end of training , i.e., the server asks the client to compute and deliver accurate model updates in the final Tf rounds. The parameters Tw, Tc, τ and Tf control the trade-off between the accuracy of the recovered global model and the computational/communication cost of the client. In particular, larger Tw, smaller Tc, smaller τ, or larger Tf may recover a more accurate global model, but will also incur greater cost to the client.

In theory, we demonstrate that the difference between the global model recovered by FedRecover and the global model recovered by train-from-scratch is bounded under certain assumptions, e.g., the loss function used to learn the global model is smooth and strong convex. Empirically, we make extensive use of four datasets, three FL methods (e.g., FedAvg, Median, and Trimmed-mean), as well as the Trim attack (an untargeted poisoning attack) and the backdoor attack (a targeted poisoning attack) to evaluate FedRecover. Our empirical results show that FedRecover can recover a global model as accurate as train-from-scratch, while saving clients substantial computation/communication costs. For example, when the data set is MNIST and the FL method is Trimmed-mean, using 40 malicious clients for backdoor attack can obtain an attack success rate of 1.00. Both FedRecover and train-from-scratch recover the global model with a test error rate of 0.07 and an attack success rate of 0.01, but FedRecover saves an average of 88% of client-side computation/communication costs than train-from-scratch. Furthermore, even if the detection method wrongly detects some malicious clients as benign and/or some benign clients as malicious, FedRecover can effectively recover a global model as accurate as train-from-scratch.

In summary, our main contributions are as follows:

• We conduct the first systematic study on recovery from FL poisoning attack models.
• FedRecover uses historical information and various optimization strategies to estimate the customer's model update, thereby recovering the global model.
• We evaluate FedRecover both theoretically and empirically.
The results show that FedRecover can recover the global model accurately and effectively

2. Background and relevant knowledge

2.1 Background knowledge of FL

Suppose the FL system has n clients, and each client has a local training data set $D_{i}$ , i=1,2,...,n. We denote the joint training dataset by D = $U_{i=1}^{n}$ $D_{i}$ , which is the union of the client's local training datasets. The n clients aim to collaboratively train a shared machine learning model (called the global model) based on a joint training dataset. To achieve this goal, n clients jointly minimize a loss function on their training dataset, namely $min_{w}$ L(D;w) = $min_{w}$ ∑︁ $_{i=1}^{n}$ L(Di;w), where w represents the global model parameters and L is the empirical Loss function (eg, cross-entropy loss). For simplicity, let Li(w) = L(Di;w) in the rest of this work. A server provided by a service provider (eg Google, Facebook, Apple) maintains the global model. The global model is iteratively updated in multiple rounds, and in round t, FL takes the following three steps:

• Step 1: The server broadcasts the current global model to the client $w_{t}$ . The server can also broadcast the global model to a subset of clients. Our method is also suitable for this scenario. However, for simplicity, we assume that each round involves all clients.

$w_{t}$ • Step 2: The i-th client calculates model update $g_{t}^{i}$ = ∂Li(wt)/∂wt using gradient descent based on the received global model and the client's local training data Di. If the local training dataset is large, the client can also use stochastic gradient descent and mini-batch processing. For simplicity, we assume gradient descent in our method description, but we employ stochastic gradient descent in our experiments. The client then reports model updates to the server. Note that clients compute their model updates in parallel.

• Step 3: The server aggregates the model update of the client according to the aggregation rule A, and then uses the aggregated model update to update the global model. The update rate is η, that is, wt+1 = wt−η·A (gt 1, gt 2, ···, gt n).

In de novo training, the server initializes the global model, and then the server and other clients update it iteratively in each round following the above three steps. Different FL methods essentially use different aggregation rules in step III. Next, we review several popular aggregation rules.

FedAvg : FedAvg [24] developed by Google Inc. computes a weighted average of client-side model updates as aggregated model updates.
Formally, given the model updates gt 1, gt 2, ..., gt n at round t, the aggregated model updates are as follows:

where |·| represents the size of the dataset.

Median : Median is a coordinate-based aggregation rule that aggregates each coordinate for model updates individually. Specifically, for each coordinate, Median calculates the number of bits of the corresponding coordinate in n times of model updates, and takes it as the corresponding coordinate of the aggregated model update.

Trimmed-mean : Trimmed-mean is also a coordinate-based aggregation rule. For each coordinate, Trimmed-mean ranks the values of the corresponding coordinate among n model updates. Then, remove the maximum and minimum values of k. Finally, the average of the remaining values is calculated as the corresponding coordinates for the aggregation model update. k < n/2 is a hyperparameter for pruning the mean.

2.2 Poisoning attack on FL

Federated learning is vulnerable to poisoning attacks, where a malicious client poisons the global model by sending malicious model updates to the server in step II of FL. Malicious clients can construct their malicious model updates by poisoning local training data and/or directly manipulating model updates without following the FL protocol specified in Step II. According to the attacker's goal, poisoning attacks can be divided into non-targeted poisoning attacks and targeted poisoning attacks. In an untargeted poisoning attack, the poisoned global model has an indiscriminately large test error rate for a large number of test inputs. In a targeted poisoning attack, the poisoned global model predicts the attacker-chosen target label for the attacker-chosen target test input; in order to maintain stealth, the poisoned global model is unaffected in the test error rate for other test inputs. For example, backdoor attacks are popular targeted poisoning attacks in which the test input chosen by the attacker is any input embedded in a trigger. Next, we review Trim attacks (a common untargeted poisoning attack) and backdoor attacks (a common targeted poisoning attack).

Pruning Attacks : Fang et al. took the untargeted poisoning attack of FL as a general framework. Roughly speaking, the framework aims to craft malicious model updates that maximize the difference between aggregated model updates before and after the attack. This framework can be applied to different aggregation rules. The Trim attack is constructed based on the Trimmed-mean aggregation rules under this framework, and it is also effective for other aggregation rules such as FedAvg and Median.

Backdoor attack : In a backdoor attack, an attacker poisons a malicious client's local training data by augmenting a copy of an embedded trigger. Specifically, for each input in the malicious client's local training dataset, the attacker makes a copy and embeds a trigger in the copy. The attacker then injects a copy of the embedded trigger into the malicious client's local training dataset and relabels it with the target label. In each round of FL, each malicious client computes a model update based on its poisoned local training data. To amplify the impact of model updates, malicious clients further amplify them by a large factor before reporting them to the server. We note that some methods have been proposed to detect and remove backdoors in neural networks. However, they are not sufficient for FL. For example, assuming a clean training dataset is available, this is usually not suitable for FL servers.

2.3 Detecting Malicious Clients

Malicious client detection aims to distinguish malicious clients from benign clients, which is essentially a binary classification problem. Roughly speaking, the key idea is to exploit some statistical differences between the characteristics (e.g., model updates) of malicious and benign clients. Different detection methods use different features and binary classifiers to perform detection. Specifically, for each client, these detection methods first extract features from its model updates in one or more rounds, and then use a classifier to predict whether it is malicious or not. For example, Zhang et al. proposed to detect malicious clients by checking the client's model update consistency. In particular, the server predicts the client's model updates based on each round of historical model updates. If the received model updates are inconsistent with the predictions over multiple rounds, the server will flag the client as malicious. Zhang et al. also used Cauchy's median value theorem and L-BFGS algorithm to predict the client's model update, but they used the same approximate Hessian matrix for all clients, and we found in experiments that this approach is not effective for model recovery, e.g., restoring the model The accuracy of is probably a near-random guess.

Detecting malicious clients is also relevant to Sybil detection in distributed systems. Therefore, traditional Sybil detection and protection methods can also be used to detect malicious clients, where malicious clients are regarded as Sybil. In particular, these Sybil detection methods leverage client ip, network behavior, and social graph (if available).

2.4 The machine forgets

The purpose of machine forgetting is to make a machine learning model "forget" some training examples. For example, a user might want the model to forget about their data due to privacy concerns. Various approaches have been proposed to achieve efficient machine undo learning. For example, Cao et al. propose to transform the learning algorithm used to train a machine learning model into a summation form. Therefore, only a small number of sums need to be updated to forget about a training example. Bourtoule et al. decompose model training into an aggregation of multiple constituent models, with each training example contributing to only one constituent model. Thus, only one constituent model needs to be retrained when a training example is discarded. Wu et al. proposed DeltaGrad, which uses the gradient on the unlearned training examples to estimate the gradient of the loss function on the remaining training examples.

Model recovery from FL intoxication episodes is associated with machine amnesia. In particular, model recovery can be viewed as forgetting detected malicious clients, i.e., making the global model forget model updates from detected malicious clients. However, existing machine-forgetting methods are insufficient for FL because 1) they need to change the FL algorithm to train multiple composition models, and are inefficient when multiple composition models involve detected malicious clients, So retraining is required, and/or 2) they require access to the client's private local training data.

3. Problem definition

3.1 Threat Model

We follow the threat model considered in previous studies on FL poisoning attacks. Specifically, we discuss the attacker's goals, capabilities, and background knowledge in detail.

Attacker's goal : In an untargeted poisoning attack, the attacker's goal is to increase the test error rate of the global model indiscriminately over a large number of test inputs. In a targeted poisoning attack, the attacker's goal is to poison the global model so that it predicts the attacker-chosen target label for the attacker-chosen target test input, but the predictions for other test inputs are unaffected. For example, in a type of targeted poisoning attack known as a backdoor attack, the target test input consists of triggers, e.g., signature patterns, arbitrarily chosen by the attacker to embed.

Attacker Capabilities : We assume that the attacker controls some malicious clients, but does not compromise the server. The malicious client may be a fake client injected into the FL system by the attacker, or it may be a real client in the FL system compromised by the attacker. Malicious clients can send arbitrary model updates to the server.

Attacker's background knowledge : There are two common settings of the attacker's background knowledge on the FL system [1], namely the partial knowledge setting and the full knowledge setting. The partial knowledge setting assumes that the attacker knows the global model, loss function, and local training data and model updates on the malicious client. The full-knowledge setting further assumes that the attacker knows the local training data and model updates on all clients as well as the server's aggregation rules . Poisoning attacks tend to be stronger with full knowledge than with partial knowledge. In this work, we consider strong poisoning attacks in the full-knowledge setting.

3.2 Design goals

Our goal is to design an accurate and efficient method for FL model restoration. We use train-from-scratch as a baseline to measure the accuracy and efficiency of the recovery method. Our method should recover a global model as accurate as that recovered by train-from-scratch, while reducing client-side computation and communication costs. Specifically, our design goals are as follows:

Accurate : The global model recovered by our recovery method should be accurate. Especially for untargeted poisoning attacks, the test error rate of recovering the global model should be close to the test error rate of train-from-scratch recovering the global model. For targeted poisoning attacks, we further require that the attack success rate of the global model recovered by our method should be as low as that of the global model recovered by train-from-scratch.

Efficient : Our recovery method requires less client-side computation and communication costs. We focus on client-side efficiency because clients are typically resource-constrained devices. Model restoration introduces a unit of communication and computation cost to the client when the client is required to compute the exact model update in one round. Therefore, we measure the efficiency of the recovery method by comparing the number of rounds the client takes to compute an accurate model update. Our goal is to devise an efficient recovery method that requires clients to compute accurate model updates in only a fraction of rounds. Note that our approach imposes acceptable computational and storage costs on the server.

Independent of detection methods : Different detection methods have been proposed to detect malicious clients. In addition, new detection methods may be developed in the future. Therefore, our goal was to design a general recovery method compatible with any detection method. Specifically, all detection methods predict a list of malicious clients, and our recovery method should be able to use this list to recover the global model without any additional information about the detection process. In practice, the detector may miss some malicious clients (i.e., false negatives) or mistakenly detect some benign clients as malicious (i.e., false positives). When the false negative rate and false positive rate of the detector are non-zero, our recovery method should still be as accurate as training from scratch, and more efficient than training from scratch.

Independent of aggregation rules : Various aggregation rules are proposed in FL, and a poisoned global model can be trained with different aggregation rules. Therefore, our goal is to design a general recovery method that is compatible with any aggregation rule. Our recovery method should not depend on FL's aggregation rules. In particular, during recovery, we use the same aggregation rules as in training the poisoned global model.

3.3 Server Requirements

We assume that the server has sufficient storage capacity to save the global model and client-side model updates collected by the server while training the poisoned global model before the malicious client is detected. We also assume that the server has the computational power to estimate client model updates during recovery. These requirements are reasonable because servers (eg, data centers) are usually powerful. We discuss server costs in detail in Section VI-C.

4、F ED R RECOVER

4.1 Overview

After removing detected malicious clients, FedRecover initializes a new global model and trains it in multiple iterations. In each round, FedRecover simulates on the server the three steps of FL we discuss in Section II-A. The server estimates model updates using stored historical information, including the original global model and original model updates, rather than asking the rest of the clients to compute and communicate model updates. Estimation errors in client-side model updates may accumulate over multiple rounds, eventually leading to an inaccurate restored global model. Therefore, we further propose several strategies including warm-up, regular correction, anomaly repair and final tuning to optimize FedRecover. In these strategies, the server asks the client to compute its accurate model updates instead of estimating them periodically in the first few rounds of the recovery process, every certain number of rounds, when the estimated model updates are abnormal, and in the last few rounds, respectively. In theory, we can constrain the difference between the global model of FedRecover recovery and the global model of train-from-scratch recovery under certain assumptions; we show that as FedRecover increases the computational/communication cost of the client, this difference increases Exponential decline.

4.2 Evaluating model updates for clients

Notation : We first define some notations (shown in Table I in the Appendix) that will help describe our method. We call the global model, the client's model update the server's original global model and the original model update collected in the original training (i.e., before detecting malicious clients). In particular, we $\bar{w}_{t}$ denote the original global model with , and update with $\bar{g}_{t}^{i}$ the original model reported by the i-th client at round t , where i = 1, 2, , n, and t = 1, 2, , T. Furthermore, we use a global model $\hat{w}_{t}$ representing the recovery in round t of FedRecover . We use the exact model update (if computed by the client) $g_{t}^{i}$ representing the i-th client during the t-th round of recovery, ie .

In train-from-scratch, the server asks each client to compute and communicate in step II of the FL framework $g_{t}^{i}$ . In FedRecover, the server stores $\bar{w}_{t}$ , $\bar{g}_{t}^{i}$ and $\hat{w}_{t}$ , where i = 1, 2, ··, n and t = 1, 2, ··, T; and the server uses them to estimate $g_{t}^{i}$ , instead of asking the client in step II of the FL framework calculate it. We $g_{t}^{i}$ denote the estimated version of $\hat{g}_{t}^{i}$ . Next, we discuss how to estimate $\hat{g}_{t}^{i}$ .

Calculating the model update using Cauchy's mean value theorem : Based on the integral version of Cauchy's mean value theorem (Theorem 4.2 on page 341 of [21]), we can compute the exact model update as follows $g_{t}^{i}$ :

where is the integrated Hessian matrix of the i-th client in the t-th round . Intuitively, the gradient g is a function of the model parameters w. The difference between the function values $g_{t}^{i}$ - can be characterized by the difference between the variable - and the integral gradient of the function g along the line between the variables (ie ). Note that the above equations involve integrating the Hessian matrix, which is challenging to compute accurately. To solve this problem, we utilize an efficient L-BFGS algorithm to compute the approximate Hessian matrix. Next, we discuss how to approximate the integral Hessian matrix. $\bar{g}_{t}^{i}$ $\hat{w}_{t}$ $\bar{w}_{t}$ $H_{t}^{i}$

Approximating the integral Hessian matrix using the L-BFGS algorithm: In optimization, the L-BFGS algorithm is a common tool for approximating the Hessian matrix or its inverse. The L-BFGS algorithm requires the difference of the global model and model updates from past rounds for approximation in the current round. Specifically, we define the global model difference at round t $\Delta w_{t}$ as = $\hat{w}_{t}$ − $\bar{w}_{t}$ , and the model update difference of the i-th client in round t as ∆ $g_{t}^{i}$ = $g_{t}^{i}$ − $\bar{g}_{t}^{i}$ . Note that the global model difference measures the difference between the recovered global model and the original global model in a round, while the model update difference measures the difference between the client's precise model update and the original model update in a round. The L-BFGS algorithm maintains the buffer of global model differences in round t $\Delta W_{t}$ = [∆Wb1,∆Wb2, ∆Wbs], where s is the buffer size. Furthermore, for each client i, the L-BFGS algorithm maintains a buffer of model update differences . The L-BFGS algorithm requires the sum $\Delta W_{t}$ as $\Delta G_{t}^{i}$ input and outputs the approximate Hessian matrix of the i-th customer in the t-th round $\tilde{H}_{t}^{i}$ , ie $\tilde{H}_{t}^{i}$ =L-BFGS( $\Delta W_{t}$ , $\Delta G_{t}^{i}$ ).

Note that the size of the Hessian matrix is the square of the number of global model parameters , so when the global model is a deep neural network, the Hessian matrix may be too large to store in memory. Also, in practice, the product of the Hessian matrix and the vector v is often required, which is called the Hessian -vector product. For example, in FedRecover, our goal is to find $H_{t}^{i}$ v , where v = $\hat{w}_{t}$ − $\bar{w}_{t}$ . Thus, modern implementations of the L-BFGS algorithm take the vector v as an additional input and directly approximate the Hessian vector product in an efficient manner, ie . The algorithm we used, details of which can be found in Algorithm 2 in the Appendix. There are other variants and implementations of L-BFGS. However, they approximate the inverse Hessian vector product, not the Hessian vector product, and thus are not suitable for FedRecover. After obtaining the approximate Hessian vector product , we can compute the estimated model update as

Note that in the standard L-BFGS algorithm, the buffer of the global model difference (or model update difference) in the t-th round consists of the global model difference or model update difference in the previous s rounds, i.e. =t−s+j− $b_{j}$ 1 .This standard L-BFGS algorithm faces a key challenge: it requires exact model updates in each round $g_{t}^{i}$ in order to compute a buffer of model update differences, but our goal is to avoid requiring clients to compute their exact Model. Next, we propose several optimization strategies to address this challenge.

4.3 Optimization strategy

Warmup: Our first optimization strategy is to warm up the L-BFGS algorithm in the first few rounds of the recovery process. In particular, in the first $T_{w}$ >s rounds, the server asks the client to compute its exact model updates $g_{t}^{i}$ and uses them to update the recovered global model. According to the warm-up of the previous s rounds, the server calculates the buffer of the global model difference in the t-th round $\Delta w_{t}$ and the model update difference ∆ ∆ of the i-th client in the t-th round $g_{t}^{i}$ , and then, in the next rounds, the server can use The L-BFGS algorithm is used to calculate the approximate Hessian matrix, then the approximate Hessin matrix is used to calculate the estimated model update, and finally the estimated model update is used to update the restored global model. However, the buffer built based on the warmup cycle may become outdated for future cycles, which leads to inaccurate approximation of the Hessian matrix, inaccurate updating of the estimated model, and ultimately inaccurate recovery of the global model. To address this challenge, we further propose periodic correction and anomaly repair strategies, which we discuss next.

Periodic Calibration and Anomaly Fixing: In Periodic Calibration, the server asks each client to periodically compute its accurate model update every Tc cycle after warmup. In anomaly fixing, if the estimated model update is abnormally large, i.e., if at least one coordinate of the estimated model update is larger than τ (we call it an anomaly threshold ), the server asks the client to compute its accurate model update in this round. Larger estimated model updates have a larger impact on the recovered global model, so larger misestimated model updates will have a substantial negative impact on the recovered global model. Therefore, we consider exception-fixing strategies to limit the impact of potentially misestimated model updates.

If the anomaly threshold τ is too small, our anomaly repair strategy will treat correctly estimated large model updates as anomalies, which increases the computation/communication cost on the client side. Therefore, we choose τ based on historical information. Specifically, for each round t, we collect the original model updates of all clients participating in the recovery $\bar{g}_{t}^{i}$ . We choose $\tau _{t}$ such that the parameters of the most alpha part of the client original model update are greater than $\tau _{t}$ . Then we choose τ as $\tau _{t}$ the maximum value in , ie τ = $max_{t}$ { }. Here, the probability that a certain parameter in a benign model update is considered an anomaly in any round is no greater than α, and we call α the tolerance rate because we allow at most α part of such mishandling. $\tau _{t}$

Final Tuning : We found that if we terminate training with one round of estimated model updates, the performance of the recovered global model can be unstable due to potential estimation errors. Therefore, we further propose a final tuning strategy, where the server asks the client to $T_{f}$ compute its accurate model update in the last round before the end of training. As we will show in experiments, only a small number of cycles (eg, $T_{f}$ = 5) are needed to ensure good performance of the recovered global model.

We note that when some malicious clients are not detected by malicious client detection methods, they can still conduct form poisoning attacks in a cycle of warm-up, regular correction, anomaly fixing, and final tuning. However, our experiments will show that FedRecover can still recover an accurate global model in this case. This is due to the low number of warmups, periodic corrections, exception fixes, and final jumps.

Updating the buffers of the L-BFGS algorithm : Recall that the buffers of L-BFGS require client-side accurate model updates. Therefore, we only update the buffer after the server asks all clients to compute their model updates $\Delta w_{t}$ , and after the server asks the i-th client to compute its exact model update $\Delta G_{t}^{i}$ . Note that the client only computes the precise model updates it uses for warmups, periodic corrections, anomaly fixes, or final adjustments . In round t, $\Delta w_{t}$ include the global model difference from the previous s rounds, where all clients compute their exact model updates; then $\Delta G_{t}^{i}$ include the model update difference for the i-th client in the previous s rounds, where the i-th client computes its Accurate model updates.

4.4 Complete Algorithm

Algorithm 1 in the appendix shows the complete algorithm of FedRecover. Without loss of generality, we assume that the top m clients are malicious. In $T_{w}$ the front-run warmup, the server follows the three steps of the FL framework discussed in Section 2.1 to update the recovered global model. In each round t after warm-up, if the server asks the client to calculate the accurate model update of the previous round t-1, the server first updates the buffer of the L-BFGS algorithm, as described in Section 4-3. The server then updates the recovered global model with periodically corrected or estimated model updates. The client is asked to compute an accurate model update if at least one coordinate of the estimated model update is greater than the anomaly threshold τ. Finally, before the server terminates the training process, it asks the client to compute precise model updates for final adjustments.

4.5 Theoretical analysis

We first analyze the computation and communication costs of the clients introduced by train-from-scratch and FedRecover. We then demonstrate that, under certain assumptions, the difference between the global model recovered by FedRecover and that recovered by train-from-scratch can be bounded in each round. Finally, we show how this difference is linked to the computational/communication cost of the client, namely the trade-off between the accuracy of the global model recovered in FedRecover and the computational/communication cost of the client. We note that our theoretical bounds analysis is based on assumptions that may not hold for complex models such as neural networks. We therefore conduct an empirical evaluation of FedRecover with neural networks in the next section.

Computational and communication costs for clients : We introduce some computational and communication costs to clients when they are asked to compute model updates. Furthermore, this computation/communication cost is roughly independent of which round the client is asked to compute model updates. Therefore, we can consider such costs as unitizable. Train-from-scratch requires each client to compute model updates in each round. Therefore, the average computation/communication cost per client for Train-from-scratch is O(T), where T is the total number of rounds. In FedRecover, the cost depends on the number of warm-up cycles Tw, the periodic correction parameter Tc, the number of cycles that trigger anomaly repair, and the number of final adjustment cycles Tf. The number of anomaly repairs depends on the dataset, FL method and threshold τ, which makes it difficult to analyze the cost of FedRecover theoretically. However, when exception repair is not used, i.e. τ=∞, we can show that the average computation/communication cost per client for FedRecover is

Constraining differences between global models for FedRecover and train-from-scratch recovery : We first describe the assumptions on which our theoretical analysis is based. We then show bounds on the differences between the global patterns recovered by FedRecover and those trained from scratch.

Assumption 1. The loss function is µ-intensity convex and L-smooth. Formally, for each customer i, for any w and w′, we have the following two inequalities:

where Li is the loss function for client i, 〈 , 〉 denotes the inner product of two vectors, and ∥ ∥ denotes the vector of the ℓ2 benchmark.

Assumption 2. The error in approximating the Hessian vector product in the L-BFGS algorithm is bounded. Formally, each approximated Hessian vector product satisfies the following condition:

where M is a finite positive value.

Theorem 1. Assuming Assumptions 1-2 hold, FedAvg is used as aggregation rule, threshold τ=∞ (i.e., no exception repair is used), learning rate η satisfies η≤ min(1/µ, 1/L), and all malicious client. Then, the difference between the global model recovered by FedRecover and train-from-scratch in each round t>0 can be shown as follows:

where wˆt and wt are the global models recovered by FedRecover and train-from-scratch respectively, and are trained from scratch in t rounds.

Proof : The idea is to recursively limit the difference each round. Appendix A shows the detailed evidence

Given Theorem 1, we restrict .
In addition, we have the following inferences:

Corollary 1. When the L-BFGS algorithm is able to compute exactly the Hessian vector product of the integral (i.e., M=0), the difference between the global model recovered by FedRecover and that recovered by train-from-scratch is limited to . Therefore, the global model recovered by FedRecover converges to the train-from-scratch global model, i.e., we have .

Trade-off between variance bound and computation/communication cost : Given Corollary 1, our variance bound is as follows when FedRecover runs T rounds. As T increases, the difference bound decreases exponentially. In addition, when τ = ∞, the computation/communication cost of FedRecover is linear with T. Therefore, the difference bound decreases exponentially with the increase of cost . In other words, we observe an accuracy-cost trade-off for FedRecover, i.e., the global model recovered by FedRecover is more accurate (i.e., closer to the train-from-scratch global model) when more cost is introduced for clients.

5 experiments

5.1 Experimental setup

1) Datasets : We consider multiple datasets for different learning tasks in our evaluation. Specifically, we use two image classification datasets (MNIST and Fashion MNIST), a Purchase style prediction dataset (Purchase), and a human activity recognition dataset (HAR). Unless otherwise stated, we show experimental results on MNIST for simplicity.

MNIST : MNIST is a 10-digit image classification dataset containing 60,000 training images and 10,000 testing images. The height and width of the image are both 28. We adopt the convolutional neural network (CNN) in NN as the global model architecture. In particular, CNNs consist of two convolutional layers and two fully connected layers, each followed by a pooling layer. We assume 100 customers and distribute training images to them using the method in , where the method has a parameter called non-iid degree, which ranges between 0.1 and 1. The client's local training data is non-iid when the degree of non-iid is greater than 0.1, and more non-iid when the degree of non-iid is larger. By default, we set the non-iid level to 0.5 when distributing training images to clients, but we will explore its impact on FedRecover.

Fashion MNIST : Fashion MNIST is another 10-class image classification dataset. Unlike MNIST, which contains digital images, Fashion MNIST contains 70,000 fashion images. The dataset is split into 60,000 training images and 10,000 testing images, where each image has a size of 28×28. We adopt the same CNN as MNIST. In addition, we also assume 100 customers, and we set the default degree of non-iid to 0.5 when distributing training images to them.

Purchases : Purchases is a retail dataset released by [1]. The task is to predict the buying style to which a customer belongs.
The dataset contains a total of 197,324 purchase records, where each record has 600 binary features and belongs to one of 100 imbalanced classes. The dataset is divided into 180,000 training records and 17,324 testing records. Following [29], we adopt a fully connected neural network with one hidden layer as the global model architecture, where the number of neurons in the hidden layer is 1024 and the activation function is Tanh. Let's also assume that there are 100 customers in total. Following [29], we equally distribute the training records among them.

Human Activity Recognition (HAR) : HAR [4] is a 6-category human activity recognition dataset. The dataset is collected from the smartphones of 30 real-world users. Each data sample consists of 561 features representing signals collected from multiple sensors of the user's smartphone and belonging to one of 6 possible activities (e.g., walking, sitting, and standing).
We consider each user in the dataset as a client. Furthermore, following [11], we use 75% of each customer's data as local training data and the remaining 25% as testing data. We adopt a fully connected neural network with two hidden layers as the global model architecture, where each hidden layer consists of 256 neurons, and use ReLU as the activation function.

2) FL Setup : Recall that the initial FL training has three steps in each round. We consider clients using stochastic gradient descent to compute model updates. Considering the different features in the dataset, we adopt the following parameter settings for the original FL training: for MNIST and Fashion MNIST, we train for 2000 rounds with a learning rate of 3 × 10−4 and a batch size of 32; for procurement, we train for 1000 rounds, The learning rate is 1 × 10−4 and the batch size is 2000; for HAR, we train for 1000 epochs with a learning rate of 3 × 10− we consider three aggregation rules: FedAvg [24], median [36] and pruned average [36 ]. We do not consider Krum because it is neither accurate nor reliable, nor FLTrust because it requires an additional clean dataset from the server. We set the pruning parameter k = n × 20% in the pruned mean of all datasets. In particular, k is 20, 20, 20, and 6 for the MNIST, Fashion MNIST, and HAR datasets, respectively.

3) Attack settings : By default, we randomly select 20% of clients as malicious clients. Specifically, the numbers of malicious clients for MNIST, Fashion MNIST, Purchase and HAR datasets are 20, 20, 20 and 6, respectively. Furthermore, we assume that the attacker performs a full-knowledge attack. We consider a Trim attack (a poisoning attack without a target) and a Backdoor attack (a poisoning attack with a target). We adopted the default parameter settings for the pruning attack in [18]. We follow the design of the triggers in the backdoor attack. In particular, for MNIST and Fashion MNIST, we adopt the same white pixel located in the bottom right corner as a trigger. For purchase and HAR, we set every 20th feature value to 0 as a trigger. We choose 0 as the target label for all datasets. In a backdoor attack, each malicious client extends its malicious model updates. We set the scaling factor of MNIST to 10, and the scaling factor of Fashion MNIST and HAR to 5, because with these settings, the backdoor attack can achieve a high attack success rate. We note that the purchased attack success rate is similar when the scaling factor varies from 1 to 100. Therefore, we set the scaling factor to 1 to make the purchase more concealed. Malicious clients perform trim attacks or backdoor attacks in each round of original FL training. Furthermore, when some malicious clients are not detected, they execute attacks during every warm-up, periodic correction, anomaly fix, and final round of tuning during the recovery process.

4) Restoration settings : When restoring the global model, we adopt the same settings as the original FL training, including the total number of rounds, learning rate, batch size and aggregation rules. FedRecover has the following parameters: warm-up cycle number Tw, correction cycle Tc, abnormal threshold τ and final adjustment cycle number Tf. By default, we set Tw=20, Tc=10, tolerance rate α=1×10−6 to select the threshold τ, and Tf=5. We use the L-BFGS algorithm with a buffer size of 2 (i.e., s = 2) and adopt the public implementation in [34]. Unless otherwise stated, we assume that all malicious clients are detected. However, we will explore the impact of false negative rate (FNR) and false positive rate (FPR) in malicious client detection on model recovery.
5) Comparison Methods: We compare FedRecover with two baseline methods:

Train-from-scratch : Train-from-scratch will remove detected malicious clients and then retrain the global model from scratch using the remaining clients following standard FL. By default, we assume that clients update their local models using a mini-batch in a global loop. However, we will also explore the impact of the local minibatch size.

History information only: Another baseline is to restore the global model using only server-stored history information. Specifically, the server first initializes the restored global model. It then uses the stored original model updates of the remaining clients to update the recovered global model in each round of the recovery process.
These two baseline methods represent two extreme cases of model recovery, where training from scratch involves re-maintaining the client in each round of the recovery process, while historical information does not involve the client at all.
In other words, the training from scratch brings the largest communication cost to the customer, while the historical information only brings zero cost to the customer.
6) Evaluation Metrics : We use Test Error Rate (TER), Attack Success Rate (ASR) and Average Cost Saving Percentage (ACP) as evaluation metrics. We define them as follows:

Test Error Rate (TER) : Given a test dataset and a (recovered or original) global model, TER is the fraction of the test input that the global model incorrectly predicts.
Attack Success Rate (ASR): For backdoor attacks, we also use ASR to evaluate the global model. Given a test dataset, we first exclude test inputs whose ground truth labels are the target labels. Then, ASR is defined as predicting the part of the remaining input with the target label when the backdoor trigger is embedded. We say that the recovery method is more accurate if the recovered global model has a smaller TER (and ASR for backdoor attacks).

Average Cost Saving Percentage (ACP) : We use ACP to measure the computation/communication cost savings of a restoration method compared to training from scratch. Specifically, the total number of rounds in the recovery process is T, i.e. each client computes the exact model updates in T rounds from scratch. For a given client, we denote by Tr the number of rounds the client is asked to compute and communicate its accurate model updates in the recovery method. Then, we define the customer's cost saving percentage (CP) as (T − Tr)\/T×100%. Our ACP is defined as the customer's average cost savings percentage. The recycling method is more efficient if the ACP is larger.

5.2 Experimental results

FedRecover is accurate and efficient : Figure 1 shows TER, ASR, and ACP trained from scratch, providing only historical information, and FedRecover for four datasets, three aggregation rules, and two attacks. We observe that FedRecover is both accurate and efficient in recovering the global model from the poisoned model. In particular, FedRecover can achieve similar TER and ASR by training from scratch. In addition, FedRecover can achieve a large ACP, that is, FedRecover can significantly reduce the computing/communication cost of the client. The historical information does not cost the client (i.e. ACP of 100), but the global model it recovers has a large TER (almost random guess).

Effect of the number of malicious clients : Figure 2 shows the effect of the number of malicious clients on recovery from a Trim attack. Figure 13 in the appendix shows the results of recovery from the rear door studs. We observe that when different numbers of clients are malicious, FedRecover can recover an accurate global model as trained from scratch, i.e., FedRecover's TERs (and ASR) are close to those trained from scratch. In addition, FedRecover can save most of the costs for customers compared to training from scratch. For example, when the aggregation rule is Trimmerd-mean and the number of malicious clients is 40, FedRecover saves 88% cost for clients on average.

Effect of non-iid degree : Figure 3 shows the effect of non-iid on client-side local training data on recovery from Trim attack. The results of recovery from the backdoor attack are shown in Figure 15 in the Appendix. We observe that FedRecover can recover global models as accurate as those trained from scratch for various degrees of non-iid. Both FedRecover and de novo trained TERs are relatively large when the non-iid degree is increased to 0.9. This is because FedRecover and training from scratch do not change the aggregation rules, and their performance depends on the aggregation rules. When the degree of non-iid is very large, the aggregation rules themselves are inaccurate even without poisoning attacks. When recovering from pruning attacks, the ACP of FedRecover decreases with increasing non-iid degree. This is because when the degree of non-iid is greater, the estimated model updates are more likely to be anomalous, leading to more frequent anomalous fixes and thus lower ACP.

Effect of warm-up rounds Tw : Figure 4 shows the effect of Tw on FedRecover when recovering from two attacks. We observed that TER and ASR remained stable while ACP decreased as the number of warm-up sessions increased. Our results show that a small number of warm-up matches is enough for FedRecover to recover the global model accurately and efficiently.

Effect of Correction Period Tc : Figure 5 shows the effect of Tc on FedRecover when recovering after two attacks. We observe that Tc controls the trade-off between accuracy and efficiency. Specifically, ACP increased with the increase of Tc, but the growth rate of ACP became smaller with the increase of Tc. When the correction period Tc is small, such as Tc≤10, TER and ASR remain basically unchanged. And when Tc is greater than a certain threshold, TER begins to increase. Our results show that Tc ≈ 10 is sufficient for FedRecover to achieve a good trade-off between accuracy and efficiency.

Effect of Tolerance Rate α: Figure 6 shows the effect of α on FedRecover. Recall that α determines the anomaly threshold τ. Smaller α leads to larger threshold τ. We observe that α controls the trade-off between accuracy and efficiency of FedRecover. In other words, when α is large, FedRecover saves less cost for clients, but also produces lower TER. Specifically, with increasing α, ACP decreased, while TER decreased slightly.

Effect of the number of final tuning rounds on Tf : Figure 7 shows the effect of Tf on FedRecover when recovering from two attacks. We observed that TER and ASR remained stable while ACP decreased slightly as the number of final tuning rounds increased. We note that although Tf does not show a large impact in Figure 7, it is necessary to achieve good accuracy in some other scenarios. For example, when the dataset is Purchase and the aggregation rule is trim@mean, the TER without final tuning is 18%, while the TER with Tf = 5 is 13%.
Figure 20 in the appendix shows more details. Our results show that a small number of final tuning rounds is sufficient for FedRecover to recover the global model accurately and efficiently.

Role of False Negative Rate (FNR) and False Positive Rate (FPR) in Malicious Client Detection : In practice, malicious client detection is not always perfect. For example, some malicious clients may escape detection, and some benign clients may be falsely detected as malicious clients.
We define FNR as the proportion of malicious clients that are not detected and FPR as the proportion of benign clients that are falsely detected as malicious. We explored the effect of FNR and FPR on model recovery. Figure 8 shows the results when recovering the global model from the Trim attack. Note that when malicious clients are asked to compute precise model updates during recovery, undetected malicious clients still perform the attack. We observe that even with non-zero FNR or FPR, FedRecover can still recover a global model as accurate as the train from scratch. In particular, the TER curve of FedRecover almost coincides with that of the de novo train, unless the FNR of FedAvg is large (eg, FNR ≥ 0.4). When FNR or FPR is in the range of 0 ~ 0.5, the acp of FedRecover is stable. Our results show that FedRecover can save clients substantial cost even when the malicious client detector has non-zero FNR or FPR.

Training from scratch with multiple local mini-batches per global round : An intuitive way to reduce the communication cost of training from scratch is to require clients to train their local models for l > 1 mini-batches per global round. In our default setup, we set l = 1.
Figure 9a shows the convergence speed of the train from zero under different l, and it can be seen that when l is larger, the number of global rounds (i.e., the communication cost) required for the train from zero to converge is indeed less. We say that a global model has converged in the global round when TER has decreased by no more than 0.1% in the past 20 rounds of global rounds. Figure 9b shows the average number of global rounds per client required to aggregate trains from head to head (train from head to tail) using different l and FedRecover. We observe that when l is smaller than a certain threshold (e.g., l≤30), training from scratch requires more global rounds (i.e., more communication cost) than FedRecover. When l is further increased, train from scratch requires fewer global rounds than FedRecover to converge. However, as shown in Figure 9c, when l increases, training from scratch imposes more computational cost on the client. Specifically, as l grows, the average number of local training mini-batches per client increases substantially. For example, when l = 50, train-from scratch reduces the communication cost by 35%, but the computational cost of the client is more than 30× compared with FedRecover. Our results show that when l is small, FedRecover is less expensive to communicate and compute than training from scratch, and when l is large, FedRecover is much less computationally expensive but slightly more expensive to communicate.

Distribution of Customer Cost Saving Percentage (CP) : We demonstrated in previous experiments that FedRecover can save customers on average. However, this is not desirable if some customers are saving a significantly lower percentage of costs than others. Therefore, we further investigated the distribution of cost saving rate (CP) among customers. Figure 10a shows the results of recovering from backdoor attacks when using Trimmed-mean as the aggregation rule. We observed little variation between the cp of individual clients. Specifically, all cps are in the small range between 88.5% and 88.9%.

Adaptive attack : If the attacker knows to use FedRecover to recover the global model, he can adapt his attack. For example, an attacker could use an undetected malicious client for an adaptive attack during recovery. We note that the Trim attack solves the same optimization problem regardless of the number of malicious clients. Therefore, during recovery, the attack strategy against untargeted attacks is already optimal. However, an attacker can perform an adaptive backdoor attack by adjusting the scale factor of the backdoor attack. Specifically, assuming m' malicious clients are not detected, and the original scaling factor is λ, the attacker can increase the scaling factor to λ m m', keeping the sum of the scaling factors on the malicious clients constant . Figure 10b shows the results on the MNIST dataset when the FNR variation for detecting malicious clients is used as the aggregation rule with Trimmed-mean. We observe that an adaptive backdoor attack can slightly increase FedRecover's TER with an increase in FNR. However, ASR is still low and ACP is still high.

Compared to fine-tuning : Fine-tuning assumes that the server has access to a clean dataset and uses it to fine-tune the poisoned global model.
Figure 11 shows the effect of the number of fine-tuning examples on the MNIST dataset. The fine-tuning examples are sampled uniformly and randomly from the MNIST training set. We fine-tune a toxic global model for 100 epochs at the same learning rate to train the global model. We observe that fine-tuning requires a large number of clean examples, e.g., 1000 examples, to achieve TER and ASR comparable to federcover. In Figure 11, we assume that the fine-tuning dataset has the same distribution as the whole training dataset. Figure 22 in the Appendix shows the results when the fine-tuning dataset is distributed differently than the full training dataset. In particular, we assume that the fine-tuning dataset contains 1000 examples and the 10 classes follow a Dirichlet distribution characterized by the parameter β. β→∞ means that the distribution among the 10 classes is uniform, that is, it is consistent with the distribution of the entire training data set.
The smaller the β, the more the fine-tuning data set distribution deviates from the overall training data distribution. We observe that fine-tuning has larger TER (i.e. less accurate global model) when the fine-tuning dataset deviates from the overall training data distribution. Our results show that even if the server can collect a clean dataset, fine-tuning is not enough when the clean dataset is small or deviates from the overall training data distribution.
More experiments : We also evaluate FedRecover without approximate local model update, which shows that ap approximate local model update is necessary for FedRecover. See Appendix b for details. Appendix Table III shows that all four optimization strategies of FedRecover are necessary.

6 Discussion and limitations

6.1 Security/privacy concerns for storing historical information

In Federrecover, the server stores historical information about clients, including their model updates in each round. A natural question, therefore, is whether stored historical information introduces additional security/privacy concerns for clients. In our threat model, we assume that the server has not been compromised by an attacker and that historical information stored therein introduces no additional security/privacy issues. Also, even though the server could be compromised by an attacker, whether FedRecover creates additional security/privacy concerns for customers depends on when the server was committed. If the server is compromised before training, then storing the history creates no additional security/privacy issues for the client, since the attacker can access the history regardless of whether the server stores it or not. However, we acknowledge that storing historical information may create additional security/privacy concerns for clients if the server is compromised after training. We believe that investigating additional security/privacy risks in this case is interesting future work.

6.2 Client exit

In this work, we focus on recovering the global model when some malicious clients are detected and deleted by the server. In practice, benign clients may also drop out of the FL system after the global model is trained for many reasons, such as privacy concerns. In particular, dropout clients may want the global model to forget what it has learned from their private local training data, or even their existence. By treating exiting benign clients as detected "malicious" clients, we can use FedRecover to recover the global model after benign clients exit. Our Corollary 1 shows that in some scenarios, the recovered global model will be identical to the global model created from scratch, implying that the recovered global model forgets the existence of exiting benign clients and preserves their privacy. We think it is an interesting future work to investigate the privacy guarantees of the recovery global model for benign clients exiting under other scenarios.

6.3 Server Storage and Computing Costs

FedRecover will bring additional storage and computing costs to the server. Suppose a local/global model has M parameters. The server requires O(nMT) additional storage space to hold the original model update and the global model, where n is the number of clients and T is the number of global rounds. For example, when there are 1 million clients, each client participates in an average of 100 global rounds, and the global model is ResNet-20, the server requires about 100 TB of additional storage. In our experiments, FedRecover required up to 200 GB of additional storage on our servers. We note that this storage can be a hard drive instead of main memory.
Furthermore, the server needs roughly O((n−m)T) model updates, where m is the number of malicious clients. The complexity of model update is estimated to be O(M2 s), where s < M is the buffer size. Therefore, the total additional computational cost of the server is O((n−m)TM2 s). For powerful servers, such as modern data centers, storage and compute costs are acceptable.

7 Conclusions and future work

In this work, we propose a model recovery method called FedRecover to remove the impact of poisoning attacks on the global model in FL. Our theoretical and empirical results show that historical information collected by the server while training a poisoned global model before malicious clients are detected is valuable for efficiently recovering an accurate global model after malicious clients are detected. An interesting future work is to explore the accuracy and efficiency of FedRecover under adaptive poisoning attacks. Specifically, an adaptive poisoning attack can be designed for the end-to-end FL pipeline, including training the global model, detecting malicious clients, and recovering the global model. Another interesting direction for future work is to extend FedRecover to other domains of FL (e.g., graphs).

FedRecover: Recovering from Poisoning Attacks inFederated Learning using Historical Information