Over-The-Air Federated Learning under Byzantine Attacks
- URL: http://arxiv.org/abs/2205.02949v1
- Date: Thu, 5 May 2022 22:09:21 GMT
- Title: Over-The-Air Federated Learning under Byzantine Attacks
- Authors: Houssem Sifaou and Geoffrey Ye Li
- Abstract summary: Federated learning (FL) is a promising solution to enable many AI applications.
FL allows the clients to participate in the training phase, governed by a central server, without sharing their local data.
One of the main challenges of FL is the communication overhead.
We propose a transmission and aggregation framework to reduce the effect of such attacks.
- Score: 43.67333971183711
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Federated learning (FL) is a promising solution to enable many AI
applications, where sensitive datasets from distributed clients are needed for
collaboratively training a global model. FL allows the clients to participate
in the training phase, governed by a central server, without sharing their
local data. One of the main challenges of FL is the communication overhead,
where the model updates of the participating clients are sent to the central
server at each global training round. Over-the-air computation (AirComp) has
been recently proposed to alleviate the communication bottleneck where the
model updates are sent simultaneously over the multiple-access channel.
However, simple averaging of the model updates via AirComp makes the learning
process vulnerable to random or intended modifications of the local model
updates of some Byzantine clients. In this paper, we propose a transmission and
aggregation framework to reduce the effect of such attacks while preserving the
benefits of AirComp for FL. For the proposed robust approach, the central
server divides the participating clients randomly into groups and allocates a
transmission time slot for each group. The updates of the different groups are
then aggregated using a robust aggregation technique. We extend our approach to
handle the case of non-i.i.d. local data, where a resampling step is added
before robust aggregation. We analyze the convergence of the proposed approach
for both cases of i.i.d. and non-i.i.d. data and demonstrate that the proposed
algorithm converges at a linear rate to a neighborhood of the optimal solution.
Experiments on real datasets are provided to confirm the robustness of the
proposed approach.
Related papers
- Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learning [51.560590617691005]
We investigate whether it is possible to squeeze more juice" out of each cohort than what is possible in a single communication round.
Our approach leads to up to 74% reduction in the total communication cost needed to train a FL model in the cross-device setting.
arXiv Detail & Related papers (2024-06-03T08:48:49Z) - An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets.
Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round.
We introduce FedAF, a novel aggregation-free FL algorithm.
arXiv Detail & Related papers (2024-04-29T05:55:23Z) - Achieving Linear Speedup in Asynchronous Federated Learning with
Heterogeneous Clients [30.135431295658343]
Federated learning (FL) aims to learn a common global model without exchanging or transferring the data that are stored locally at different clients.
In this paper, we propose an efficient federated learning (AFL) framework called DeFedAvg.
DeFedAvg is the first AFL algorithm that achieves the desirable linear speedup property, which indicates its high scalability.
arXiv Detail & Related papers (2024-02-17T05:22:46Z) - Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training.
In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework.
Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
arXiv Detail & Related papers (2023-08-11T09:58:47Z) - FedCME: Client Matching and Classifier Exchanging to Handle Data
Heterogeneity in Federated Learning [5.21877373352943]
Data heterogeneity across clients is one of the key challenges in Federated Learning (FL)
We propose a novel FL framework named FedCME by client matching and classifier exchanging.
Experimental results demonstrate that FedCME performs better than FedAvg, FedProx, MOON and FedRS on popular federated learning benchmarks.
arXiv Detail & Related papers (2023-07-17T15:40:45Z) - Acceleration of Federated Learning with Alleviated Forgetting in Local
Training [61.231021417674235]
Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy.
We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage.
Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
arXiv Detail & Related papers (2022-03-05T02:31:32Z) - Server Averaging for Federated Learning [14.846231685735592]
Federated learning allows distributed devices to collectively train a model without sharing or disclosing the local dataset with a central server.
The improved privacy of federated learning also introduces challenges including higher computation and communication costs.
We propose the server averaging algorithm to accelerate convergence.
arXiv Detail & Related papers (2021-03-22T07:07:00Z) - A Bayesian Federated Learning Framework with Online Laplace
Approximation [144.7345013348257]
Federated learning allows multiple clients to collaboratively learn a globally shared model.
We propose a novel FL framework that uses online Laplace approximation to approximate posteriors on both the client and server side.
We achieve state-of-the-art results on several benchmarks, clearly demonstrating the advantages of the proposed method.
arXiv Detail & Related papers (2021-02-03T08:36:58Z) - Coded Federated Learning [5.375775284252717]
Federated learning is a method of training a global model from decentralized data distributed across client devices.
Our results show that CFL allows the global model to converge nearly four times faster when compared to an uncoded approach.
arXiv Detail & Related papers (2020-02-21T23:06:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.