Towards Robust Federated Learning via Logits Calibration on Non-IID Data
- URL: http://arxiv.org/abs/2403.02803v1
- Date: Tue, 5 Mar 2024 09:18:29 GMT
- Title: Towards Robust Federated Learning via Logits Calibration on Non-IID Data
- Authors: Yu Qiao, Apurba Adhikary, Chaoning Zhang, Choong Seon Hong
- Abstract summary: Federated learning (FL) is a privacy-preserving distributed management framework based on collaborative model training of distributed devices in edge networks.
Recent studies have shown that FL is vulnerable to adversarial examples, leading to a significant drop in its performance.
In this work, we adopt the adversarial training (AT) framework to improve the robustness of FL models against adversarial example (AE) attacks.
- Score: 49.286558007937856
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Federated learning (FL) is a privacy-preserving distributed management
framework based on collaborative model training of distributed devices in edge
networks. However, recent studies have shown that FL is vulnerable to
adversarial examples (AEs), leading to a significant drop in its performance.
Meanwhile, the non-independent and identically distributed (non-IID) challenge
of data distribution between edge devices can further degrade the performance
of models. Consequently, both AEs and non-IID pose challenges to deploying
robust learning models at the edge. In this work, we adopt the adversarial
training (AT) framework to improve the robustness of FL models against
adversarial example (AE) attacks, which can be termed as federated adversarial
training (FAT). Moreover, we address the non-IID challenge by implementing a
simple yet effective logits calibration strategy under the FAT framework, which
can enhance the robustness of models when subjected to adversarial attacks.
Specifically, we employ a direct strategy to adjust the logits output by
assigning higher weights to classes with small samples during training. This
approach effectively tackles the class imbalance in the training data, with the
goal of mitigating biases between local and global models. Experimental results
on three dataset benchmarks, MNIST, Fashion-MNIST, and CIFAR-10 show that our
strategy achieves competitive results in natural and robust accuracy compared
to several baselines.
Related papers
- On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairness [16.595935469099306]
We propose FLAME, an optimization framework by utilizing the alternating direction method of multipliers (ADMM) to train personalized and global models.
Our theoretical analysis establishes the global convergence and two kinds of convergence rates for FLAME under mild assumptions.
Our experimental findings show that FLAME outperforms state-of-the-art methods in convergence and accuracy, and it achieves higher test accuracy under various attacks.
arXiv Detail & Related papers (2024-07-23T11:35:42Z) - Reinforcement Learning as a Catalyst for Robust and Fair Federated
Learning: Deciphering the Dynamics of Client Contributions [6.318638597489423]
Reinforcement Federated Learning (RFL) is a novel framework that leverages deep reinforcement learning to adaptively optimize client contribution during aggregation.
In terms of robustness, RFL outperforms state-of-the-art methods, while maintaining comparable levels of fairness.
arXiv Detail & Related papers (2024-02-08T10:22:12Z) - Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
We propose a novel doubly-robust instance reweighted adversarial framework.
Our importance weights are obtained by optimizing the KL-divergence regularized loss function.
Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
arXiv Detail & Related papers (2023-08-01T06:16:18Z) - Alleviating the Effect of Data Imbalance on Adversarial Training [26.36714114672729]
We study adversarial training on datasets that obey the long-tailed distribution.
We propose a new adversarial training framework -- Re-balancing Adversarial Training (REAT)
arXiv Detail & Related papers (2023-07-14T07:01:48Z) - Combating Exacerbated Heterogeneity for Robust Models in Federated
Learning [91.88122934924435]
Combination of adversarial training and federated learning can lead to the undesired robustness deterioration.
We propose a novel framework called Slack Federated Adversarial Training (SFAT)
We verify the rationality and effectiveness of SFAT on various benchmarked and real-world datasets.
arXiv Detail & Related papers (2023-03-01T06:16:15Z) - Delving into the Adversarial Robustness of Federated Learning [41.409961662754405]
In Federated Learning (FL), models are as fragile as centrally trained models against adversarial examples.
We propose a novel algorithm called Decision Boundary based Federated Adversarial Training (DBFAT) to improve both accuracy and robustness of FL systems.
arXiv Detail & Related papers (2023-02-19T04:54:25Z) - RelaxLoss: Defending Membership Inference Attacks without Losing Utility [68.48117818874155]
We propose a novel training framework based on a relaxed loss with a more achievable learning target.
RelaxLoss is applicable to any classification model with added benefits of easy implementation and negligible overhead.
Our approach consistently outperforms state-of-the-art defense mechanisms in terms of resilience against MIAs.
arXiv Detail & Related papers (2022-07-12T19:34:47Z) - Self-Damaging Contrastive Learning [92.34124578823977]
Unlabeled data in reality is commonly imbalanced and shows a long-tail distribution.
This paper proposes a principled framework called Self-Damaging Contrastive Learning to automatically balance the representation learning without knowing the classes.
Our experiments show that SDCLR significantly improves not only overall accuracies but also balancedness.
arXiv Detail & Related papers (2021-06-06T00:04:49Z) - Adversarial Distributional Training for Robust Deep Learning [53.300984501078126]
Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples.
Most existing AT methods adopt a specific attack to craft adversarial examples, leading to the unreliable robustness against other unseen attacks.
In this paper, we introduce adversarial distributional training (ADT), a novel framework for learning robust models.
arXiv Detail & Related papers (2020-02-14T12:36:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.