Anomaly Localization in Model Gradients Under Backdoor Attacks Against Federated Learning

• URL: http://arxiv.org/abs/2111.14683v1
• Date: Mon, 29 Nov 2021 16:46:01 GMT
• Title: Anomaly Localization in Model Gradients Under Backdoor Attacks Against Federated Learning
• Authors: Zeki Bilgin
• Abstract summary: In this study, we make a deep gradient-level analysis on the expected variations in model gradients under several backdoor attack scenarios. Our main novel finding is that backdoor-induced anomalies in local model updates (weights or gradients) appear in the final layer bias weights of the malicious local models.
• Score: 0.6091702876917281
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inserting a backdoor into the joint model in federated learning (FL) is a recent threat raising concerns. Existing studies mostly focus on developing effective countermeasures against this threat, assuming that backdoored local models, if any, somehow reveal themselves by anomalies in their gradients. However, this assumption needs to be elaborated by identifying specifically which gradients are more likely to indicate an anomaly to what extent under which conditions. This is an important issue given that neural network models usually have huge parametric space and consist of a large number of weights. In this study, we make a deep gradient-level analysis on the expected variations in model gradients under several backdoor attack scenarios against FL. Our main novel finding is that backdoor-induced anomalies in local model updates (weights or gradients) appear in the final layer bias weights of the malicious local models. We support and validate our findings by both theoretical and experimental analysis in various FL settings. We also investigate the impact of the number of malicious clients, learning rate, and malicious data rate on the observed anomaly. Our implementation is publicly available\footnote{\url{ https://github.com/ArcelikAcikKaynak/Federated_Learning.git}}.

Related papers

• Multiple Descents in Unsupervised Learning: The Role of Noise, Domain Shift and Anomalies [14.399035468023161]
  We study the presence of double descent in unsupervised learning, an area that has received little attention and is not yet fully understood. We use synthetic and real data and identify model-wise, epoch-wise, and sample-wise double descent for various applications.
  arXiv  Detail & Related papers  (2024-06-17T16:24:23Z)
• Data-Agnostic Model Poisoning against Federated Learning: A Graph Autoencoder Approach [65.2993866461477]
  This paper proposes a data-agnostic, model poisoning attack on Federated Learning (FL) The attack requires no knowledge of FL training data and achieves both effectiveness and undetectability. Experiments show that the FL accuracy drops gradually under the proposed attack and existing defense mechanisms fail to detect it.
  arXiv  Detail & Related papers  (2023-11-30T12:19:10Z)
• Understanding Deep Gradient Leakage via Inversion Influence Functions [53.1839233598743]
  Deep Gradient Leakage (DGL) is a highly effective attack that recovers private training images from gradient vectors. We propose a novel Inversion Influence Function (I$2$F) that establishes a closed-form connection between the recovered images and the private gradients. We empirically demonstrate that I$2$F effectively approximated the DGL generally on different model architectures, datasets, attack implementations, and perturbation-based defenses.
  arXiv  Detail & Related papers  (2023-09-22T17:26:24Z)
• From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space [13.763716495058294]
  Deep Neural Networks are prone to learning spurious correlations embedded in the training data, leading to potentially biased predictions. This poses risks when deploying these models for high-stake decision-making, such as in medical applications. We present a novel method for model correction on the concept level that explicitly reduces model sensitivity towards biases via gradient penalization.
  arXiv  Detail & Related papers  (2023-08-18T10:07:46Z)
• Are we certain it's anomalous? [57.729669157989235]
  Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations. Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD) HypAD learns self-supervisedly to reconstruct the input signal.
  arXiv  Detail & Related papers  (2022-11-16T21:31:39Z)
• Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning [112.69497636932955]
  Federated learning aims to train models across different clients without the sharing of data for privacy considerations. We study how data heterogeneity affects the representations of the globally aggregated models. We propose sc FedDecorr, a novel method that can effectively mitigate dimensional collapse in federated learning.
  arXiv  Detail & Related papers  (2022-10-01T09:04:17Z)
• Identifying Backdoor Attacks in Federated Learning via Anomaly Detection [31.197488921578984]
  Federated learning is vulnerable to backdoor attacks. This paper proposes an effective defense against the attack by examining shared model updates. We demonstrate through extensive analyses that our proposed methods effectively mitigate state-of-the-art backdoor attacks.
  arXiv  Detail & Related papers  (2022-02-09T07:07:42Z)
• Backdoor Attack and Defense for Deep Regression [23.20365307988698]
  We demonstrate a backdoor attack on a deep neural network used for regression. The backdoor attack is localized based on training-set data poisoning wherein the mislabeled samples are surrounded by correctly labeled ones. We also study the performance of a backdoor defense using gradient-based discovery of local error maximizers.
  arXiv  Detail & Related papers  (2021-09-06T11:58:03Z)
• Can Adversarial Weight Perturbations Inject Neural Backdoors? [22.83199547214051]
  Adversarial machine learning has exposed several security hazards of neural models. We introduce adversarial perturbations in the model weights using a composite loss on the predictions of the original model. Our results show that backdoors can be successfully injected with a very small average relative change in model weight values.
  arXiv  Detail & Related papers  (2020-08-04T18:26:13Z)
• Unbiased Risk Estimators Can Mislead: A Case Study of Learning with Complementary Labels [92.98756432746482]
  We study a weakly supervised problem called learning with complementary labels. We show that the quality of gradient estimation matters more in risk minimization. We propose a novel surrogate complementary loss(SCL) framework that trades zero bias with reduced variance.
  arXiv  Detail & Related papers  (2020-07-05T04:19:37Z)
• Scalable Backdoor Detection in Neural Networks [61.39635364047679]
  Deep learning models are vulnerable to Trojan attacks, where an attacker can install a backdoor during training time to make the resultant model misidentify samples contaminated with a small trigger patch. We propose a novel trigger reverse-engineering based approach whose computational complexity does not scale with the number of labels, and is based on a measure that is both interpretable and universal across different network and patch types. In experiments, we observe that our method achieves a perfect score in separating Trojaned models from pure models, which is an improvement over the current state-of-the art method.
  arXiv  Detail & Related papers  (2020-06-10T04:12:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.