Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Federated Learning

• URL: http://arxiv.org/abs/2108.10241v1
• Date: Mon, 23 Aug 2021 15:29:45 GMT
• Title: Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Federated Learning
• Authors: Virat Shejwalkar, Amir Houmansadr, Peter Kairouz and Daniel Ramage
• Abstract summary: Recent works have indicated that federated learning (FL) is vulnerable to poisoning attacks by compromised clients. We show that these works make a number of unrealistic assumptions and arrive at somewhat misleading conclusions. We perform the first critical analysis of poisoning attacks under practical production FL environments.
• Score: 32.88150721857589
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While recent works have indicated that federated learning (FL) is vulnerable to poisoning attacks by compromised clients, we show that these works make a number of unrealistic assumptions and arrive at somewhat misleading conclusions. For instance, they often use impractically high percentages of compromised clients or assume unrealistic capabilities for the adversary. We perform the first critical analysis of poisoning attacks under practical production FL environments by carefully characterizing the set of realistic threat models and adversarial capabilities. Our findings are rather surprising: contrary to the established belief, we show that FL, even without any defenses, is highly robust in practice. In fact, we go even further and propose novel, state-of-the-art poisoning attacks under two realistic threat models, and show via an extensive set of experiments across three benchmark datasets how (in)effective poisoning attacks are, especially when simple defense mechanisms are used. We correct previous misconceptions and give concrete guidelines that we hope will encourage our community to conduct more accurate research in this space and build stronger (and more realistic) attacks and defenses.

Related papers

• Benchmarking Misuse Mitigation Against Covert Adversaries [80.74502950627736]
  Existing language model safety evaluations focus on overt attacks and low-stakes tasks.<n>We develop Benchmarks for Stateful Defenses (BSD), a data generation pipeline that automates evaluations of covert attacks and corresponding defenses.<n>Our evaluations indicate that decomposition attacks are effective misuse enablers, and highlight stateful defenses as a countermeasure.
  arXiv  Detail & Related papers  (2025-06-06T17:33:33Z)
• A Critical Evaluation of Defenses against Prompt Injection Attacks [95.81023801370073]
  Large Language Models (LLMs) are vulnerable to prompt injection attacks.<n>Several defenses have recently been proposed, often claiming to mitigate these attacks successfully.<n>We argue that existing studies lack a principled approach to evaluating these defenses.
  arXiv  Detail & Related papers  (2025-05-23T19:39:56Z)
• Decoding FL Defenses: Systemization, Pitfalls, and Remedies [16.907513505608666]
  There are no guidelines for evaluating Federated Learning (FL) defenses. We design a comprehensive systemization of FL defenses along three dimensions. We survey 50 top-tier defense papers and identify the commonly used components in their evaluation setups.
  arXiv  Detail & Related papers  (2025-02-03T23:14:02Z)
• Mitigating Malicious Attacks in Federated Learning via Confidence-aware Defense [3.685395311534351]
  Federated Learning (FL) is a distributed machine learning diagram that enables multiple clients to collaboratively train a global model without sharing their private local data. FL systems are vulnerable to attacks that are happening in malicious clients through data poisoning and model poisoning. Existing defense methods typically focus on mitigating specific types of poisoning and are often ineffective against unseen types of attack.
  arXiv  Detail & Related papers  (2024-08-05T20:27:45Z)
• Model Poisoning Attacks to Federated Learning via Multi-Round Consistency [42.132028389365075]
  We propose PoisonedFL, which enforces multi-round consistency among the malicious clients' model updates. Our empirical evaluation on five benchmark datasets shows that PoisonedFL breaks eight state-of-the-art defenses and outperforms seven existing model poisoning attacks.
  arXiv  Detail & Related papers  (2024-04-24T03:02:21Z)
• RECESS Vaccine for Federated Learning: Proactive Defense Against Model Poisoning Attacks [20.55681622921858]
  Model poisoning attacks greatly jeopardize the application of federated learning (FL) In this work, we propose a novel proactive defense named RECESS against model poisoning attacks. Unlike previous methods that score each iteration, RECESS considers clients' performance correlation across multiple iterations to estimate the trust score.
  arXiv  Detail & Related papers  (2023-10-09T06:09:01Z)
• On Practical Aspects of Aggregation Defenses against Data Poisoning Attacks [58.718697580177356]
  Attacks on deep learning models with malicious training samples are known as data poisoning. Recent advances in defense strategies against data poisoning have highlighted the effectiveness of aggregation schemes in achieving certified poisoning robustness. Here we focus on Deep Partition Aggregation, a representative aggregation defense, and assess its practical aspects, including efficiency, performance, and robustness.
  arXiv  Detail & Related papers  (2023-06-28T17:59:35Z)
• Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations [55.2480439325792]
  Federated Learning (FL) facilitates decentralized machine learning model training, preserving data privacy, lowering communication costs, and boosting model performance through diversified data sources. FL faces vulnerabilities such as poisoning attacks, undermining model integrity with both untargeted performance degradation and targeted backdoor attacks. We define a new notion of strong adaptive adversaries, capable of adapting to multiple objectives simultaneously. MESAS is the first defense robust against strong adaptive adversaries, effective in real-world data scenarios, with an average overhead of just 24.37 seconds.
  arXiv  Detail & Related papers  (2023-06-06T11:44:42Z)
• Rethinking Textual Adversarial Defense for Pre-trained Language Models [79.18455635071817]
  A literature review shows that pre-trained language models (PrLMs) are vulnerable to adversarial attacks. We propose a novel metric (Degree of Anomaly) to enable current adversarial attack approaches to generate more natural and imperceptible adversarial examples. We show that our universal defense framework achieves comparable or even higher after-attack accuracy with other specific defenses.
  arXiv  Detail & Related papers  (2022-07-21T07:51:45Z)
• On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks [9.247680268877795]
  We study whether unrealistic adversarial examples can be used to protect models against realistic examples. Our results reveal discrepancies across the use cases, where unrealistic examples can either be as effective as the realistic ones or may offer only limited improvement. We shed light on the patterns that discriminate which unrealistic examples can be used for effective hardening.
  arXiv  Detail & Related papers  (2022-02-07T15:08:10Z)
• Provable Defense Against Delusive Poisoning [64.69220849669948]
  We show that adversarial training can be a principled defense method against delusive poisoning. This implies that adversarial training can be a principled defense method against delusive poisoning.
  arXiv  Detail & Related papers  (2021-02-09T09:19:47Z)
• Guided Adversarial Attack for Evaluating and Enhancing Adversarial Defenses [59.58128343334556]
  We introduce a relaxation term to the standard loss, that finds more suitable gradient-directions, increases attack efficacy and leads to more efficient adversarial training. We propose Guided Adversarial Margin Attack (GAMA), which utilizes function mapping of the clean image to guide the generation of adversaries. We also propose Guided Adversarial Training (GAT), which achieves state-of-the-art performance amongst single-step defenses.
  arXiv  Detail & Related papers  (2020-11-30T16:39:39Z)
• Defending Regression Learners Against Poisoning Attacks [25.06658793731661]
  We introduce a novel Local Intrinsic Dimensionality (LID) based measure called N-LID that measures the local deviation of a given data point's LID with respect to its neighbors. N-LID can distinguish poisoned samples from normal samples and propose an N-LID based defense approach that makes no assumptions of the attacker. We show that the proposed defense mechanism outperforms the state of the art defenses in terms of prediction accuracy (up to 76% lower MSE compared to an undefended ridge model) and running time.
  arXiv  Detail & Related papers  (2020-08-21T03:02:58Z)
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks [65.20660287833537]
  In this paper we propose two extensions of the PGD-attack overcoming failures due to suboptimal step size and problems of the objective function. We then combine our novel attacks with two complementary existing ones to form a parameter-free, computationally affordable and user-independent ensemble of attacks to test adversarial robustness.
  arXiv  Detail & Related papers  (2020-03-03T18:15:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.