Run-Off Election: Improved Provable Defense against Data Poisoning Attacks

• URL: http://arxiv.org/abs/2302.02300v3
• Date: Tue, 16 May 2023 04:18:08 GMT
• Title: Run-Off Election: Improved Provable Defense against Data Poisoning Attacks
• Abstract summary: In data poisoning attacks, an adversary tries to change a model's prediction by adding, modifying, or removing samples in the training data. We show that merely considering the majority vote in ensemble defenses is wasteful as it does not effectively utilize available information. We propose Run-Off Election (ROE), a novel aggregation method based on a two-round election across the base models.
• Score: 43.10561893357565
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In data poisoning attacks, an adversary tries to change a model's prediction by adding, modifying, or removing samples in the training data. Recently, ensemble-based approaches for obtaining provable defenses against data poisoning have been proposed where predictions are done by taking a majority vote across multiple base models. In this work, we show that merely considering the majority vote in ensemble defenses is wasteful as it does not effectively utilize available information in the logits layers of the base models. Instead, we propose Run-Off Election (ROE), a novel aggregation method based on a two-round election across the base models: In the first round, models vote for their preferred class and then a second, Run-Off election is held between the top two classes in the first round. Based on this approach, we propose DPA+ROE and FA+ROE defense methods based on Deep Partition Aggregation (DPA) and Finite Aggregation (FA) approaches from prior work. We evaluate our methods on MNIST, CIFAR-10, and GTSRB and obtain improvements in certified accuracy by up to 3%-4%. Also, by applying ROE on a boosted version of DPA, we gain improvements around 12%-27% comparing to the current state-of-the-art, establishing a new state-of-the-art in (pointwise) certified robustness against data poisoning. In many cases, our approach outperforms the state-of-the-art, even when using 32 times less computational power.

Related papers

• The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage [71.8564105095189]
  We introduce N-Gram Coverage Attack, a membership inference attack that relies solely on text outputs from the target model.<n>We first demonstrate on a diverse set of existing benchmarks that N-Gram Coverage Attack outperforms other black-box methods.<n>We find that more recent models, such as GPT-4o, exhibit increased robustness to membership inference.
  arXiv  Detail & Related papers  (2025-08-13T08:35:16Z)
• SoK: Benchmarking Poisoning Attacks and Defenses in Federated Learning [21.73177249075515]
  Federated learning (FL) enables collaborative model training while preserving data privacy, but its decentralized nature exposes it to client-side data poisoning attacks (DPAs) and model poisoning attacks (MPAs) This paper aims to provide a unified benchmark and analysis of defenses against DPAs and MPAs, clarifying the distinction between these two similar but slightly distinct domains.
  arXiv  Detail & Related papers  (2025-02-06T06:05:00Z)
• A Whole-Process Certifiably Robust Aggregation Method Against Backdoor Attacks in Federated Learning [11.239604882889498]
  Federated Learning (FL) has garnered widespread adoption across various domains such as finance, healthcare, and cybersecurity. FL remains under significant threat from backdoor attacks, wherein malicious actors insert triggers into trained models. We propose a novel whole-process certifiably robust aggregation (WPCRA) method for FL, which enhances robustness against backdoor attacks.
  arXiv  Detail & Related papers  (2024-06-30T15:00:49Z)
• Diffence: Fencing Membership Privacy With Diffusion Models [14.633898825111828]
  Deep learning models are vulnerable to membership inference attacks (MIAs) We introduce a novel defense framework against MIAs by leveraging generative models. Our defense, called DIFFENCE, works pre inference, which is unlike prior defenses that are either training-time or post-inference time.
  arXiv  Detail & Related papers  (2023-12-07T20:45:09Z)
• Alternating Objectives Generates Stronger PGD-Based Adversarial Attacks [78.2700757742992]
  Projected Gradient Descent (PGD) is one of the most effective and conceptually simple algorithms to generate such adversaries. We experimentally verify this assertion on a synthetic-data example and by evaluating our proposed method across 25 different $ell_infty$-robust models and 3 datasets. Our strongest adversarial attack outperforms all of the white-box components of AutoAttack ensemble.
  arXiv  Detail & Related papers  (2022-12-15T17:44:31Z)
• Lethal Dose Conjecture on Data Poisoning [122.83280749890078]
  Data poisoning considers an adversary that distorts the training set of machine learning algorithms for malicious purposes. In this work, we bring to light one conjecture regarding the fundamentals of data poisoning, which we call the Lethal Dose Conjecture.
  arXiv  Detail & Related papers  (2022-08-05T17:53:59Z)
• Robust Trajectory Prediction against Adversarial Attacks [84.10405251683713]
  Trajectory prediction using deep neural networks (DNNs) is an essential component of autonomous driving systems. These methods are vulnerable to adversarial attacks, leading to serious consequences such as collisions. In this work, we identify two key ingredients to defend trajectory prediction models against adversarial attacks.
  arXiv  Detail & Related papers  (2022-07-29T22:35:05Z)
• Defending against the Label-flipping Attack in Federated Learning [5.769445676575767]
  Federated learning (FL) provides autonomy and privacy by design to participating peers. The label-flipping (LF) attack is a targeted poisoning attack where the attackers poison their training data by flipping the labels of some examples. We propose a novel defense that first dynamically extracts those gradients from the peers' local updates.
  arXiv  Detail & Related papers  (2022-07-05T12:02:54Z)
• Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregation [122.83280749890078]
  We propose an improved certified defense against general poisoning attacks, namely Finite Aggregation. In contrast to DPA, which directly splits the training set into disjoint subsets, our method first splits the training set into smaller disjoint subsets. We offer an alternative view of our method, bridging the designs of deterministic and aggregation-based certified defenses.
  arXiv  Detail & Related papers  (2022-02-05T20:08:58Z)
• Voting based ensemble improves robustness of defensive models [82.70303474487105]
  We study whether it is possible to create an ensemble to further improve robustness. By ensembling several state-of-the-art pre-trained defense models, our method can achieve a 59.8% robust accuracy.
  arXiv  Detail & Related papers  (2020-11-28T00:08:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.