On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks

• URL: http://arxiv.org/abs/2202.03277v2
• Date: Mon, 22 May 2023 02:10:09 GMT
• Title: On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks
• Authors: Salijona Dyrmishi and Salah Ghamizi and Thibault Simonetto and Yves Le Traon and Maxime Cordy
• Abstract summary: We study whether unrealistic adversarial examples can be used to protect models against realistic examples. Our results reveal discrepancies across the use cases, where unrealistic examples can either be as effective as the realistic ones or may offer only limited improvement. We shed light on the patterns that discriminate which unrealistic examples can be used for effective hardening.
• Score: 9.247680268877795
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While the literature on security attacks and defense of Machine Learning (ML) systems mostly focuses on unrealistic adversarial examples, recent research has raised concern about the under-explored field of realistic adversarial attacks and their implications on the robustness of real-world systems. Our paper paves the way for a better understanding of adversarial robustness against realistic attacks and makes two major contributions. First, we conduct a study on three real-world use cases (text classification, botnet detection, malware detection)) and five datasets in order to evaluate whether unrealistic adversarial examples can be used to protect models against realistic examples. Our results reveal discrepancies across the use cases, where unrealistic examples can either be as effective as the realistic ones or may offer only limited improvement. Second, to explain these results, we analyze the latent representation of the adversarial examples generated with realistic and unrealistic attacks. We shed light on the patterns that discriminate which unrealistic examples can be used for effective hardening. We release our code, datasets and models to support future research in exploring how to reduce the gap between unrealistic and realistic adversarial attacks.

Related papers

• DUMB and DUMBer: Is Adversarial Training Worth It in the Real World? [15.469010487781931]
  Adversarial examples are small and often imperceptible perturbations crafted to fool machine learning models.<n>Evasion attacks, a form of adversarial attack where input is modified at test time to cause misclassification, are particularly insidious due to their transferability.<n>We introduce DUMBer, an attack framework built on the foundation of the DUMB methodology to evaluate the resilience of adversarially trained models.
  arXiv  Detail & Related papers  (2025-06-23T11:16:21Z)
• Practical Adversarial Attacks on Stochastic Bandits via Fake Data Injection [5.311665176634655]
  Adversarial attacks on bandits have traditionally relied on some unrealistic assumptions.<n>We propose a more practical threat model, which reflects realistic adversarial constraints.<n>We design efficient attack strategies under this model, explicitly addressing both magnitude constraints and temporal constraints.
  arXiv  Detail & Related papers  (2025-05-28T03:47:13Z)
• Interactive Trimming against Evasive Online Data Manipulation Attacks: A Game-Theoretic Approach [10.822843258077997]
  Malicious data poisoning attacks can disrupt machine learning processes and lead to severe consequences. To mitigate these attacks, distance-based defenses, such as trimming, have been proposed. We present an interactive game-theoretical model to defend online data manipulation attacks using the trimming strategy.
  arXiv  Detail & Related papers  (2024-03-15T13:59:05Z)
• On Practical Aspects of Aggregation Defenses against Data Poisoning Attacks [58.718697580177356]
  Attacks on deep learning models with malicious training samples are known as data poisoning. Recent advances in defense strategies against data poisoning have highlighted the effectiveness of aggregation schemes in achieving certified poisoning robustness. Here we focus on Deep Partition Aggregation, a representative aggregation defense, and assess its practical aspects, including efficiency, performance, and robustness.
  arXiv  Detail & Related papers  (2023-06-28T17:59:35Z)
• The Best Defense is Attack: Repairing Semantics in Textual Adversarial Examples [7.622122513456483]
  We introduce a novel approach named Reactive Perturbation Defocusing (Rapid) Rapid employs an adversarial detector to identify fake labels of adversarial examples and leverage adversarial attackers to repair the semantics in adversarial examples. Our extensive experimental results conducted on four public datasets, convincingly demonstrate the effectiveness of Rapid in various adversarial attack scenarios.
  arXiv  Detail & Related papers  (2023-05-06T15:14:11Z)
• Rethinking Textual Adversarial Defense for Pre-trained Language Models [79.18455635071817]
  A literature review shows that pre-trained language models (PrLMs) are vulnerable to adversarial attacks. We propose a novel metric (Degree of Anomaly) to enable current adversarial attack approaches to generate more natural and imperceptible adversarial examples. We show that our universal defense framework achieves comparable or even higher after-attack accuracy with other specific defenses.
  arXiv  Detail & Related papers  (2022-07-21T07:51:45Z)
• An Equivalence Between Data Poisoning and Byzantine Gradient Attacks [5.601217969637838]
  "Byzantine" literature considers a strong threat model where workers can report arbitrary gradients to a parameter server. We show a surprising equivalence between this model and data poisoning, a threat considered much more realistic.
  arXiv  Detail & Related papers  (2022-02-17T10:53:52Z)
• On the Real-World Adversarial Robustness of Real-Time Semantic Segmentation Models for Autonomous Driving [59.33715889581687]
  The existence of real-world adversarial examples (commonly in the form of patches) poses a serious threat for the use of deep learning models in safety-critical computer vision tasks. This paper presents an evaluation of the robustness of semantic segmentation models when attacked with different types of adversarial patches. A novel loss function is proposed to improve the capabilities of attackers in inducing a misclassification of pixels.
  arXiv  Detail & Related papers  (2022-01-05T22:33:43Z)
• Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Federated Learning [32.88150721857589]
  Recent works have indicated that federated learning (FL) is vulnerable to poisoning attacks by compromised clients. We show that these works make a number of unrealistic assumptions and arrive at somewhat misleading conclusions. We perform the first critical analysis of poisoning attacks under practical production FL environments.
  arXiv  Detail & Related papers  (2021-08-23T15:29:45Z)
• Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attacks [70.11599738647963]
  Adversarial Training is one of the few defenses that withstand strong attacks. Traditional defense mechanisms assume a uniform attack over the examples according to the underlying data distribution. We present a weighted minimax risk optimization that defends against non-uniform attacks.
  arXiv  Detail & Related papers  (2020-10-24T21:20:35Z)
• Learning to Attack: Towards Textual Adversarial Attacking in Real-world Situations [81.82518920087175]
  Adversarial attacking aims to fool deep neural networks with adversarial examples. We propose a reinforcement learning based attack model, which can learn from attack history and launch attacks more efficiently.
  arXiv  Detail & Related papers  (2020-09-19T09:12:24Z)
• Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News [57.9843300852526]
  We introduce the more realistic and challenging task of defending against machine-generated news that also includes images and captions. To identify the possible weaknesses that adversaries can exploit, we create a NeuralNews dataset composed of 4 different types of generated articles. In addition to the valuable insights gleaned from our user study experiments, we provide a relatively effective approach based on detecting visual-semantic inconsistencies.
  arXiv  Detail & Related papers  (2020-09-16T14:13:15Z)
• Detection Defense Against Adversarial Attacks with Saliency Map [7.736844355705379]
  It is well established that neural networks are vulnerable to adversarial examples, which are almost imperceptible on human vision. Existing defenses are trend to harden the robustness of models against adversarial attacks. We propose a novel method combined with additional noises and utilize the inconsistency strategy to detect adversarial examples.
  arXiv  Detail & Related papers  (2020-09-06T13:57:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.