How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks

• URL: http://arxiv.org/abs/2305.15587v1
• Date: Wed, 24 May 2023 21:52:13 GMT
• Title: How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks
• Authors: Salijona Dyrmishi, Salah Ghamizi, Maxime Cordy
• Abstract summary: adversarial attacks are malicious algorithms that imperceptibly modify input text to force models into making incorrect predictions. We surveyed 378 human participants about the perceptibility of text adversarial examples produced by state-of-the-art methods. Our results underline that existing text attacks are impractical in real-world scenarios where humans are involved.
• Score: 4.297786261992324
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Natural Language Processing (NLP) models based on Machine Learning (ML) are susceptible to adversarial attacks -- malicious algorithms that imperceptibly modify input text to force models into making incorrect predictions. However, evaluations of these attacks ignore the property of imperceptibility or study it under limited settings. This entails that adversarial perturbations would not pass any human quality gate and do not represent real threats to human-checked NLP systems. To bypass this limitation and enable proper assessment (and later, improvement) of NLP model robustness, we have surveyed 378 human participants about the perceptibility of text adversarial examples produced by state-of-the-art methods. Our results underline that existing text attacks are impractical in real-world scenarios where humans are involved. This contrasts with previous smaller-scale human studies, which reported overly optimistic conclusions regarding attack success. Through our work, we hope to position human perceptibility as a first-class success criterion for text attacks, and provide guidance for research to build effective attack algorithms and, in turn, design appropriate defence mechanisms.

Related papers

• Semantic Stealth: Adversarial Text Attacks on NLP Using Several Methods [0.0]
  A text adversarial attack involves the deliberate manipulation of input text to mislead the predictions of the model. BERT, BERT-on-BERT attack, and Fraud Bargain's Attack (FBA) are explored in this paper. PWWS emerges as the most potent adversary, consistently outperforming other methods across multiple evaluation scenarios.
  arXiv  Detail & Related papers  (2024-04-08T02:55:01Z)
• SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation [56.622250514119294]
  In contrast to white-box adversarial attacks, transfer attacks are more reflective of real-world scenarios. We propose a self-augment-based transfer attack method, termed SA-Attack.
  arXiv  Detail & Related papers  (2023-12-08T09:08:50Z)
• TextDefense: Adversarial Text Detection based on Word Importance Entropy [38.632552667871295]
  We propose TextDefense, a new adversarial example detection framework for NLP models. Our experiments show that TextDefense can be applied to different architectures, datasets, and attack methods. We provide our insights into the adversarial attacks in NLP and the principles of our defense method.
  arXiv  Detail & Related papers  (2023-02-12T11:12:44Z)
• Rethinking Textual Adversarial Defense for Pre-trained Language Models [79.18455635071817]
  A literature review shows that pre-trained language models (PrLMs) are vulnerable to adversarial attacks. We propose a novel metric (Degree of Anomaly) to enable current adversarial attack approaches to generate more natural and imperceptible adversarial examples. We show that our universal defense framework achieves comparable or even higher after-attack accuracy with other specific defenses.
  arXiv  Detail & Related papers  (2022-07-21T07:51:45Z)
• A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks [72.7373468905418]
  We develop an open-source toolkit OpenBackdoor to foster the implementations and evaluations of textual backdoor learning. We also propose CUBE, a simple yet strong clustering-based defense baseline.
  arXiv  Detail & Related papers  (2022-06-17T02:29:23Z)
• Perturbations in the Wild: Leveraging Human-Written Text Perturbations for Realistic Adversarial Attack and Defense [19.76930957323042]
  ANTHRO inductively extracts over 600K human-written text perturbations in the wild and leverages them for realistic adversarial attack. We find that adversarial texts generated by ANTHRO achieve the best trade-off between (1) attack success rate, (2) semantic preservation of the original text, and (3) stealthiness--i.e. indistinguishable from human writings.
  arXiv  Detail & Related papers  (2022-03-19T16:00:01Z)
• Adversarial Robustness of Neural-Statistical Features in Detection of Generative Transformers [6.209131728799896]
  We evaluate neural and non-neural approaches on their ability to detect computer-generated text. We find that while statistical features underperform neural features, statistical features provide additional adversarial robustness. We pioneer the usage of $Delta$MAUVE as a proxy measure for human judgement of adversarial text quality.
  arXiv  Detail & Related papers  (2022-03-02T16:46:39Z)
• Contrasting Human- and Machine-Generated Word-Level Adversarial Examples for Text Classification [12.750016480098262]
  We report on crowdsourcing studies in which we task humans with iteratively modifying words in an input text. We analyze how human-generated adversarial examples compare to the recently proposed TextFooler, Genetic, BAE and SememePSO attack algorithms.
  arXiv  Detail & Related papers  (2021-09-09T16:16:04Z)
• Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution [83.84968082791444]
  Deep neural networks are vulnerable to intentionally crafted adversarial examples. Various methods have been proposed to defend against adversarial word-substitution attacks for neural NLP models.
  arXiv  Detail & Related papers  (2021-08-29T08:11:36Z)
• Learning to Attack: Towards Textual Adversarial Attacking in Real-world Situations [81.82518920087175]
  Adversarial attacking aims to fool deep neural networks with adversarial examples. We propose a reinforcement learning based attack model, which can learn from attack history and launch attacks more efficiently.
  arXiv  Detail & Related papers  (2020-09-19T09:12:24Z)
• Perceptual Adversarial Robustness: Defense Against Unseen Threat Models [58.47179090632039]
  A key challenge in adversarial robustness is the lack of a precise mathematical characterization of human perception. Under the neural perceptual threat model, we develop novel perceptual adversarial attacks and defenses. Because the NPTM is very broad, we find that Perceptual Adrial Training (PAT) against a perceptual attack gives robustness against many other types of adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-22T22:40:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.