Defensive Dual Masking for Robust Adversarial Defense

• URL: http://arxiv.org/abs/2412.07078v1
• Date: Tue, 10 Dec 2024 00:41:25 GMT
• Title: Defensive Dual Masking for Robust Adversarial Defense
• Authors: Wangli Yang, Jie Yang, Yi Guo, Johan Barthelemy,
• Abstract summary: This paper introduces the Defensive Dual Masking (DDM) algorithm, a novel approach designed to enhance model robustness against such attacks. DDM utilizes a unique adversarial training strategy where [MASK] tokens are strategically inserted into training samples to prepare the model to handle adversarial perturbations more effectively. During inference, potentially adversarial tokens are dynamically replaced with [MASK] tokens to neutralize potential threats while preserving the core semantics of the input.
• Score: 5.932787778915417
• License:
• Abstract: The field of textual adversarial defenses has gained considerable attention in recent years due to the increasing vulnerability of natural language processing (NLP) models to adversarial attacks, which exploit subtle perturbations in input text to deceive models. This paper introduces the Defensive Dual Masking (DDM) algorithm, a novel approach designed to enhance model robustness against such attacks. DDM utilizes a unique adversarial training strategy where [MASK] tokens are strategically inserted into training samples to prepare the model to handle adversarial perturbations more effectively. During inference, potentially adversarial tokens are dynamically replaced with [MASK] tokens to neutralize potential threats while preserving the core semantics of the input. The theoretical foundation of our approach is explored, demonstrating how the selective masking mechanism strengthens the model's ability to identify and mitigate adversarial manipulations. Our empirical evaluation across a diverse set of benchmark datasets and attack mechanisms consistently shows that DDM outperforms state-of-the-art defense techniques, improving model accuracy and robustness. Moreover, when applied to Large Language Models (LLMs), DDM also enhances their resilience to adversarial attacks, providing a scalable defense mechanism for large-scale NLP applications.

Related papers

• Protecting Feed-Forward Networks from Adversarial Attacks Using Predictive Coding [0.20718016474717196]
  An adversarial example is a modified input image designed to cause a Machine Learning (ML) model to make a mistake. This study presents a practical and effective solution -- using predictive coding networks (PCnets) as an auxiliary step for adversarial defence.
  arXiv  Detail & Related papers  (2024-10-31T21:38:05Z)
• MirrorCheck: Efficient Adversarial Defense for Vision-Language Models [55.73581212134293]
  We propose a novel, yet elegantly simple approach for detecting adversarial samples in Vision-Language Models. Our method leverages Text-to-Image (T2I) models to generate images based on captions produced by target VLMs. Empirical evaluations conducted on different datasets validate the efficacy of our approach.
  arXiv  Detail & Related papers  (2024-06-13T15:55:04Z)
• Efficient Adversarial Training in LLMs with Continuous Attacks [99.5882845458567]
  Large language models (LLMs) are vulnerable to adversarial attacks that can bypass their safety guardrails. We propose a fast adversarial training algorithm (C-AdvUL) composed of two losses. C-AdvIPO is an adversarial variant of IPO that does not require utility data for adversarially robust alignment.
  arXiv  Detail & Related papers  (2024-05-24T14:20:09Z)
• Improving the Robustness of Transformer-based Large Language Models with Dynamic Attention [43.95101492654236]
  Transformer-based models, such as BERT and GPT, have been widely adopted in natural language processing (NLP) Recent studies show their vulnerability to textual adversarial attacks where the model's output can be misled by intentionally manipulating the text inputs. We propose a novel method called dynamic attention, tailored for the transformer architecture, to enhance the inherent robustness of the model itself against various adversarial attacks.
  arXiv  Detail & Related papers  (2023-11-29T07:09:13Z)
• A Prompting-based Approach for Adversarial Example Generation and Robustness Enhancement [18.532308729844598]
  We propose a novel prompt-based adversarial attack to compromise NLP models. We generate adversarial examples via mask-and-filling under the effect of a malicious purpose. Our training method does not actually generate adversarial samples, it can be applied to large-scale training sets efficiently.
  arXiv  Detail & Related papers  (2022-03-21T03:21:32Z)
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness [53.094682754683255]
  We propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the in adversarial attacks parameterized by a recurrent neural network. We develop a model-agnostic training algorithm to improve the ability of the learned when attacking unseen defenses.
  arXiv  Detail & Related papers  (2021-10-13T13:54:24Z)
• Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
  CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications. We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths. Our method is trained to automatically align features of arbitrary attacking strength.
  arXiv  Detail & Related papers  (2021-05-31T17:01:05Z)
• A Hamiltonian Monte Carlo Method for Probabilistic Adversarial Attack and Learning [122.49765136434353]
  We present an effective method, called Hamiltonian Monte Carlo with Accumulated Momentum (HMCAM), aiming to generate a sequence of adversarial examples. We also propose a new generative method called Contrastive Adversarial Training (CAT), which approaches equilibrium distribution of adversarial examples. Both quantitative and qualitative analysis on several natural image datasets and practical systems have confirmed the superiority of the proposed algorithm.
  arXiv  Detail & Related papers  (2020-10-15T16:07:26Z)
• Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models [82.03536496686763]
  The vulnerability of deep networks to adversarial attacks is a central problem for deep learning from the perspective of both cognition and security. We focus on defending naturally-trained classifiers using Markov Chain Monte Carlo (MCMC) sampling with an Energy-Based Model (EBM) for adversarial purification. Our contributions are 1) an improved method for training EBM's with realistic long-run MCMC samples, 2) Expectation-Over-Transformation (EOT) defense that resolves theoretical ambiguities for defenses, and 3) state-of-the-art adversarial defense for naturally-trained classifiers and competitive defense.
  arXiv  Detail & Related papers  (2020-05-27T17:53:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.