MAGIC: Mastering Physical Adversarial Generation in Context through Collaborative LLM Agents

• URL: http://arxiv.org/abs/2412.08014v1
• Date: Wed, 11 Dec 2024 01:41:19 GMT
• Title: MAGIC: Mastering Physical Adversarial Generation in Context through Collaborative LLM Agents
• Authors: Yun Xing, Nhat Chung, Jie Zhang, Yue Cao, Ivor Tsang, Yang Liu, Lei Ma, Qing Guo,
• Abstract summary: We reformulate physical adversarial attacks as a one-shot patch-generation problem. Our approach generates adversarial patches through a deep generative model. We propose MAGIC, a novel framework powered by multi-modal LLM agents.
• Score: 18.1558732924808
• License:
• Abstract: Physical adversarial attacks in driving scenarios can expose critical vulnerabilities in visual perception models. However, developing such attacks remains challenging due to diverse real-world backgrounds and the requirement for maintaining visual naturality. Building upon this challenge, we reformulate physical adversarial attacks as a one-shot patch-generation problem. Our approach generates adversarial patches through a deep generative model that considers the specific scene context, enabling direct physical deployment in matching environments. The primary challenge lies in simultaneously achieving two objectives: generating adversarial patches that effectively mislead object detection systems while determining contextually appropriate placement within the scene. We propose MAGIC (Mastering Physical Adversarial Generation In Context), a novel framework powered by multi-modal LLM agents to address these challenges. MAGIC automatically understands scene context and orchestrates adversarial patch generation through the synergistic interaction of language and vision capabilities. MAGIC orchestrates three specialized LLM agents: The adv-patch generation agent (GAgent) masters the creation of deceptive patches through strategic prompt engineering for text-to-image models. The adv-patch deployment agent (DAgent) ensures contextual coherence by determining optimal placement strategies based on scene understanding. The self-examination agent (EAgent) completes this trilogy by providing critical oversight and iterative refinement of both processes. We validate our method on both digital and physical level, \ie, nuImage and manually captured real scenes, where both statistical and visual results prove that our MAGIC is powerful and effectively for attacking wide-used object detection systems.

Related papers

• SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments [29.107550321162122]
  We present the first approach to generate scene-coherent typographic adversarial attacks that mislead advanced vision-language models. Our approach addresses three critical questions: what adversarial text to generate, where to place it within the scene, and how to integrate it seamlessly. Our experiments show that our scene-coherent adversarial text successfully misleads state-of-the-art LVLMs.
  arXiv  Detail & Related papers  (2024-11-28T05:55:13Z)
• Compromising Embodied Agents with Contextual Backdoor Attacks [69.71630408822767]
  Large language models (LLMs) have transformed the development of embodied intelligence. This paper uncovers a significant backdoor security threat within this process. By poisoning just a few contextual demonstrations, attackers can covertly compromise the contextual environment of a black-box LLM.
  arXiv  Detail & Related papers  (2024-08-06T01:20:12Z)
• MirrorCheck: Efficient Adversarial Defense for Vision-Language Models [55.73581212134293]
  We propose a novel, yet elegantly simple approach for detecting adversarial samples in Vision-Language Models. Our method leverages Text-to-Image (T2I) models to generate images based on captions produced by target VLMs. Empirical evaluations conducted on different datasets validate the efficacy of our approach.
  arXiv  Detail & Related papers  (2024-06-13T15:55:04Z)
• Safeguarding Vision-Language Models Against Patched Visual Prompt Injectors [31.383591942592467]
  Vision-language models (VLMs) offer innovative ways to combine visual and textual data for enhanced understanding and interaction. Patch-based adversarial attack is considered the most realistic threat model in physical vision applications. We introduce SmoothVLM, a defense mechanism rooted in smoothing techniques, to protectVLMs from the threat of patched visual prompt injectors.
  arXiv  Detail & Related papers  (2024-05-17T04:19:19Z)
• Adversarial Robustness for Visual Grounding of Multimodal Large Language Models [49.71757071535619]
  Multi-modal Large Language Models (MLLMs) have recently achieved enhanced performance across various vision-language tasks. adversarial robustness of visual grounding remains unexplored in MLLMs. We propose three adversarial attack paradigms as follows.
  arXiv  Detail & Related papers  (2024-05-16T10:54:26Z)
• Empowering Embodied Visual Tracking with Visual Foundation Models and Offline RL [19.757030674041037]
  Embodied visual tracking is a vital and challenging skill for embodied agents. Existing methods suffer from inefficient training and poor generalization. We propose a novel framework that combines visual foundation models and offline reinforcement learning.
  arXiv  Detail & Related papers  (2024-04-15T15:12:53Z)
• Few-Shot Adversarial Prompt Learning on Vision-Language Models [62.50622628004134]
  The vulnerability of deep neural networks to imperceptible adversarial perturbations has attracted widespread attention. Previous efforts achieved zero-shot adversarial robustness by aligning adversarial visual features with text supervision. We propose a few-shot adversarial prompt framework where adapting input sequences with limited data makes significant adversarial robustness improvement.
  arXiv  Detail & Related papers  (2024-03-21T18:28:43Z)
• Mutual-modality Adversarial Attack with Semantic Perturbation [81.66172089175346]
  We propose a novel approach that generates adversarial attacks in a mutual-modality optimization scheme. Our approach outperforms state-of-the-art attack methods and can be readily deployed as a plug-and-play solution.
  arXiv  Detail & Related papers  (2023-12-20T05:06:01Z)
• Effective Unsupervised Domain Adaptation with Adversarially Trained Language Models [54.569004548170824]
  We show that careful masking strategies can bridge the knowledge gap of masked language models. We propose an effective training strategy by adversarially masking out those tokens which are harder to adversarial by the underlying.
  arXiv  Detail & Related papers  (2020-10-05T01:49:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.