Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving

• URL: http://arxiv.org/abs/2501.13563v1
• Date: Thu, 23 Jan 2025 11:10:02 GMT
• Title: Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving
• Authors: Lu Wang, Tianyuan Zhang, Yang Qu, Siyuan Liang, Yuwei Chen, Aishan Liu, Xianglong Liu, Dacheng Tao,
• Abstract summary: We take the first step toward designing black-box adversarial attacks specifically targeting vision-language models (VLMs) in autonomous driving systems. We propose Cascading Adversarial Disruption (CAD), which targets low-level reasoning breakdown by generating and injecting semantics. We present Risky Scene Induction, which addresses dynamic adaptation by leveraging a surrogate VLM to understand and construct high-level risky scenarios.
• Score: 65.61999354218628
• License:
• Abstract: Vision-language models (VLMs) have significantly advanced autonomous driving (AD) by enhancing reasoning capabilities; however, these models remain highly susceptible to adversarial attacks. While existing research has explored white-box attacks to some extent, the more practical and challenging black-box scenarios remain largely underexplored due to their inherent difficulty. In this paper, we take the first step toward designing black-box adversarial attacks specifically targeting VLMs in AD. We identify two key challenges for achieving effective black-box attacks in this context: the effectiveness across driving reasoning chains in AD systems and the dynamic nature of driving scenarios. To address this, we propose Cascading Adversarial Disruption (CAD). It first introduces Decision Chain Disruption, which targets low-level reasoning breakdown by generating and injecting deceptive semantics, ensuring the perturbations remain effective across the entire decision-making chain. Building on this, we present Risky Scene Induction, which addresses dynamic adaptation by leveraging a surrogate VLM to understand and construct high-level risky scenarios that are likely to result in critical errors in the current driving contexts. Extensive experiments conducted on multiple AD VLMs and benchmarks demonstrate that CAD achieves state-of-the-art attack effectiveness, significantly outperforming existing methods (+13.43% on average). Moreover, we validate its practical applicability through real-world attacks on AD vehicles powered by VLMs, where the route completion rate drops by 61.11% and the vehicle crashes directly into the obstacle vehicle with adversarial patches. Finally, we release CADA dataset, comprising 18,808 adversarial visual-question-answer pairs, to facilitate further evaluation and research in this critical domain. Our codes and dataset will be available after paper's acceptance.

Related papers

• Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models [53.580928907886324]
  Reasoning-Augmented Conversation is a novel multi-turn jailbreak framework. It reformulates harmful queries into benign reasoning tasks. We show that RACE achieves state-of-the-art attack effectiveness in complex conversational scenarios.
  arXiv  Detail & Related papers  (2025-02-16T09:27:44Z)
• LLM-attacker: Enhancing Closed-loop Adversarial Scenario Generation for Autonomous Driving with Large Language Models [39.139025989575686]
  AClosed-loop adversarial scenario generation framework leveraging large language models (LLMs) adversarial scenario generation methods are developed, in which behaviors of traffic participants are manipulated to induce safety-critical events. LLMs-attacker can create more dangerous scenarios than other methods, and the ADS trained with it achieves a collision rate half that of training with normal scenarios.
  arXiv  Detail & Related papers  (2025-01-27T08:18:52Z)
• Visual Adversarial Attack on Vision-Language Models for Autonomous Driving [34.520523134588345]
  Vision-language models (VLMs) have significantly advanced autonomous driving (AD) by enhancing reasoning capabilities. These models remain highly vulnerable to adversarial attacks. We propose ADvLM, the first visual adversarial attack framework specifically designed for ADVLMs.
  arXiv  Detail & Related papers  (2024-11-27T12:09:43Z)
• Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks [34.40254709148148]
  Pre-trained vision-language models (VLMs) have showcased remarkable performance in image and natural language understanding. Their potential safety and robustness issues raise concerns that adversaries may evade the system and cause these models to generate toxic content through malicious attacks. We present Chain of Attack (CoA), which iteratively enhances the generation of adversarial examples based on the multi-modal semantic update.
  arXiv  Detail & Related papers  (2024-11-24T05:28:07Z)
• Towards Transferable Attacks Against Vision-LLMs in Autonomous Driving with Typography [21.632703081999036]
  Vision-Large-Language-Models (Vision-LLMs) are increasingly being integrated into autonomous driving (AD) systems. We propose to leverage typographic attacks against AD systems relying on the decision-making capabilities of Vision-LLMs.
  arXiv  Detail & Related papers  (2024-05-23T04:52:02Z)
• CANEDERLI: On The Impact of Adversarial Training and Transferability on CAN Intrusion Detection Systems [17.351539765989433]
  A growing integration of vehicles with external networks has led to a surge in attacks targeting their Controller Area Network (CAN) internal bus. As a countermeasure, various Intrusion Detection Systems (IDSs) have been suggested in the literature to prevent and mitigate these threats. Most of these systems rely on data-driven approaches such as Machine Learning (ML) and Deep Learning (DL) models. In this paper, we present CANEDERLI, a novel framework for securing CAN-based IDSs.
  arXiv  Detail & Related papers  (2024-04-06T14:54:11Z)
• Pre-trained Trojan Attacks for Visual Recognition [106.13792185398863]
  Pre-trained vision models (PVMs) have become a dominant component due to their exceptional performance when fine-tuned for downstream tasks. We propose the Pre-trained Trojan attack, which embeds backdoors into a PVM, enabling attacks across various downstream vision tasks. We highlight the challenges posed by cross-task activation and shortcut connections in successful backdoor attacks.
  arXiv  Detail & Related papers  (2023-12-23T05:51:40Z)
• Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models [79.0183835295533]
  We introduce the first benchmark for indirect prompt injection attacks, named BIPIA, to assess the risk of such vulnerabilities. Our analysis identifies two key factors contributing to their success: LLMs' inability to distinguish between informational context and actionable instructions, and their lack of awareness in avoiding the execution of instructions within external content. We propose two novel defense mechanisms-boundary awareness and explicit reminder-to address these vulnerabilities in both black-box and white-box settings.
  arXiv  Detail & Related papers  (2023-12-21T01:08:39Z)
• On Evaluating Adversarial Robustness of Large Vision-Language Models [64.66104342002882]
  We evaluate the robustness of large vision-language models (VLMs) in the most realistic and high-risk setting. In particular, we first craft targeted adversarial examples against pretrained models such as CLIP and BLIP. Black-box queries on these VLMs can further improve the effectiveness of targeted evasion.
  arXiv  Detail & Related papers  (2023-05-26T13:49:44Z)
• Downlink Power Allocation in Massive MIMO via Deep Learning: Adversarial Attacks and Training [62.77129284830945]
  This paper considers a regression problem in a wireless setting and shows that adversarial attacks can break the DL-based approach. We also analyze the effectiveness of adversarial training as a defensive technique in adversarial settings and show that the robustness of DL-based wireless system against attacks improves significantly.
  arXiv  Detail & Related papers  (2022-06-14T04:55:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.