Technical Report on the CVPR 2026@AdvML Workshop Challenge

• URL: http://arxiv.org/abs/2607.11560v1
• Date: Mon, 13 Jul 2026 13:42:48 GMT
• Title: Technical Report on the CVPR 2026@AdvML Workshop Challenge
• Abstract summary: Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning.<n>This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs.
• Score: 212.8018524968403
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning. This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs. Built on DriveLM-style multi-view visual question answering, the challenge represents each scene with six synchronized camera images and a structured collection of driving-related question-answer pairs. Participants generate adversarial images and suffix-only textual perturbations that induce model responses to deviate from reference answers while preserving image fidelity and limiting textual cost. The competition comprises two phases, with Phase II adding a hidden black-box model to assess transferability. We describe the task design, submission rules, evaluation protocol, and leaderboard results, and then examine five leading submissions for which technical reports were available. Across these reports, several recurring patterns emerge: image-side attacks are favored by the suffix penalty; scene-level, multi-view optimization is more effective than treating views in isolation; QA types and graph structure provide useful priors for allocating attack budget; feature-space objectives can improve black-box transfer; and typographic content embedded in camera images exposes a persistent vulnerability in driving VLAs. These findings provide a practical reference for future robustness evaluation and defense design in multimodal autonomous-driving systems.

Related papers

• Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model [73.59026525350808]
  We propose a dual adversarial fine-tuning framework that jointly optimize visual and semantic supervision signals from two modalities.<n>Our approach outperforms the state-of-the-art method in adversarial robustness evaluation across zero-shot classification, image captioning, and visual question answering tasks.
  arXiv  Detail & Related papers  (2026-07-21T10:49:29Z)
• V2XCrafter: Learning to Generate Driving Scene Across Agents [52.22336670088219]
  Collaborative driving systems leverage vehicle-to-everything (V2X) communication for collaborative perception to enhance driving safety.<n>We propose V2XCrafter, the first framework for generating controllable and realistic collaborative driving scene across agents' camera views.<n>Experiments have shown that V2XCrafter can generate high-fidelity and controllable street views with consistency across agents, thereby effectively enhancing the downstream collaborative 3D object detection tasks.
  arXiv  Detail & Related papers  (2026-05-28T07:03:51Z)
• Perception-Aware Multimodal Spatial Reasoning from Monocular Images [57.42071289037214]
  spatial reasoning from monocular images is essential for autonomous driving.<n>Current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception.<n>We propose a perception-aware multimodal reasoning framework that equipsVLMs with explicit object-centric grounding ability.
  arXiv  Detail & Related papers  (2026-03-07T02:05:12Z)
• Multi-view Phase-aware Pedestrian-Vehicle Incident Reasoning Framework with Vision-Language Models [6.117371161379208]
  This paper introduces Multi-view Phase-aware Pedestrian-Vehicle Incident Reasoning (MP-PVIR)<n>MP-PVIR is a unified framework that processes multi-view video streams into structured diagnostic reports through four stages.<n>It operationalizes behavioral theory by automatically segmenting incidents into cognitive phases, performing synchronized multi-view analysis within each phase, and synthesizing results into causal chains with targeted prevention strategies.
  arXiv  Detail & Related papers  (2025-11-18T04:12:24Z)
• ERMV: Editing 4D Robotic Multi-view images to enhance embodied agents [14.75400720374728]
  ERMV ( Robotic Multi-View 4D data framework) efficiently edits an entire multi-view sequence based on single-frame editing and robot state conditions.<n>Emerged data significantly boosts robustness and guidance of models in both simulated and real-world environments.
  arXiv  Detail & Related papers  (2025-07-23T12:41:11Z)
• AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models [39.34959092321762]
  Vision-Language Models (VLMs) are vulnerable to image-based adversarial attacks.<n>We present AnyAttack, a self-supervised framework that transcends the limitations of conventional attacks.
  arXiv  Detail & Related papers  (2024-10-07T09:45:18Z)
• Multimodal Adversarial Defense for Vision-Language Models by Leveraging One-To-Many Relationships [9.059990548158716]
  This work is the first to explore defense strategies against multimodal attacks in vision-language (VL) tasks.<n>We propose multimodal adversarial training (MAT), which incorporates adversarial perturbations in both image and text modalities during training.<n>To address this, we conduct a comprehensive study on leveraging one-to-many relationships to enhance robustness.
  arXiv  Detail & Related papers  (2024-05-29T05:20:02Z)
• Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective [42.04728834962863]
  Pretrained vision-language models (VLMs) like CLIP exhibit exceptional generalization across diverse downstream tasks. Recent studies reveal their vulnerability to adversarial attacks, with defenses against text-based and multimodal attacks remaining largely unexplored. This work presents the first comprehensive study on improving the adversarial robustness of VLMs against attacks targeting image, text, and multimodal inputs.
  arXiv  Detail & Related papers  (2024-04-30T06:34:21Z)
• VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models [58.21452697997078]
  We propose a novel VQAttack model, which can generate both image and text perturbations with the designed modules. Experimental results on two VQA datasets with five validated models demonstrate the effectiveness of the proposed VQAttack.
  arXiv  Detail & Related papers  (2024-02-16T21:17:42Z)
• Adversarial Prompt Tuning for Vision-Language Models [86.5543597406173]
  Adversarial Prompt Tuning (AdvPT) is a technique to enhance the adversarial robustness of image encoders in Vision-Language Models (VLMs) We demonstrate that AdvPT improves resistance against white-box and black-box adversarial attacks and exhibits a synergistic effect when combined with existing image-processing-based defense techniques.
  arXiv  Detail & Related papers  (2023-11-19T07:47:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.