Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective

• URL: http://arxiv.org/abs/2607.05783v1
• Date: Tue, 07 Jul 2026 03:15:00 GMT
• Title: Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective
• Abstract summary: Environmental illusions are naturally existing yet overlooked phenomena in real-world driving environments.<n>They can disturb visual perception, leading to misinterpretation of the scene and posing serious safety risks to autonomous driving systems.<n>We introduce the first benchmark, LanEvil++, for evaluating the robustness of lane perception under environmental illusions.
• Score: 76.02772556944073
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Environmental illusions (eg., shadows, reflections, and tire marks) are naturally existing yet overlooked phenomena in real-world driving environments. They can disturb visual perception, leading to misinterpretation of the scene and posing serious safety risks to autonomous driving (AD) systems. However, existing researches largely overlook these phenomena, leaving a critical gap. To address this issue, we study AD robustness through the lane perception perspective, a fundamental task supporting core functions like cruise control and lane centering. We focus on two representative models: conventional lane detection (LD) and vision-language model-based systems (ADVLMs). In this work, we introduce the first benchmark, LanEvil++, for evaluating the robustness of lane perception under environmental illusions. LanEvil++ encompasses 14 types of illusions and leverages the CARLA simulator to generate 94 high-fidelity, fully controllable 3D scenes, yielding a dataset of 90,292 annotated images, 1,596 video clips, and 41,855 visual question answering pairs. Extensive evaluations demonstrate that environmental illusions substantially degrade the performance of state-of-the-art LD methods. On average, LD models experience a 5.27% drop in Accuracy and a 10.49% decline in F1-score, while ADVLMs show a 2.03% reduction in GPT-score and a 0.75% drop in Language-score. Among all illusions, shadows emerge as the most disruptive factor, reducing accuracy by up to 7.20%. Furthermore, closed-loop simulations reveal that these illusions can lead to incorrect driving decisions. Complementary real-world case studies highlight safety-critical failures in actual traffic scenes. To enhance robustness, we propose the Multimodal Illusion Defense Approach (MIDA). MIDA achieves substantial gains under challenging conditions, boosting robustness by 4.23% on LD models and 3.82% on ADVLMs.

Related papers

• Beyond Localization: A Comprehensive Diagnosis of Perspective-Conditioned Spatial Reasoning in MLLMs from Omnidirectional Images [52.50752250573993]
  Multimodal Large Language Models (MLLMs) show strong visual perception, yet remain limited in reasoning about space under changing viewpoints.<n>We study this challenge as Perspective-Conditioned Spatial Reasoning (PCSR) in 360-degree omnidirectional images.<n>We introduce PCSR-Bench, a diagnostic benchmark of 84,373 question-answer pairs from 2,600 omnidirectional images.
  arXiv  Detail & Related papers  (2026-05-12T17:11:17Z)
• When World Models Dream Wrong: Physical-Conditioned Adversarial Attacks against World Models [54.08784776767683]
  We present Physical-Conditioned World Model Attack (PhysCond-WMA), the first white-box world model attack that perturbs physical-condition channels.<n>PhysCond-WMA induces semantic, logic, or decision-level distortion while preserving perceptual fidelity.
  arXiv  Detail & Related papers  (2026-02-21T07:22:37Z)
• Cross-Paradigm Evaluation of Gaze-Based Semantic Object Identification for Intelligent Vehicles [2.867517731896504]
  This paper tackles this challenge as a semantic identification task from the road scenes captured by a vehicle's front-view camera.<n>Three vision-based approaches are investigated: direct object detection (YOLOv13), segmentation-assisted classification (SAM2 paired with EfficientNetV2 versus YOLOv13), and query-based Vision-Language Models, VLMs.<n>The results demonstrate that the direct object detection (YOLOv13) and Qwen2.5-VL-32b significantly outperform other approaches, achieving Macro F1-Scores over 0.84.
  arXiv  Detail & Related papers  (2026-02-01T21:43:02Z)
• OmniDrive-R1: Reinforcement-driven Interleaved Multi-modal Chain-of-Thought for Trustworthy Vision-Language Autonomous Driving [11.793091014248025]
  We introduce OmniDrive-R1, an end-to-end VLM framework designed for autonomous driving.<n>Our core innovation is an Reinforcement-driven visual grounding capability.<n>This capability is enabled by our pure two-stage reinforcement learning training pipeline and Clip-GRPO algorithm.
  arXiv  Detail & Related papers  (2025-12-16T03:19:28Z)
• Sequence-Preserving Dual-FoV Defense for Traffic Sign and Light Recognition in Autonomous Vehicles [0.07646713951724012]
  This study proposes a dual FoV, sequence-preserving robustness framework for traffic lights and signs in the USA.<n>Over a series of experiments on a real-life application of anomaly detection, this study outlines a unified three-layer defense stack framework.
  arXiv  Detail & Related papers  (2025-10-03T00:43:25Z)
• Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives [56.528835143531694]
  We introduce DriveBench, a benchmark dataset designed to evaluate Vision-Language Models (VLMs)<n>Our findings reveal that VLMs often generate plausible responses derived from general knowledge or textual cues rather than true visual grounding.<n>We propose refined evaluation metrics that prioritize robust visual grounding and multi-modal understanding.
  arXiv  Detail & Related papers  (2025-01-07T18:59:55Z)
• AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language Models [91.78328878860003]
  Large vision-language models (LVLMs) are prone to hallucinations. benchmarks often rely on hand-crafted corner cases whose failure patterns may not generalize well. We develop AutoHallusion, the first automated benchmark generation approach.
  arXiv  Detail & Related papers  (2024-06-16T11:44:43Z)
• LanEvil: Benchmarking the Robustness of Lane Detection to Environmental Illusions [61.87108000328186]
  Lane detection (LD) is an essential component of autonomous driving systems, providing fundamental functionalities like adaptive cruise control and automated lane centering. Existing LD benchmarks primarily focus on evaluating common cases, neglecting the robustness of LD models against environmental illusions. This paper studies the potential threats caused by these environmental illusions to LD and establishes the first comprehensive benchmark LanEvil.
  arXiv  Detail & Related papers  (2024-06-03T02:12:27Z)
• EyeDAS: Securing Perception of Autonomous Cars Against the Stereoblindness Syndrome [36.39622328877822]
  We present EyeDAS, a novel few-shot learning-based method aimed at securing an object detector (OD) for autonomous driving. We evaluate EyeDAS's real-time performance using 2,000 objects extracted from seven YouTube video recordings of street views taken by a dash cam from the driver's seat perspective.
  arXiv  Detail & Related papers  (2022-05-13T16:53:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.