ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness

• URL: http://arxiv.org/abs/2607.23537v1
• Date: Sun, 26 Jul 2026 08:13:46 GMT
• Title: ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness
• Abstract summary: We introduce textbfObsDriveBench, a real-world multi-modal benchmark for adverse-weather autonomous driving.<n>Our benchmark is designed with three capability dimensions: textbfobservability awareness, textbfspatial reliability, and textbfrisk-aware decision-making.<n>Experiments reveal consistent performance degradation of existing vision-language models.
• Score: 65.3988208273595
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous driving under adverse weather remains a critical challenge, yet existing vision-language benchmarks mainly evaluate under standard conditions, synthetic corruptions, or single modality. As a result, it remains unclear how vision-language models behave under real-world adverse weather with multi-modal inputs. We argue that a key difficulty lies in degraded environmental observability: under fog, rain, snow, and low illumination, multi-modal observations become unreliable and cross-modally inconsistent, posing challenges to scene understanding, and subsequent decision-making. To study this, we introduce \textbf{ObsDriveBench}, a real-world multi-modal benchmark for adverse-weather autonomous driving. Our benchmark is designed with three capability dimensions: \textbf{observability awareness}, \textbf{spatial reliability}, and \textbf{risk-aware decision-making}, enabling fine-grained diagnosis of model behavior under degraded observations. We construct the benchmark through observability meta-annotation, scene description, and capability oriented multiple-choice tasks over synchronized camera, LiDAR, and radar inputs, forming a benchmark with over 14k training and 13k test questions. Experiments reveal consistent performance degradation of existing vision-language models. We further introduce \textbf{ObsDrive} model with normal-weather supervised fine-tuning and adverse-weather reinforcement learning, improving robustness across all three capabilities. The dataset and evaluation code will be released at \href{https://github.com/russellyq/ObsDriveBench}{\texttt{ObsDriveBench}}.

Related papers

• CRUISE: Vision-Language Model-Guided Uncertainty-Aware Cross-Modal Sensor Fusion for Robust Autonomous Driving [15.316817374554828]
  Cruise is a novel uncertainty-aware cross-modal sensor fusion framework.<n>It integrates a vision-language model (VLM)-guided UQ module that generates fine-grained, pixel-level uncertainty estimates.<n>We introduce a dynamic adaptive mechanism that explicitly models and captures cross-modal dependencies.
  arXiv  Detail & Related papers  (2026-08-10T07:17:00Z)
• EgoInertia-MI: A Multimodal Egocentric Vision and IMU Benchmark for Motor Impairment Assessment [2.7963067040946723]
  EgoInertia-MI is a multimodal benchmark dataset combining egocentric video and wearable IMU signals for motor impairment analysis.<n>We show that egocentric video provides strong cues for motor impairment assessment.<n>These findings highlight the potential of combining egocentric vision and wearable sensing for ecologically valid and privacy-aware motor assessment.
  arXiv  Detail & Related papers  (2026-07-04T16:02:39Z)
• Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games [69.57330692969543]
  We introduce RNG-Bench, a benchmark suite designed to isolate a base model's ability to reconstruct past observations.<n>RNG-Bench includes two complementary games: Matching Pairs, where card identities briefly revealed at specific locations must later be recalled, and 3D Maze, where egocentric views must be integrated into a spatial map.<n>The hardest configurations require contexts of roughly 128K tokens and 350 image inputs per episode, and remain far from saturated by frontier MLLMs.
  arXiv  Detail & Related papers  (2026-06-17T17:59:34Z)
• MM-Snowball: Evaluating and Mitigating Hallucination Snowballing in Multimodal Multi-Turn Dialogue [98.58433916474472]
  We introduce MM-Snowball, the first benchmark for diagnosis of hallucination snowballing within dialogues.<n>We propose Conflict-Aware Visual Rectification (CAVR) to counteract this degradation.
  arXiv  Detail & Related papers  (2026-05-30T08:53:34Z)
• Same Answer, Different Representations: Hidden instability in VLMs [65.36933543377346]
  We introduce a representation-aware and frequency-aware evaluation framework that measures internal embedding drift, spectral sensitivity, and structural smoothness.<n>We apply this framework to modern Vision Language Models (VLMs) across the SEEDBench, MMMU, and POPE datasets.
  arXiv  Detail & Related papers  (2026-02-06T12:24:26Z)
• CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution [20.823419395675412]
  CrossCheck-Bench is a diagnostic benchmark for evaluating contradiction detection in multimodal inputs.<n>We evaluate 13 state-of-the-art vision-language models and observe a consistent performance drop as tasks shift from perceptual matching to logical contradiction detection.
  arXiv  Detail & Related papers  (2025-11-19T12:17:15Z)
• AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond [101.20320617562321]
  AccidentBench is a large-scale benchmark that combines vehicle accident scenarios with Beyond domains.<n>The benchmark contains approximately 2000 videos and over 19000 human-annotated question-answer pairs.
  arXiv  Detail & Related papers  (2025-09-30T17:59:13Z)
• Self-Consistency as a Free Lunch: Reducing Hallucinations in Vision-Language Models via Self-Reflection [71.8243083897721]
  Vision-language models often hallucinate details, generating non-existent objects or inaccurate attributes that compromise output reliability.<n>We present a novel framework that leverages the model's self-consistency between long responses and short answers to generate preference pairs for training.
  arXiv  Detail & Related papers  (2025-09-27T10:37:11Z)
• PRIMEDrive-CoT: A Precognitive Chain-of-Thought Framework for Uncertainty-Aware Object Interaction in Driving Scene Scenario [1.1142444517901016]
  PRIMEDrive-CoT is a novel uncertainty-aware model for object interaction and Chain-of-Thought (CoT) reasoning in driving scenarios.<n>Our approach combines LiDAR-based 3D object detection with multi-view RGB references to ensure interpretable and reliable scene understanding.
  arXiv  Detail & Related papers  (2025-04-08T11:06:02Z)
• MultiCorrupt: A Multi-Modal Robustness Dataset and Benchmark of LiDAR-Camera Fusion for 3D Object Detection [5.462358595564476]
  Multi-modal 3D object detection models for automated driving have demonstrated exceptional performance on computer vision benchmarks like nuScenes. However, their reliance on densely sampled LiDAR point clouds and meticulously calibrated sensor arrays poses challenges for real-world applications. We introduce MultiCorrupt, a benchmark designed to evaluate the robustness of multi-modal 3D object detectors against ten distinct types of corruptions.
  arXiv  Detail & Related papers  (2024-02-18T18:56:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.