CrashSight: A Phase-Aware, Infrastructure-Centric Video Benchmark for Traffic Crash Scene Understanding and Reasoning

• URL: http://arxiv.org/abs/2604.08457v2
• Date: Fri, 10 Apr 2026 14:52:27 GMT
• Title: CrashSight: A Phase-Aware, Infrastructure-Centric Video Benchmark for Traffic Crash Scene Understanding and Reasoning
• Abstract summary: We present textbfCrashSight, a vision-language benchmark for roadway crash understanding using real-world roadside camera data.<n>The dataset comprises 250 crash videos, annotated with 13K multiple-choice question-answer pairs organized under a two-tier taxonomy.<n>We benchmark 8 state-of-the-art VLMs and show that, despite strong scene description capabilities, current models struggle with temporal and causal reasoning in safety-critical scenarios.
• Score: 27.23760411917563
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cooperative autonomous driving requires traffic scene understanding from both vehicle and infrastructure perspectives. While vision-language models (VLMs) show strong general reasoning capabilities, their performance in safety-critical traffic scenarios remains insufficiently evaluated due to the ego-vehicle focus of existing benchmarks. To bridge this gap, we present \textbf{CrashSight}, a large-scale vision-language benchmark for roadway crash understanding using real-world roadside camera data. The dataset comprises 250 crash videos, annotated with 13K multiple-choice question-answer pairs organized under a two-tier taxonomy. Tier 1 evaluates the visual grounding of scene context and involved parties, while Tier 2 probes higher-level reasoning, including crash mechanics, causal attribution, temporal progression, and post-crash outcomes. We benchmark 8 state-of-the-art VLMs and show that, despite strong scene description capabilities, current models struggle with temporal and causal reasoning in safety-critical scenarios. We provide a detailed analysis of failure scenarios and discuss directions for improving VLM crash understanding. The benchmark provides a standardized evaluation framework for infrastructure-assisted perception in cooperative autonomous driving. The CrashSight benchmark, including the full dataset and code, is accessible at https://mcgrche.github.io/crashsight.

Related papers

• CAViAR: A Causal Video Dataset for Fine-Grained Accident Reasoning in Real-World Scenarios [10.195587813770075]
  We introduce CAViAR (Causal Accident Video and Incident Analysis Repository), a human-annotated dashcam benchmark comprising 2,249 real-world accident videos.<n>Each video is annotated with structured labels spanning environmental conditions, accident type, causal explanation, apparent At-Fault Agent, affected agent, and apparent rule-violation category.<n>We benchmark state-of-the-art vision-language models (VLMs), including Cosmos-Reason2, Qwen3-VL, and InternVL3.
  arXiv  Detail & Related papers  (2026-08-19T18:51:21Z)
• ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness [65.3988208273595]
  We introduce textbfObsDriveBench, a real-world multi-modal benchmark for adverse-weather autonomous driving.<n>Our benchmark is designed with three capability dimensions: textbfobservability awareness, textbfspatial reliability, and textbfrisk-aware decision-making.<n>Experiments reveal consistent performance degradation of existing vision-language models.
  arXiv  Detail & Related papers  (2026-07-26T08:13:46Z)
• AUTOPILOT VQA: Benchmarking Vision-Language Models for Incident-Centric Dashcam Understanding [12.229867730747996]
  We present AUTOPILOT-VQA, an incident-centric visual question answering benchmark for dashcam video understanding.<n>The dataset evaluates different systems through structured questions designed around real-world driving incidents and near-incidents.<n>Our benchmark support developments for more interpretable, robust, and safety-conscious vision-language systems for real-world autonomous driving.
  arXiv  Detail & Related papers  (2026-07-09T17:46:24Z)
• Search-based Testing of Vision Language Models for In-Car Scene Understanding [2.9682153664947317]
  In the automotive domain, in-car scene understanding (ISU) enables the detection of safety-critical events.<n>We present ISU-Test, an automated testing approach that combines rendering-based scene generation with search-based testing to evaluate ISU systems.
  arXiv  Detail & Related papers  (2026-07-02T15:17:44Z)
• ACCIDENT: A Benchmark Dataset for Vehicle Accident Detection from Traffic Surveillance Videos [0.0]
  ACCIDENT is a benchmark dataset for traffic accident detection in CCTV footage.<n>It is designed to evaluate models in supervised (IID and OOD) and zero-shot settings.
  arXiv  Detail & Related papers  (2026-04-10T18:51:19Z)
• How Well Do Vision-Language Models Understand Sequential Driving Scenes? A Sensitivity Study [2.6954666679827137]
  VENUSS is a framework for systematic sensitivity analysis of VLM performance on sequential driving scenes.<n>It extracts temporal sequences from driving videos, and generates structured evaluations across custom categories.<n>We show how even top models achieve only 57% accuracy, not matching human performance in similar constraints.
  arXiv  Detail & Related papers  (2026-04-08T07:14:55Z)
• SAVeD: A First-Person Social Media Video Dataset for ADAS-equipped vehicle Near-Miss and Crash Event Analyses [0.7874708385247353]
  This paper introduces SAVeD, a large-scale video dataset curated from publicly available social media content.<n>SAVED features 2,119 first-person videos, capturing ADAS vehicle operations in diverse locations, lighting conditions, and weather scenarios.<n>The dataset includes video frame-level annotations for collisions, evasive maneuvers, and disengagements, enabling analysis of both perception and decision-making failures.
  arXiv  Detail & Related papers  (2025-12-19T15:58:52Z)
• Hierarchical Reasoning with Vision-Language Models for Incident Reports from Dashcam Videos [0.03598453624340711]
  We present a hierarchical reasoning framework for incident report generation from dashcam videos.<n>We integrate frame-level captioning, incident frame detection, and fine-grained reasoning within vision-language models.<n>On the official 2COOOL open leaderboard, our method ranks 2nd among 29 teams and achieves the best CIDEr-D score.
  arXiv  Detail & Related papers  (2025-10-14T06:36:41Z)
• AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond [101.20320617562321]
  AccidentBench is a large-scale benchmark that combines vehicle accident scenarios with Beyond domains.<n>The benchmark contains approximately 2000 videos and over 19000 human-annotated question-answer pairs.
  arXiv  Detail & Related papers  (2025-09-30T17:59:13Z)
• Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives [56.528835143531694]
  We introduce DriveBench, a benchmark dataset designed to evaluate Vision-Language Models (VLMs)<n>Our findings reveal that VLMs often generate plausible responses derived from general knowledge or textual cues rather than true visual grounding.<n>We propose refined evaluation metrics that prioritize robust visual grounding and multi-modal understanding.
  arXiv  Detail & Related papers  (2025-01-07T18:59:55Z)
• OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mapping [84.65114565766596]
  We present OpenLane-V2, the first dataset on topology reasoning for traffic scene structure. OpenLane-V2 consists of 2,000 annotated road scenes that describe traffic elements and their correlation to the lanes. We evaluate various state-of-the-art methods, and present their quantitative and qualitative results on OpenLane-V2 to indicate future avenues for investigating topology reasoning in traffic scenes.
  arXiv  Detail & Related papers  (2023-04-20T16:31:22Z)
• DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Driving [76.29141888408265]
  We propose a large-scale dataset containing diverse accident scenarios that frequently occur in real-world driving. The proposed DeepAccident dataset includes 57K annotated frames and 285K annotated samples, approximately 7 times more than the large-scale nuScenes dataset.
  arXiv  Detail & Related papers  (2023-04-03T17:37:00Z)
• Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark [77.54411007883962]
  We propose a Cognitive Accident Prediction (CAP) method that explicitly leverages human-inspired cognition of text description on the visual observation and the driver attention to facilitate model training. CAP is formulated by an attentive text-to-vision shift fusion module, an attentive scene context transfer module, and the driver attention guided accident prediction module. We construct a new large-scale benchmark consisting of 11,727 in-the-wild accident videos with over 2.19 million frames.
  arXiv  Detail & Related papers  (2022-12-19T11:43:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.