Responsibility Distribution Estimation in Ego-View Accident Videos with Multimodal Large Language Models

• URL: http://arxiv.org/abs/2607.03591v1
• Date: Fri, 03 Jul 2026 20:23:05 GMT
• Title: Responsibility Distribution Estimation in Ego-View Accident Videos with Multimodal Large Language Models
• Abstract summary: ego-view accident videos represent the driver's visual perspective, making them suitable for reasoning about avoidability and driver responsibility.<n>We introduce responsibility distribution estimation for ego-view traffic accident videos, a new task in which a model predicts the percentage of responsibility assigned to each involved agent.<n>Our findings suggest that ego-centric accident videos provide a promising foundation for socially and legally meaningful multimodal reasoning.
• Score: 2.0339465062586566
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent studies on multimodal traffic accident understanding have mainly relied on infrastructure-camera footage, satellite imagery, or structured crash records. However, such data sources are costly to deploy and maintain at large scale, and they cannot objectively capture what the driver was actually able to observe before the accident. In contrast, ego-view accident videos directly represent the driver's visual perspective, making them suitable for reasoning about avoidability and driver responsibility. In this paper, we introduce responsibility distribution estimation for ego-view traffic accident videos, a new task in which a model predicts the percentage of responsibility assigned to each involved agent. We construct an LLM-assisted responsibility annotation pipeline and fine-tune multimodal large language models under multiple input settings, including raw frames, segmentation-enhanced input, and textual descriptions. Experimental results establish a strong initial benchmark, demonstrating that multimodal LLMs can effectively perform this nuanced, constraint-based reasoning task. Our findings suggest that ego-centric accident videos provide a promising foundation for socially and legally meaningful multimodal reasoning beyond conventional accident classification and explanation tasks.

Related papers

• CAViAR: A Causal Video Dataset for Fine-Grained Accident Reasoning in Real-World Scenarios [10.195587813770075]
  We introduce CAViAR (Causal Accident Video and Incident Analysis Repository), a human-annotated dashcam benchmark comprising 2,249 real-world accident videos.<n>Each video is annotated with structured labels spanning environmental conditions, accident type, causal explanation, apparent At-Fault Agent, affected agent, and apparent rule-violation category.<n>We benchmark state-of-the-art vision-language models (VLMs), including Cosmos-Reason2, Qwen3-VL, and InternVL3.
  arXiv  Detail & Related papers  (2026-08-19T18:51:21Z)
• Seeing Is Not Deciding: Can Multimodal LLMs Act as Effective CEOs? [53.05042283845008]
  We introduce C-SUITEBENCH, a controlled multimodal benchmark for executive business decisions.<n>We place nine models in the role of a chief executive officer and evaluate their decision-making ability.<n>We uncover a multimodal integration paradox: adding visual business information degrades constrained resource allocation for all nine models.
  arXiv  Detail & Related papers  (2026-08-06T10:43:33Z)
• PEDESTRIANQA: A Benchmark for Vision-Language Models on Pedestrian Intention and Trajectory Prediction [17.259627750358046]
  We introduce PedestrianQA, a large-scale video-based dataset that formulates pedestrian intention and trajectory prediction.<n>PedestrianQA expresses richly annotated pedestrian sequences, in natural language, enabling VLMs to learn from visual dynamics, contextual cues, and interactions among traffic agents.<n> Empirical evaluations show that finetuning state-of-the-art VLMs on PedestrianQA significantly improves intention classification, trajectory forecasting accuracy, and the quality of explanatory rationales.
  arXiv  Detail & Related papers  (2026-05-23T12:56:44Z)
• Perception-Aware Multimodal Spatial Reasoning from Monocular Images [57.42071289037214]
  spatial reasoning from monocular images is essential for autonomous driving.<n>Current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception.<n>We propose a perception-aware multimodal reasoning framework that equipsVLMs with explicit object-centric grounding ability.
  arXiv  Detail & Related papers  (2026-03-07T02:05:12Z)
• Advancing Adaptive Multi-Stage Video Anomaly Reasoning: A Benchmark Dataset and Method [96.63801368613177]
  We present a new task that elevates video anomaly analysis from descriptive understanding to structured, multi-stage reasoning.<n>We present a new dataset with 8,641 videos, totaling more than 50,000 samples, making it one of the largest datasets for video anomaly understanding.<n>Building upon the proposed task and dataset, we develop an end-to-end MLLM-based VAR model termed Vad-R1-Plus, which supports adaptive hierarchical reasoning and risk-aware decision making.
  arXiv  Detail & Related papers  (2026-01-15T08:09:04Z)
• Hierarchical Reasoning with Vision-Language Models for Incident Reports from Dashcam Videos [0.03598453624340711]
  We present a hierarchical reasoning framework for incident report generation from dashcam videos.<n>We integrate frame-level captioning, incident frame detection, and fine-grained reasoning within vision-language models.<n>On the official 2COOOL open leaderboard, our method ranks 2nd among 29 teams and achieves the best CIDEr-D score.
  arXiv  Detail & Related papers  (2025-10-14T06:36:41Z)
• Towards Safer and Understandable Driver Intention Prediction [30.136400523083907]
  We introduce the task of interpretability in maneuver prediction before they occur for driver safety.<n>To foster research in interpretable DIP, we curate the DAAD-X, a new multimodal, ego-centric video dataset.<n>Next, we propose Video Concept Bottleneck Model (VCBM), a framework that generates coherent explanations inherently.
  arXiv  Detail & Related papers  (2025-10-10T09:41:25Z)
• LMAD: Integrated End-to-End Vision-Language Model for Explainable Autonomous Driving [58.535516533697425]
  Large vision-language models (VLMs) have shown promising capabilities in scene understanding.<n>We propose a novel vision-language framework tailored for autonomous driving, called LMAD.<n>Our framework emulates modern end-to-end driving paradigms by incorporating comprehensive scene understanding and a task-specialized structure with VLMs.
  arXiv  Detail & Related papers  (2025-08-17T15:42:54Z)
• Causal-Entity Reflected Egocentric Traffic Accident Video Synthesis [78.14763828578904]
  Egocentricly comprehending the causes and effects of car accidents is crucial for the safety of self-driving cars.<n>This work argues that precisely identifying the accident participants and capturing their related behaviors are of critical importance.<n>We propose a novel diffusion model, Causal-VidSyn, for synthesizing egocentric traffic accident videos.
  arXiv  Detail & Related papers  (2025-06-29T14:37:48Z)
• Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives [56.528835143531694]
  We introduce DriveBench, a benchmark dataset designed to evaluate Vision-Language Models (VLMs)<n>Our findings reveal that VLMs often generate plausible responses derived from general knowledge or textual cues rather than true visual grounding.<n>We propose refined evaluation metrics that prioritize robust visual grounding and multi-modal understanding.
  arXiv  Detail & Related papers  (2025-01-07T18:59:55Z)
• Learning Traffic Crashes as Language: Datasets, Benchmarks, and What-if Causal Analyses [76.59021017301127]
  We propose a large-scale traffic crash language dataset, named CrashEvent, summarizing 19,340 real-world crash reports. We further formulate the crash event feature learning as a novel text reasoning problem and further fine-tune various large language models (LLMs) to predict detailed accident outcomes. Our experiments results show that our LLM-based approach not only predicts the severity of accidents but also classifies different types of accidents and predicts injury outcomes.
  arXiv  Detail & Related papers  (2024-06-16T03:10:16Z)
• RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model [22.25903116720301]
  explainability plays a critical role in trustworthy autonomous decision-making. Recent advancements in Multi-Modal Large Language models (MLLMs) have shown promising potential in enhancing the explainability as a driving agent. We present RAG-Driver, a novel retrieval-augmented multi-modal large language model that leverages in-context learning for high-performance, explainable, and generalisable autonomous driving.
  arXiv  Detail & Related papers  (2024-02-16T16:57:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.