When, Where, and What? A Novel Benchmark for Accident Anticipation and Localization with Large Language Models

• URL: http://arxiv.org/abs/2407.16277v2
• Date: Fri, 26 Jul 2024 06:00:08 GMT
• Title: When, Where, and What? A Novel Benchmark for Accident Anticipation and Localization with Large Language Models
• Authors: Haicheng Liao, Yongkang Li, Chengyue Wang, Yanchen Guan, KaHou Tam, Chunlin Tian, Li Li, Chengzhong Xu, Zhenning Li,
• Abstract summary: This study introduces a novel framework that integrates Large Language Models (LLMs) to enhance predictive capabilities across multiple dimensions. We develop an innovative chain-based attention mechanism that dynamically adjusts to prioritize high-risk elements within complex driving scenes. Empirical validation on the DAD, CCD, and A3D datasets demonstrates superior performance in Average Precision (AP) and Mean Time-To-Accident (mTTA)
• Score: 14.090582912396467
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: As autonomous driving systems increasingly become part of daily transportation, the ability to accurately anticipate and mitigate potential traffic accidents is paramount. Traditional accident anticipation models primarily utilizing dashcam videos are adept at predicting when an accident may occur but fall short in localizing the incident and identifying involved entities. Addressing this gap, this study introduces a novel framework that integrates Large Language Models (LLMs) to enhance predictive capabilities across multiple dimensions--what, when, and where accidents might occur. We develop an innovative chain-based attention mechanism that dynamically adjusts to prioritize high-risk elements within complex driving scenes. This mechanism is complemented by a three-stage model that processes outputs from smaller models into detailed multimodal inputs for LLMs, thus enabling a more nuanced understanding of traffic dynamics. Empirical validation on the DAD, CCD, and A3D datasets demonstrates superior performance in Average Precision (AP) and Mean Time-To-Accident (mTTA), establishing new benchmarks for accident prediction technology. Our approach not only advances the technological framework for autonomous driving safety but also enhances human-AI interaction, making predictive insights generated by autonomous systems more intuitive and actionable.

Related papers

• Real-time Accident Anticipation for Autonomous Driving Through Monocular Depth-Enhanced 3D Modeling [18.071748815365005]
  We introduce an innovative framework, AccNet, which significantly advances the prediction capabilities beyond the current state-of-the-art (SOTA) 2D-based methods. We propose the Binary Adaptive Loss for Early Anticipation (BA-LEA) to address the prevalent challenge of skewed data distribution in traffic accident datasets.
  arXiv  Detail & Related papers  (2024-09-02T13:46:25Z)
• CRASH: Crash Recognition and Anticipation System Harnessing with Context-Aware and Temporal Focus Attentions [13.981748780317329]
  Accurately and promptly predicting accidents among surrounding traffic agents from camera footage is crucial for the safety of autonomous vehicles (AVs) This study introduces a novel accident anticipation framework for AVs, termed CRASH. It seamlessly integrates five components: object detector, feature extractor, object-aware module, context-aware module, and multi-layer fusion. Our model surpasses existing top baselines in critical evaluation metrics like Average Precision (AP) and mean Time-To-Accident (mTTA)
  arXiv  Detail & Related papers  (2024-07-25T04:12:49Z)
• Real-Time Anomaly Detection and Reactive Planning with Large Language Models [18.57162998677491]
  Foundation models, e.g., large language models (LLMs), trained on internet-scale data possess zero-shot capabilities. We present a two-stage reasoning framework that incorporates the judgement regarding potential anomalies into a safe control framework. This enables our monitor to improve the trustworthiness of dynamic robotic systems, such as quadrotors or autonomous vehicles.
  arXiv  Detail & Related papers  (2024-07-11T17:59:22Z)
• AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving [68.73885845181242]
  We propose an Automatic Data Engine (AIDE) that automatically identifies issues, efficiently curates data, improves the model through auto-labeling, and verifies the model through generation of diverse scenarios. We further establish a benchmark for open-world detection on AV datasets to comprehensively evaluate various learning paradigms, demonstrating our method's superior performance at a reduced cost.
  arXiv  Detail & Related papers  (2024-03-26T04:27:56Z)
• Towards Safe and Reliable Autonomous Driving: Dynamic Occupancy Set Prediction [12.336412741837407]
  This study introduces a novel method for Dynamic Occupancy Set (DOS) prediction, it effectively combines advanced trajectory prediction networks with a DOS prediction module. The innovative contributions of this study include the development of a novel DOS prediction model specifically tailored for navigating complex scenarios.
  arXiv  Detail & Related papers  (2024-02-29T17:36:39Z)
• RACER: Rational Artificial Intelligence Car-following-model Enhanced by Reality [51.244807332133696]
  This paper introduces RACER, a cutting-edge deep learning car-following model to predict Adaptive Cruise Control (ACC) driving behavior. Unlike conventional models, RACER effectively integrates Rational Driving Constraints (RDCs), crucial tenets of actual driving. RACER excels across key metrics, such as acceleration, velocity, and spacing, registering zero violations.
  arXiv  Detail & Related papers  (2023-12-12T06:21:30Z)
• FBLNet: FeedBack Loop Network for Driver Attention Prediction [75.83518507463226]
  Nonobjective driving experience is difficult to model. In this paper, we propose a FeedBack Loop Network (FBLNet) which attempts to model the driving experience accumulation procedure. Under the guidance of the incremental knowledge, our model fuses the CNN feature and Transformer feature that are extracted from the input image to predict driver attention.
  arXiv  Detail & Related papers  (2022-12-05T08:25:09Z)
• Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving [104.32241082170044]
  We study a new task, safety-aware motion prediction with unseen vehicles for autonomous driving. Unlike the existing trajectory prediction task for seen vehicles, we aim at predicting an occupancy map. Our approach is the first one that can predict the existence of unseen vehicles in most cases.
  arXiv  Detail & Related papers  (2021-09-03T13:33:33Z)
• A model for traffic incident prediction using emergency braking data [77.34726150561087]
  We address the fundamental problem of data scarcity in road traffic accident prediction by training our model on emergency braking events instead of accidents. We present a prototype implementing a traffic incident prediction model for Germany based on emergency braking data from Mercedes-Benz vehicles.
  arXiv  Detail & Related papers  (2021-02-12T18:17:12Z)
• Uncertainty-based Traffic Accident Anticipation with Spatio-Temporal Relational Learning [30.59728753059457]
  Traffic accident anticipation aims to predict accidents from dashcam videos as early as possible. Current deterministic deep neural networks could be overconfident in false predictions. We propose an uncertainty-based accident anticipation model with relational-temporal learning.
  arXiv  Detail & Related papers  (2020-08-01T20:21:48Z)
• The Importance of Prior Knowledge in Precise Multimodal Prediction [71.74884391209955]
  Roads have well defined geometries, topologies, and traffic rules. In this paper we propose to incorporate structured priors as a loss function. We demonstrate the effectiveness of our approach on real-world self-driving datasets.
  arXiv  Detail & Related papers  (2020-06-04T03:56:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.