Exploring the Potential of Multi-Modal AI for Driving Hazard Prediction

• URL: http://arxiv.org/abs/2310.04671v4
• Date: Mon, 1 Jul 2024 09:29:39 GMT
• Title: Exploring the Potential of Multi-Modal AI for Driving Hazard Prediction
• Authors: Korawat Charoenpitaks, Van-Quang Nguyen, Masanori Suganuma, Masahiro Takahashi, Ryoma Niihara, Takayuki Okatani,
• Abstract summary: We formulate it as a task of anticipating impending accidents using a single input image captured by car dashcams. The problem needs predicting and reasoning about future events based on uncertain observations. To enable research in this understudied area, a new dataset named the DHPR dataset is created.
• Score: 18.285227911703977
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper addresses the problem of predicting hazards that drivers may encounter while driving a car. We formulate it as a task of anticipating impending accidents using a single input image captured by car dashcams. Unlike existing approaches to driving hazard prediction that rely on computational simulations or anomaly detection from videos, this study focuses on high-level inference from static images. The problem needs predicting and reasoning about future events based on uncertain observations, which falls under visual abductive reasoning. To enable research in this understudied area, a new dataset named the DHPR (Driving Hazard Prediction and Reasoning) dataset is created. The dataset consists of 15K dashcam images of street scenes, and each image is associated with a tuple containing car speed, a hypothesized hazard description, and visual entities present in the scene. These are annotated by human annotators, who identify risky scenes and provide descriptions of potential accidents that could occur a few seconds later. We present several baseline methods and evaluate their performance on our dataset, identifying remaining issues and discussing future directions. This study contributes to the field by introducing a novel problem formulation and dataset, enabling researchers to explore the potential of multi-modal AI for driving hazard prediction.

Related papers

• Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving [68.95178518732965]
  A self-driving vehicle (SDV) must be able to perceive its surroundings and predict the future behavior of other traffic participants. Existing works either perform object detection followed by trajectory of the detected objects, or predict dense occupancy and flow grids for the whole scene. This motivates our unified approach to perception and future prediction that implicitly represents occupancy and flow over time with a single neural network.
  arXiv  Detail & Related papers  (2023-08-02T23:39:24Z)
• DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Driving [76.29141888408265]
  We propose a large-scale dataset containing diverse accident scenarios that frequently occur in real-world driving. The proposed DeepAccident dataset includes 57K annotated frames and 285K annotated samples, approximately 7 times more than the large-scale nuScenes dataset.
  arXiv  Detail & Related papers  (2023-04-03T17:37:00Z)
• Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark [77.54411007883962]
  We propose a Cognitive Accident Prediction (CAP) method that explicitly leverages human-inspired cognition of text description on the visual observation and the driver attention to facilitate model training. CAP is formulated by an attentive text-to-vision shift fusion module, an attentive scene context transfer module, and the driver attention guided accident prediction module. We construct a new large-scale benchmark consisting of 11,727 in-the-wild accident videos with over 2.19 million frames.
  arXiv  Detail & Related papers  (2022-12-19T11:43:02Z)
• Behavioral Intention Prediction in Driving Scenes: A Survey [70.53285924851767]
  Behavioral Intention Prediction (BIP) simulates a human consideration process and fulfills the early prediction of specific behaviors. This work provides a comprehensive review of BIP from the available datasets, key factors and challenges, pedestrian-centric and vehicle-centric BIP approaches, and BIP-aware applications.
  arXiv  Detail & Related papers  (2022-11-01T11:07:37Z)
• Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving [104.32241082170044]
  We study a new task, safety-aware motion prediction with unseen vehicles for autonomous driving. Unlike the existing trajectory prediction task for seen vehicles, we aim at predicting an occupancy map. Our approach is the first one that can predict the existence of unseen vehicles in most cases.
  arXiv  Detail & Related papers  (2021-09-03T13:33:33Z)
• Towards explainable artificial intelligence (XAI) for early anticipation of traffic accidents [8.34084323253809]
  An accident anticipation model aims to predict accidents promptly and accurately before they occur. Existing Artificial Intelligence (AI) models of accident anticipation lack a human-interpretable explanation of their decision-making. This paper presents a Gated Recurrent Unit (RU) network that learns maps-temporal features for the early anticipation of traffic accidents from dashcam video data.
  arXiv  Detail & Related papers  (2021-07-31T15:53:32Z)
• DRIVE: Deep Reinforced Accident Anticipation with Visual Explanation [36.350348194248014]
  Traffic accident anticipation aims to accurately and promptly predict the occurrence of a future accident from dashcam videos. Existing approaches typically focus on capturing the cues of spatial and temporal context before a future accident occurs. We propose Deep ReInforced accident anticipation with Visual Explanation, named DRIVE.
  arXiv  Detail & Related papers  (2021-07-21T16:33:21Z)
• A system of vision sensor based deep neural networks for complex driving scene analysis in support of crash risk assessment and prevention [12.881094474374231]
  This paper develops a system for driving scene analysis using dash cameras on vehicles and deep learning algorithms. The Multi-Net of the system includes two multi-task neural networks that perform scene classification to provide four labels for each scene. Two completely new datasets have been developed and made available to the public, which were proved to be effective in training the proposed deep neural networks.
  arXiv  Detail & Related papers  (2021-06-18T19:07:59Z)
• Vision based Pedestrian Potential Risk Analysis based on Automated Behavior Feature Extraction for Smart and Safe City [5.759189800028578]
  We propose a comprehensive analytical model for pedestrian potential risk using video footage gathered by road security cameras deployed at such crossings. The proposed system automatically detects vehicles and pedestrians, calculates trajectories by frames, and extracts behavioral features affecting the likelihood of potentially dangerous scenes between these objects. We validated feasibility and applicability by applying it in multiple crosswalks in Osan city, Korea.
  arXiv  Detail & Related papers  (2021-05-06T11:03:10Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• Vehicle trajectory prediction in top-view image sequences based on deep learning method [1.181206257787103]
  Estimating and predicting surrounding vehicles' movement is essential for an automated vehicle and advanced safety systems. A model with low computational complexity is proposed, which is trained by images taken from the road's aerial image. The proposed model can predict the vehicle's future path in any freeway only by viewing the images related to the history of the target vehicle's movement and its neighbors.
  arXiv  Detail & Related papers  (2021-02-02T20:48:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.