Looking Inside Out: Anticipating Driver Intent From Videos

• URL: http://arxiv.org/abs/2312.01444v1
• Date: Sun, 3 Dec 2023 16:24:50 GMT
• Title: Looking Inside Out: Anticipating Driver Intent From Videos
• Authors: Yung-chi Kung, Arthur Zhang, Junmin Wang, Joydeep Biswas
• Abstract summary: Driver intention can be leveraged to improve road safety, such as warning surrounding vehicles in the event the driver is attempting a dangerous maneuver. We propose a novel method of utilizing in-cabin and external camera data to improve state-of-the-art (SOTA) performance in predicting future driver actions. Our models predict driver maneuvers more accurately and earlier than existing approaches, with an accuracy of 87.5% and an average prediction time of 4.35 seconds before the maneuver takes place.
• Score: 20.501288763809036
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Anticipating driver intention is an important task when vehicles of mixed and varying levels of human/machine autonomy share roadways. Driver intention can be leveraged to improve road safety, such as warning surrounding vehicles in the event the driver is attempting a dangerous maneuver. In this work, we propose a novel method of utilizing in-cabin and external camera data to improve state-of-the-art (SOTA) performance in predicting future driver actions. Compared to existing methods, our approach explicitly extracts object and road-level features from external camera data, which we demonstrate are important features for predicting driver intention. Using our handcrafted features as inputs for both a transformer and an LSTM-based architecture, we empirically show that jointly utilizing in-cabin and external features improves performance compared to using in-cabin features alone. Furthermore, our models predict driver maneuvers more accurately and earlier than existing approaches, with an accuracy of 87.5% and an average prediction time of 4.35 seconds before the maneuver takes place. We release our model configurations and training scripts on https://github.com/ykung83/Driver-Intent-Prediction

Related papers

• CEMFormer: Learning to Predict Driver Intentions from In-Cabin and External Cameras via Spatial-Temporal Transformers [5.572431452586636]
  We introduce a new framework called Cross-View Episodic Memory Transformer (CEM) CEM employs unified memory representations to learn for an improved driver intention prediction. We propose a novel context-consistency loss that incorporates driving context as an auxiliary supervision signal to improve prediction performance.
  arXiv  Detail & Related papers  (2023-05-13T05:27:36Z)
• Policy Pre-training for End-to-end Autonomous Driving via Self-supervised Geometric Modeling [96.31941517446859]
  We propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pretraining in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only.
  arXiv  Detail & Related papers  (2023-01-03T08:52:49Z)
• FBLNet: FeedBack Loop Network for Driver Attention Prediction [75.83518507463226]
  Nonobjective driving experience is difficult to model. In this paper, we propose a FeedBack Loop Network (FBLNet) which attempts to model the driving experience accumulation procedure. Under the guidance of the incremental knowledge, our model fuses the CNN feature and Transformer feature that are extracted from the input image to predict driver attention.
  arXiv  Detail & Related papers  (2022-12-05T08:25:09Z)
• Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving [104.32241082170044]
  We study a new task, safety-aware motion prediction with unseen vehicles for autonomous driving. Unlike the existing trajectory prediction task for seen vehicles, we aim at predicting an occupancy map. Our approach is the first one that can predict the existence of unseen vehicles in most cases.
  arXiv  Detail & Related papers  (2021-09-03T13:33:33Z)
• Maneuver-Aware Pooling for Vehicle Trajectory Prediction [3.5851903214591663]
  This paper focuses on predicting the behavior of the surrounding vehicles of an autonomous vehicle on highways. We propose a novel pooling strategy to capture the inter-dependencies between the neighbor vehicles. We incorporated the proposed pooling mechanism into a generative encoder-decoder model, and evaluated our method on the public NGSIM dataset.
  arXiv  Detail & Related papers  (2021-04-29T02:12:08Z)
• Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction [71.97877759413272]
  Trajectory prediction is a safety-critical tool for autonomous vehicles to plan and execute actions. Recent methods have achieved strong performances using Multi-Choice Learning objectives like winner-takes-all (WTA) or best-of-many. Our work addresses two key challenges in trajectory prediction, learning outputs, and better predictions by imposing constraints using driving knowledge.
  arXiv  Detail & Related papers  (2021-04-16T17:58:56Z)
• Vehicle trajectory prediction in top-view image sequences based on deep learning method [1.181206257787103]
  Estimating and predicting surrounding vehicles' movement is essential for an automated vehicle and advanced safety systems. A model with low computational complexity is proposed, which is trained by images taken from the road's aerial image. The proposed model can predict the vehicle's future path in any freeway only by viewing the images related to the history of the target vehicle's movement and its neighbors.
  arXiv  Detail & Related papers  (2021-02-02T20:48:19Z)
• IntentNet: Learning to Predict Intention from Raw Sensor Data [86.74403297781039]
  In this paper, we develop a one-stage detector and forecaster that exploits both 3D point clouds produced by a LiDAR sensor as well as dynamic maps of the environment. Our multi-task model achieves better accuracy than the respective separate modules while saving computation, which is critical to reducing reaction time in self-driving applications.
  arXiv  Detail & Related papers  (2021-01-20T00:31:52Z)
• VRUNet: Multi-Task Learning Model for Intent Prediction of Vulnerable Road Users [3.6265173818019947]
  We propose a multi-task learning model to predict pedestrian actions, crossing intent and forecast their future path from video sequences. We have trained the model on naturalistic driving open-source JAAD dataset, which is rich in behavioral annotations and real world scenarios. Experimental results show state-of-the-art performance on JAAD dataset.
  arXiv  Detail & Related papers  (2020-07-10T14:02:25Z)
• Driver Intention Anticipation Based on In-Cabin and Driving Scene Monitoring [52.557003792696484]
  We present a framework for the detection of the drivers' intention based on both in-cabin and traffic scene videos. Our framework achieves a prediction with the accuracy of 83.98% and F1-score of 84.3%.
  arXiv  Detail & Related papers  (2020-06-20T11:56:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.