The Multimodal Driver Monitoring Database: A Naturalistic Corpus to Study Driver Attention

• URL: http://arxiv.org/abs/2101.04639v1
• Date: Wed, 23 Dec 2020 16:37:17 GMT
• Title: The Multimodal Driver Monitoring Database: A Naturalistic Corpus to Study Driver Attention
• Authors: Sumit Jha, Mohamed F. Marzban, Tiancheng Hu, Mohamed H. Mahmoud, Naofal Al-Dhahir, Carlos Busso
• Abstract summary: A smart vehicle should be able to monitor the actions and behaviors of the human driver to provide critical warnings or intervene when necessary. Recent advancements in deep learning and computer vision have shown great promise in monitoring human behaviors and activities. A vast amount of in-domain data is required to train models that provide high performance in predicting driving related tasks.
• Score: 44.94118128276982
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: A smart vehicle should be able to monitor the actions and behaviors of the human driver to provide critical warnings or intervene when necessary. Recent advancements in deep learning and computer vision have shown great promise in monitoring human behaviors and activities. While these algorithms work well in a controlled environment, naturalistic driving conditions add new challenges such as illumination variations, occlusions and extreme head poses. A vast amount of in-domain data is required to train models that provide high performance in predicting driving related tasks to effectively monitor driver actions and behaviors. Toward building the required infrastructure, this paper presents the multimodal driver monitoring (MDM) dataset, which was collected with 59 subjects that were recorded performing various tasks. We use the Fi- Cap device that continuously tracks the head movement of the driver using fiducial markers, providing frame-based annotations to train head pose algorithms in naturalistic driving conditions. We ask the driver to look at predetermined gaze locations to obtain accurate correlation between the driver's facial image and visual attention. We also collect data when the driver performs common secondary activities such as navigation using a smart phone and operating the in-car infotainment system. All of the driver's activities are recorded with high definition RGB cameras and time-of-flight depth camera. We also record the controller area network-bus (CAN-Bus), extracting important information. These high quality recordings serve as the ideal resource to train various efficient algorithms for monitoring the driver, providing further advancements in the field of in-vehicle safety systems.

Related papers

• Improving automatic detection of driver fatigue and distraction using machine learning [0.0]
  Driver fatigue and distracted driving are important factors in traffic accidents. We present techniques for simultaneously detecting fatigue and distracted driving behaviors using vision-based and machine learning-based approaches.
  arXiv  Detail & Related papers  (2024-01-04T06:33:46Z)
• Visual Saliency Detection in Advanced Driver Assistance Systems [7.455416595124159]
  We present an intelligent system that combines a drowsiness detection system for drivers with a scene comprehension pipeline based on saliency. We employ an innovative biosensor embedded on the car steering wheel to monitor the driver. A dedicated 1D temporal deep convolutional network has been devised to classify the collected PPG time-series.
  arXiv  Detail & Related papers  (2023-07-26T15:41:54Z)
• Vision-Language Models can Identify Distracted Driver Behavior from Naturalistic Videos [29.529768377746194]
  This paper proposes a CLIP-based driver activity recognition approach that identifies driver distraction from naturalistic driving images and videos. Our results show that this framework offers state-of-the-art performance on zero-shot transfer and video-based CLIP for predicting the driver's state on two public datasets.
  arXiv  Detail & Related papers  (2023-06-16T20:02:51Z)
• FBLNet: FeedBack Loop Network for Driver Attention Prediction [75.83518507463226]
  Nonobjective driving experience is difficult to model. In this paper, we propose a FeedBack Loop Network (FBLNet) which attempts to model the driving experience accumulation procedure. Under the guidance of the incremental knowledge, our model fuses the CNN feature and Transformer feature that are extracted from the input image to predict driver attention.
  arXiv  Detail & Related papers  (2022-12-05T08:25:09Z)
• Scalable and Real-time Multi-Camera Vehicle Detection, Re-Identification, and Tracking [58.95210121654722]
  We propose a real-time city-scale multi-camera vehicle tracking system that handles real-world, low-resolution CCTV instead of idealized and curated video streams. Our method is ranked among the top five performers on the public leaderboard.
  arXiv  Detail & Related papers  (2022-04-15T12:47:01Z)
• TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration [31.908276711898548]
  We present a vision-based framework for recognizing secondary driver behaviours based on visual transformers and an augmented feature distribution calibration module. Our framework consistently leads to better recognition rates, surpassing previous state-of-the-art results of the public Drive&Act benchmark on all levels.
  arXiv  Detail & Related papers  (2022-03-02T08:14:06Z)
• Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation [77.60050239225086]
  We propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images. Our approach is fully automatic without any human interaction. We present a multi-task network for VUS parsing and a multi-stream network for VHI parsing.
  arXiv  Detail & Related papers  (2020-12-15T03:03:38Z)
• Driver Drowsiness Classification Based on Eye Blink and Head Movement Features Using the k-NN Algorithm [8.356765961526955]
  This work is to extend the driver drowsiness detection in vehicles using signals of a driver monitoring camera. For this purpose, 35 features related to the driver's eye blinking behavior and head movements are extracted in driving simulator experiments. A concluding analysis of the best performing feature sets yields valuable insights about the influence of drowsiness on the driver's blink behavior and head movements.
  arXiv  Detail & Related papers  (2020-09-28T12:37:38Z)
• DMD: A Large-Scale Multi-Modal Driver Monitoring Dataset for Attention and Alertness Analysis [54.198237164152786]
  Vision is the richest and most cost-effective technology for Driver Monitoring Systems (DMS) The lack of sufficiently large and comprehensive datasets is currently a bottleneck for the progress of DMS development. In this paper, we introduce the Driver Monitoring dataset (DMD), an extensive dataset which includes real and simulated driving scenarios.
  arXiv  Detail & Related papers  (2020-08-27T12:33:54Z)
• Driver Intention Anticipation Based on In-Cabin and Driving Scene Monitoring [52.557003792696484]
  We present a framework for the detection of the drivers' intention based on both in-cabin and traffic scene videos. Our framework achieves a prediction with the accuracy of 83.98% and F1-score of 84.3%.
  arXiv  Detail & Related papers  (2020-06-20T11:56:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.