ViT-DD: Multi-Task Vision Transformer for Semi-Supervised Driver Distraction Detection

• URL: http://arxiv.org/abs/2209.09178v4
• Date: Tue, 6 Feb 2024 17:48:10 GMT
• Title: ViT-DD: Multi-Task Vision Transformer for Semi-Supervised Driver Distraction Detection
• Abstract summary: This paper presents a multi-modal Vision Transformer for Driver Distraction Detection (termed ViT-DD) It incorporates inductive information from training signals related to both distraction detection and driver emotion recognition. The proposed ViT-DD surpasses existing state-of-the-art methods for driver distraction detection by 6.5% and 0.9% on the SFDDD and AUCDD datasets, respectively.
• Score: 5.709028939815015
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Ensuring traffic safety and mitigating accidents in modern driving is of paramount importance, and computer vision technologies have the potential to significantly contribute to this goal. This paper presents a multi-modal Vision Transformer for Driver Distraction Detection (termed ViT-DD), which incorporates inductive information from training signals related to both distraction detection and driver emotion recognition. Additionally, a self-learning algorithm is developed, allowing for the seamless integration of driver data without emotion labels into the multi-task training process of ViT-DD. Experimental results reveal that the proposed ViT-DD surpasses existing state-of-the-art methods for driver distraction detection by 6.5% and 0.9% on the SFDDD and AUCDD datasets, respectively.

Related papers

• UV-M3TL: A Unified and Versatile Multimodal Multi-Task Learning Framework for Assistive Driving Perception [71.19234323863314]
  We propose a framework to simultaneously recognize driver behavior, driver emotion, vehicle behavior, and traffic context.<n>Our framework incorporates two core components: dual-branch spatial channel multimodal embedding and adaptive feature-decoupled multi-task loss.<n>We evaluate our method on the AIDE dataset, and the experimental results demonstrate that UV-M3TL achieves state-of-the-art performance across all four tasks.
  arXiv  Detail & Related papers  (2026-02-02T03:35:24Z)
• ALERT Open Dataset and Input-Size-Agnostic Vision Transformer for Driver Activity Recognition using IR-UWB [9.138582336148065]
  We present the ALERT dataset, which contains 10,220 radar samples of seven distracted driving activities collected in real driving conditions.<n>We also propose the input-size-agnostic Vision Transformer (ISA-ViT), a framework designed for radar-based driver activity recognition (DAR)<n>Experiments demonstrate that ISA-ViT achieves a 22.68% accuracy improvement over an existing ViT-based approach for UWB-based DAR.
  arXiv  Detail & Related papers  (2025-12-13T06:33:02Z)
• A Dynamic Transformer Network for Vehicle Detection [57.4144097001218]
  We present a dynamic Transformer network for vehicle detection (DTNet)<n>DTNet utilizes a dynamic convolution to guide a deep network to dynamically generate weights to enhance adaptability of an obtained detector.<n>To overcome the drawback of difference in an image account, a translation-variant convolution relies on spatial location information to refine obtained structural information for vehicle detection.
  arXiv  Detail & Related papers  (2025-06-03T11:29:35Z)
• VTD: Visual and Tactile Database for Driver State and Behavior Perception [1.6277623188953556]
  We introduce a novel visual-tactile perception method to address subjective uncertainties in driver state and interaction behaviors. A comprehensive dataset has been developed that encompasses multi-modal data under fatigue and distraction conditions.
  arXiv  Detail & Related papers  (2024-12-06T09:31:40Z)
• Cross-Camera Distracted Driver Classification through Feature Disentanglement and Contrastive Learning [13.613407983544427]
  We introduce a robust model designed to withstand changes in camera position within the vehicle. Our Driver Behavior Monitoring Network (DBMNet) relies on a lightweight backbone and integrates a disentanglement module. Experiments conducted on the daytime and nighttime subsets of the 100-Driver dataset validate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-11-20T10:27:12Z)
• Efficient Mixture-of-Expert for Video-based Driver State and Physiological Multi-task Estimation in Conditional Autonomous Driving [12.765198683804094]
  Road safety remains a critical challenge worldwide, with approximately 1.35 million fatalities annually attributed to traffic accidents. We propose a novel multi-task DMS, termed VDMoE, which leverages RGB video input to monitor driver states non-invasively.
  arXiv  Detail & Related papers  (2024-10-28T14:49:18Z)
• Towards Infusing Auxiliary Knowledge for Distracted Driver Detection [11.816566371802802]
  Distracted driving is a leading cause of road accidents globally. We propose KiD3, a novel method for distracted driver detection (DDD) by infusing auxiliary knowledge about semantic relations between entities in a scene and the structural configuration of the driver's pose. Specifically, we construct a unified framework that integrates the scene graphs, and driver pose information with the visual cues in video frames to create a holistic representation of the driver's actions.
  arXiv  Detail & Related papers  (2024-08-29T15:28:42Z)
• Federated Learning for Drowsiness Detection in Connected Vehicles [0.19116784879310028]
  Driver monitoring systems can assist in determining the driver's state. Driver drowsiness detection presents a potential solution. transmitting the data to a central machine for model training is impractical due to the large data size and privacy concerns. We propose a federated learning framework for drowsiness detection within a vehicular network, leveraging the YawDD dataset.
  arXiv  Detail & Related papers  (2024-05-06T09:39:13Z)
• Improving automatic detection of driver fatigue and distraction using machine learning [0.0]
  Driver fatigue and distracted driving are important factors in traffic accidents. We present techniques for simultaneously detecting fatigue and distracted driving behaviors using vision-based and machine learning-based approaches.
  arXiv  Detail & Related papers  (2024-01-04T06:33:46Z)
• G-MEMP: Gaze-Enhanced Multimodal Ego-Motion Prediction in Driving [71.9040410238973]
  We focus on inferring the ego trajectory of a driver's vehicle using their gaze data. Next, we develop G-MEMP, a novel multimodal ego-trajectory prediction network that combines GPS and video input with gaze data. The results show that G-MEMP significantly outperforms state-of-the-art methods in both benchmarks.
  arXiv  Detail & Related papers  (2023-12-13T23:06:30Z)
• Camera-Radar Perception for Autonomous Vehicles and ADAS: Concepts, Datasets and Metrics [77.34726150561087]
  This work aims to carry out a study on the current scenario of camera and radar-based perception for ADAS and autonomous vehicles. Concepts and characteristics related to both sensors, as well as to their fusion, are presented. We give an overview of the Deep Learning-based detection and segmentation tasks, and the main datasets, metrics, challenges, and open questions in vehicle perception.
  arXiv  Detail & Related papers  (2023-03-08T00:48:32Z)
• COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles [54.61668577827041]
  We introduce COOPERNAUT, an end-to-end learning model that uses cross-vehicle perception for vision-based cooperative driving. Our experiments on AutoCastSim suggest that our cooperative perception driving models lead to a 40% improvement in average success rate.
  arXiv  Detail & Related papers  (2022-05-04T17:55:12Z)
• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [59.60483620730437]
  We propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
  arXiv  Detail & Related papers  (2021-04-19T11:48:13Z)
• Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation [77.60050239225086]
  We propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images. Our approach is fully automatic without any human interaction. We present a multi-task network for VUS parsing and a multi-stream network for VHI parsing.
  arXiv  Detail & Related papers  (2020-12-15T03:03:38Z)
• DMD: A Large-Scale Multi-Modal Driver Monitoring Dataset for Attention and Alertness Analysis [54.198237164152786]
  Vision is the richest and most cost-effective technology for Driver Monitoring Systems (DMS) The lack of sufficiently large and comprehensive datasets is currently a bottleneck for the progress of DMS development. In this paper, we introduce the Driver Monitoring dataset (DMD), an extensive dataset which includes real and simulated driving scenarios.
  arXiv  Detail & Related papers  (2020-08-27T12:33:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.