EyeCue: Driver Cognitive Distraction Detection via Gaze-Empowered Egocentric Video Understanding

• URL: http://arxiv.org/abs/2605.07859v1
• Date: Fri, 08 May 2026 15:20:20 GMT
• Title: EyeCue: Driver Cognitive Distraction Detection via Gaze-Empowered Egocentric Video Understanding
• Abstract summary: EyeCue is a gaze-empowered egocentric video understanding framework to detect driver cognitive distraction.<n>A key insight is that cognitive distraction manifests in the interaction between eye gaze and visual context.<n>To tackle the limited scale and diversity of existing datasets, we introduce CogDrive, a comprehensive multi-scenario dataset.
• Score: 8.102838918541108
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Driver cognitive distraction is a major cause of road collisions and remains difficult to detect. Unlike manual or visual distraction, cognitive distraction is diverted by thoughts unrelated to driving, even when the driver appears visually attentive and exhibits no explicit physical movements. In this work, we propose EyeCue, a gaze-empowered egocentric video understanding framework, to detect driver cognitive distraction. A key insight is that cognitive distraction manifests in the interaction between eye gaze and visual context. To capture this interaction, EyeCue integrates eye gaze with egocentric video to enable context-aware modeling of the driver's attention over time. Furthermore, to tackle the limited scale and diversity of existing datasets, we introduce CogDrive, a comprehensive multi-scenario dataset that augments four existing driving datasets with cognitive distraction annotations. Through extensive evaluations on CogDrive, we show that EyeCue achieves the highest accuracy of 74.38%, outperforming 11 baselines from 6 model families by over 7%. Notably, EyeCue can achieve an accuracy of over 70% across various driving scenarios (different road types, times of day, and weather conditions) with strong generalizability. These results highlight the importance of modeling gaze-context interactions and the effectiveness of cross-modal interaction modeling for multimodal cognitive distraction detection. Our codes and CogDrive dataset resources are available at https://github.com/langzhang2000/EyeCue.

Related papers

• Seeing Beyond Views: Multi-View Driving Scene Video Generation with Holistic Attention [61.3281618482513]
  We present CogDriving, a novel network designed for synthesizing high-quality multi-view driving videos.<n>CogDriving leverages a Diffusion Transformer architecture with holistic-4D attention modules, enabling simultaneous associations across the dimensions.<n>CogDriving demonstrates strong performance on the nuScenes validation set, achieving an FVD score of 37.8, highlighting its ability to generate realistic driving videos.
  arXiv  Detail & Related papers  (2024-12-04T18:02:49Z)
• Improving automatic detection of driver fatigue and distraction using machine learning [0.0]
  Driver fatigue and distracted driving are important factors in traffic accidents. We present techniques for simultaneously detecting fatigue and distracted driving behaviors using vision-based and machine learning-based approaches.
  arXiv  Detail & Related papers  (2024-01-04T06:33:46Z)
• Gaze-Informed Vision Transformers: Predicting Driving Decisions Under Uncertainty [5.006068984003071]
  Vision Transformers (ViT) have advanced computer vision, yet their efficacy in complex tasks like driving remains less explored.<n>This study enhances ViT by integrating human eye gaze, captured via eye-tracking, to increase prediction accuracy in driving scenarios under uncertainty.
  arXiv  Detail & Related papers  (2023-08-26T22:48:06Z)
• BI AVAN: Brain inspired Adversarial Visual Attention Network [67.05560966998559]
  We propose a brain-inspired adversarial visual attention network (BI-AVAN) to characterize human visual attention directly from functional brain activity. Our model imitates the biased competition process between attention-related/neglected objects to identify and locate the visual objects in a movie frame the human brain focuses on in an unsupervised manner.
  arXiv  Detail & Related papers  (2022-10-27T22:20:36Z)
• Exploring Contextual Representation and Multi-Modality for End-to-End Autonomous Driving [58.879758550901364]
  Recent perception systems enhance spatial understanding with sensor fusion but often lack full environmental context. We introduce a framework that integrates three cameras to emulate the human field of view, coupled with top-down bird-eye-view semantic data to enhance contextual representation. Our method achieves displacement error by 0.67m in open-loop settings, surpassing current methods by 6.9% on the nuScenes dataset.
  arXiv  Detail & Related papers  (2022-10-13T05:56:20Z)
• Surround-View Cameras based Holistic Visual Perception for Automated Driving [0.6091702876917281]
  We focus on developing near-field perception algorithms with high performance and low computational complexity. These capabilities for computers is critical for various applications, including self-driving cars, augmented reality, and architectural surveying.
  arXiv  Detail & Related papers  (2022-06-11T14:51:30Z)
• CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Driving [117.87070488537334]
  We introduce a challenging dataset named CODA that exposes this critical problem of vision-based detectors. The performance of standard object detectors trained on large-scale autonomous driving datasets significantly drops to no more than 12.8% in mAR. We experiment with the state-of-the-art open-world object detector and find that it also fails to reliably identify the novel objects in CODA.
  arXiv  Detail & Related papers  (2022-03-15T08:32:56Z)
• Do Pedestrians Pay Attention? Eye Contact Detection in the Wild [75.54077277681353]
  In urban environments, humans rely on eye contact for fast and efficient communication with nearby people. In this paper, we focus on eye contact detection in the wild, i.e., real-world scenarios for autonomous vehicles with no control over the environment or the distance of pedestrians. We introduce a model that leverages semantic keypoints to detect eye contact and show that this high-level representation achieves state-of-the-art results on the publicly-available dataset JAAD. To study domain adaptation, we create LOOK: a large-scale dataset for eye contact detection in the wild, which focuses on diverse and un
  arXiv  Detail & Related papers  (2021-12-08T10:21:28Z)
• Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation [77.60050239225086]
  We propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images. Our approach is fully automatic without any human interaction. We present a multi-task network for VUS parsing and a multi-stream network for VHI parsing.
  arXiv  Detail & Related papers  (2020-12-15T03:03:38Z)
• Heatmap-Based Method for Estimating Drivers' Cognitive Distraction [0.0]
  In this study, the influence of cognitive processes on the drivers gaze behavior is explored. A novel image-based representation of the driver's eye-gaze dispersion is proposed to estimate cognitive distraction.
  arXiv  Detail & Related papers  (2020-05-28T16:37:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.