In-the-wild Drowsiness Detection from Facial Expressions

• URL: http://arxiv.org/abs/2010.11162v1
• Date: Wed, 21 Oct 2020 17:28:56 GMT
• Title: In-the-wild Drowsiness Detection from Facial Expressions
• Abstract summary: Driving in a state of drowsiness is a major cause of road accidents, resulting in tremendous damage to life and property. We propose a data collection protocol that involves outfitting vehicles of overnight shift workers with camera kits that record their faces while driving. We experiment with different convolutional and temporal neural network architectures to predict drowsiness states from pose, expression and emotion-based representation of the input video of the driver's face.
• Score: 6.569756709977793
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Driving in a state of drowsiness is a major cause of road accidents, resulting in tremendous damage to life and property. Developing robust, automatic, real-time systems that can infer drowsiness states of drivers has the potential of making life-saving impact. However, developing drowsiness detection systems that work well in real-world scenarios is challenging because of the difficulties associated with collecting high-volume realistic drowsy data and modeling the complex temporal dynamics of evolving drowsy states. In this paper, we propose a data collection protocol that involves outfitting vehicles of overnight shift workers with camera kits that record their faces while driving. We develop a drowsiness annotation guideline to enable humans to label the collected videos into 4 levels of drowsiness: `alert', `slightly drowsy', `moderately drowsy' and `extremely drowsy'. We experiment with different convolutional and temporal neural network architectures to predict drowsiness states from pose, expression and emotion-based representation of the input video of the driver's face. Our best performing model achieves a macro ROC-AUC of 0.78, compared to 0.72 for a baseline model.

Related papers

• SurgVista: Long-Horizon Surgical World Modeling with Plausible Instrument-Tissue Dynamics [49.771521708359955]
  SurgVista is a surgical world model that mitigates spatial interaction incoherence and temporal fidelity collapse.<n>It consistently outperforms state-of-the-art methods across visual quality, temporal consistency, and interaction fidelity.
  arXiv  Detail & Related papers  (2026-06-18T07:47:28Z)
• Drowsiness-Aware Adaptive Autonomous Braking System based on Deep Reinforcement Learning for Enhanced Road Safety [0.0]
  Driver drowsiness is estimated to contribute to 10%-20% of road accidents in Europe.<n>This paper proposes a deep reinforcement learning-based autonomous braking system that integrates vehicle dynamics with driver physiological data.<n>The proposed agent achieves a 99.99% success rate in avoiding collisions under both drowsy and non-drowsy conditions.
  arXiv  Detail & Related papers  (2026-04-15T13:39:56Z)
• HetroD: A High-Fidelity Drone Dataset and Benchmark for Autonomous Driving in Heterogeneous Traffic [49.31491001465465]
  HetroD is a dataset and benchmark for developing autonomous driving systems in heterogeneous environments.<n>HetroD targets the critical challenge of navi- gating real-world heterogeneous traffic dominated by vulner- able road users (VRUs)
  arXiv  Detail & Related papers  (2026-02-03T12:12:47Z)
• DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving [49.11389494068169]
  We present DrivingGen, the first comprehensive benchmark for generative driving world models.<n>DrivingGen combines a diverse evaluation dataset curated from both driving datasets and internet-scale video sources.<n>General models look better but break physics, while driving-specific ones capture motion realistically but lag in visual quality.
  arXiv  Detail & Related papers  (2026-01-04T13:36:21Z)
• Optimization-Guided Diffusion for Interactive Scene Generation [52.23368750264419]
  We present OMEGA, an optimization-guided, training-free framework that enforces structural consistency and interaction awareness during diffusion-based sampling.<n>We show that OMEGA improves generation realism, consistency, and controllability, increasing the ratio of physically and behaviorally valid scenes.<n>Our approach can also generate $5times$ more near-collision frames with a time-to-collision under three seconds.
  arXiv  Detail & Related papers  (2025-12-08T15:56:18Z)
• A Real-Time Driver Drowsiness Detection System Using MediaPipe and Eye Aspect Ratio [0.0]
  This study shows development of a Driver Drowsiness Detection System meant to improve the safety of the road by alerting drivers who are showing signs of being drowsy.<n>The system is based on a standard webcam that tracks the facial features of the driver with the main emphasis on the examination of eye movements.<n>The system detects the moments of long eye shutdowns or a very low rate of blinking which are manifestations of drowsiness and alerts the driver through sound to get her attention back.
  arXiv  Detail & Related papers  (2025-11-17T17:22:48Z)
• Real-Time Drivers' Drowsiness Detection and Analysis through Deep Learning [0.1274452325287335]
  Drowsiness in driving can be life-threatening to any individual and can affect other drivers' safety.<n>This research develops a real-time driver drowsiness detection system utilizing deep convolutional neural networks (DCNNs) and OpenCV.
  arXiv  Detail & Related papers  (2025-11-16T03:39:17Z)
• Natural Reflection Backdoor Attack on Vision Language Model for Autonomous Driving [55.96227460521096]
  Vision-Language Models (VLMs) have been integrated into autonomous driving systems to enhance reasoning capabilities.<n>We propose a natural reflection-based backdoor attack targeting VLM systems in autonomous driving scenarios.<n>Our findings uncover a new class of attacks that exploit the stringent real-time requirements of autonomous driving.
  arXiv  Detail & Related papers  (2025-05-09T20:28:17Z)
• Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models [60.87795376541144]
  A world model is a neural network capable of predicting an agent's next state given past states and actions. During end-to-end training, our policy learns how to recover from errors by aligning with states observed in human demonstrations. We present qualitative and quantitative results, demonstrating significant improvements upon prior state of the art in closed-loop testing.
  arXiv  Detail & Related papers  (2024-09-25T06:48:25Z)
• Adversarial Robustness and Feature Impact Analysis for Driver Drowsiness Detection [0.0]
  Drowsy driving is a major cause of road accidents, but drivers are dismissive of the impact that fatigue can have on their reaction times. To detect drowsiness before any impairment occurs, a promising strategy is using Machine Learning (ML) to monitor Heart Rate Variability (HRV) signals.
  arXiv  Detail & Related papers  (2023-03-23T20:13:44Z)
• Video Waterdrop Removal via Spatio-Temporal Fusion in Driving Scenes [53.16726447796844]
  The waterdrops on windshields during driving can cause severe visual obstructions, which may lead to car accidents. We propose an attention-based framework that fuses the representations from multiple frames to restore visual information occluded by waterdrops.
  arXiv  Detail & Related papers  (2023-02-12T13:47:26Z)
• FBLNet: FeedBack Loop Network for Driver Attention Prediction [75.83518507463226]
  Nonobjective driving experience is difficult to model. In this paper, we propose a FeedBack Loop Network (FBLNet) which attempts to model the driving experience accumulation procedure. Under the guidance of the incremental knowledge, our model fuses the CNN feature and Transformer feature that are extracted from the input image to predict driver attention.
  arXiv  Detail & Related papers  (2022-12-05T08:25:09Z)
• SleepyWheels: An Ensemble Model for Drowsiness Detection leading to Accident Prevention [0.0]
  SleepyWheels is a revolutionary method that uses a lightweight neural network in conjunction with facial landmark identification. The model is trained on a specially created dataset on driver sleepiness and it achieves an accuracy of 97 percent.
  arXiv  Detail & Related papers  (2022-11-01T19:36:47Z)
• Applying Spatiotemporal Attention to Identify Distracted and Drowsy Driving with Vision Transformers [0.0]
  A 20% rise in car crashes in 2021 compared to 2020 has been observed as a result of increased distraction and drowsiness. Drowsy and distracted driving are the cause of 45% of all car crashes. This work investigated the use of the vision transformer to outperform state-of-the-art accuracy from 3D-CNNs.
  arXiv  Detail & Related papers  (2022-07-22T16:36:48Z)
• On the Real-World Adversarial Robustness of Real-Time Semantic Segmentation Models for Autonomous Driving [59.33715889581687]
  The existence of real-world adversarial examples (commonly in the form of patches) poses a serious threat for the use of deep learning models in safety-critical computer vision tasks. This paper presents an evaluation of the robustness of semantic segmentation models when attacked with different types of adversarial patches. A novel loss function is proposed to improve the capabilities of attackers in inducing a misclassification of pixels.
  arXiv  Detail & Related papers  (2022-01-05T22:33:43Z)
• Driving-Signal Aware Full-Body Avatars [49.89791440532946]
  We present a learning-based method for building driving-signal aware full-body avatars. Our model is a conditional variational autoencoder that can be animated with incomplete driving signals. We demonstrate the efficacy of our approach on the challenging problem of full-body animation for virtual telepresence.
  arXiv  Detail & Related papers  (2021-05-21T16:22:38Z)
• Robust SleepNets [7.23389716633927]
  In this study, we investigate eye closedness detection to prevent vehicle accidents related to driver disengagements and driver drowsiness. We develop two models to detect eye closedness: first model on eye images and a second model on face images. We adversarially attack the models with Projected Gradient Descent, Fast Gradient Sign and DeepFool methods and report adversarial success rate.
  arXiv  Detail & Related papers  (2021-02-24T20:48:13Z)
• Driver Drowsiness Classification Based on Eye Blink and Head Movement Features Using the k-NN Algorithm [8.356765961526955]
  This work is to extend the driver drowsiness detection in vehicles using signals of a driver monitoring camera. For this purpose, 35 features related to the driver's eye blinking behavior and head movements are extracted in driving simulator experiments. A concluding analysis of the best performing feature sets yields valuable insights about the influence of drowsiness on the driver's blink behavior and head movements.
  arXiv  Detail & Related papers  (2020-09-28T12:37:38Z)
• Physically Realizable Adversarial Examples for LiDAR Object Detection [72.0017682322147]
  We present a method to generate universal 3D adversarial objects to fool LiDAR detectors. In particular, we demonstrate that placing an adversarial object on the rooftop of any target vehicle to hide the vehicle entirely from LiDAR detectors with a success rate of 80%. This is one step closer towards safer self-driving under unseen conditions from limited training data.
  arXiv  Detail & Related papers  (2020-04-01T16:11:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.