Real-Time In-Cabin Driver Behavior Recognition on Low-Cost Edge Hardware

• URL: http://arxiv.org/abs/2512.22298v1
• Date: Fri, 26 Dec 2025 00:54:24 GMT
• Title: Real-Time In-Cabin Driver Behavior Recognition on Low-Cost Edge Hardware
• Abstract summary: Single-camera in-cabin driver behavior recognition system designed for deployment on Raspberry Pi 5 and Google Coral Edge TPU.<n>System covers 17 behavior classes, including multiple phone-use modes, eating/drinking, smoking, reaching behind, gaze/attention shifts, passenger interaction, grooming, control-panel interaction, yawning, and eyes-closed sleep.
• Score: 1.1100764382749708
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In-cabin Driver Monitoring Systems (DMS) must recognize distraction- and drowsiness-related behaviors with low latency under strict constraints on compute, power, and cost. We present a single-camera in-cabin driver behavior recognition system designed for deployment on two low-cost edge platforms: Raspberry Pi 5 (CPU-only) and Google Coral Edge TPU. The proposed pipeline combines (i) a compact per-frame vision model, (ii) a confounder-aware label design to reduce visually similar false positives, and (iii) a temporal decision head that triggers alerts only when predictions are both confident and sustained. The system covers 17 behavior classes, including multiple phone-use modes, eating/drinking, smoking, reaching behind, gaze/attention shifts, passenger interaction, grooming, control-panel interaction, yawning, and eyes-closed sleep. Training and evaluation use licensed datasets spanning diverse drivers, vehicles, and lighting conditions (details in Section 6), and we further validate runtime behavior in real in-vehicle tests. The optimized deployments achieve about 16 FPS on Raspberry Pi 5 with INT8 inference (per-frame latency under 60 ms) and about 25 FPS on Coral Edge TPU, enabling real-time monitoring and stable alert generation on inexpensive hardware. Finally, we discuss how reliable in-cabin human-state perception can serve as an upstream input for human-centered vehicle intelligence, including emerging agentic vehicle concepts.

Related papers

• DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting [19.185390523777187]
  We introduce DriveMotion, a benchmark for continuous driver motion forecasting.<n>DriveMotion contains 393 hours of 133-keypoint motion sequences at 10 Hz from 360 drivers.<n>We use dynamics-anchored evaluation, placing forecasting windows around vehicle maneuvers identified offline from CAN signals.
  arXiv  Detail & Related papers  (2026-09-08T01:49:18Z)
• Risk-Aware Selective Multimodal Driver Monitoring with Driver-State World Modeling [13.540675152706589]
  Continuous driver monitoring in automated vehicles requires low-latency inference.<n>We propose a cost-aware selective inference framework for deployable multimodal driver monitoring.
  arXiv  Detail & Related papers  (2026-06-25T11:59:21Z)
• Vision-Based Lane Following and Traffic Sign Recognition for Resource-Constrained Autonomous Vehicles [0.0]
  This paper presents a vision-based framework that integrates lane detection, lane tracking, and traffic sign recognition for embedded autonomous vehicles.<n>A rule-based steering controller generates steering commands to maintain stable vehicle navigation.<n>Experiments show that the system achieves real-time performance while maintaining accurate lane tracking with only 3.16% maximum offset RMSE.
  arXiv  Detail & Related papers  (2026-04-23T20:19:32Z)
• Video Object Recognition in Mobile Edge Networks: Local Tracking or Edge Detection? [57.000348519630286]
  Recent advances in mobile edge computing have made it possible to offload-intensive object detection to edge servers equipped with high-accuracy neural networks.<n>This hybrid approach offers a promising solution but introduces a new challenge: deciding when to perform edge detection versus local tracking.<n>We propose the LTED-Ada in single-device setting, a deep reinforcement learning-based algorithm that adaptively selects between local tracking and edge detection.
  arXiv  Detail & Related papers  (2025-11-25T04:54:51Z)
• Driver-Net: Multi-Camera Fusion for Assessing Driver Take-Over Readiness in Automated Vehicles [5.543220407902113]
  Driver-Net is a novel deep learning framework that fuses multi-camera inputs to estimate driver take-over readiness.<n>It captures synchronised visual cues from the driver's head, hands, and body posture through a triple-camera setup.<n>The proposed method achieves an accuracy of up to 95.8% in driver readiness classification.
  arXiv  Detail & Related papers  (2025-07-05T19:27:03Z)
• Cross-Camera Distracted Driver Classification through Feature Disentanglement and Contrastive Learning [13.613407983544427]
  Driver Behavior Monitoring Network (DBMNet) relies on a lightweight backbone and integrates a disentanglement module to discard camera view information.<n>DBMNet achieves an improvement of 7% in Top-1 accuracy compared to existing approaches.
  arXiv  Detail & Related papers  (2024-11-20T10:27:12Z)
• Real-Time Pedestrian Detection on IoT Edge Devices: A Lightweight Deep Learning Approach [1.4732811715354455]
  This research explores implementing a lightweight deep learning model on Artificial Intelligence of Things (AIoT) edge devices. An optimized You Only Look Once (YOLO) based DL model is deployed for real-time pedestrian detection. The simulation results demonstrate that the optimized YOLO model can achieve real-time pedestrian detection, with a fast inference speed of 147 milliseconds, a frame rate of 2.3 frames per second, and an accuracy of 78%.
  arXiv  Detail & Related papers  (2024-09-24T04:48:41Z)
• Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation [58.21683603243387]
  We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework. These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents. Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
  arXiv  Detail & Related papers  (2023-11-27T18:57:42Z)
• Leveraging the Edge and Cloud for V2X-Based Real-Time Object Detection in Autonomous Driving [0.0]
  Environmental perception is a key element of autonomous driving. In this paper, we investigate the best trade-off between detection quality and latency for real-time perception in autonomous vehicles. We show that models with adequate compression can be run in real-time on the cloud while outperforming local detection performance.
  arXiv  Detail & Related papers  (2023-08-09T21:39:10Z)
• DeepIPC: Deeply Integrated Perception and Control for an Autonomous Vehicle in Real Environments [7.642646077340124]
  We introduce DeepIPC, a novel end-to-end model tailored for autonomous driving. DeepIPC seamlessly integrates perception and control tasks. Our evaluation demonstrates DeepIPC's superior performance in terms of drivability and multi-task efficiency.
  arXiv  Detail & Related papers  (2022-07-20T14:20:35Z)
• COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles [54.61668577827041]
  We introduce COOPERNAUT, an end-to-end learning model that uses cross-vehicle perception for vision-based cooperative driving. Our experiments on AutoCastSim suggest that our cooperative perception driving models lead to a 40% improvement in average success rate.
  arXiv  Detail & Related papers  (2022-05-04T17:55:12Z)
• Real-time Object Detection for Streaming Perception [84.2559631820007]
  Streaming perception is proposed to jointly evaluate the latency and accuracy into a single metric for video online perception. We build a simple and effective framework for streaming perception. Our method achieves competitive performance on Argoverse-HD dataset and improves the AP by 4.9% compared to the strong baseline.
  arXiv  Detail & Related papers  (2022-03-23T11:33:27Z)
• Real Time Monocular Vehicle Velocity Estimation using Synthetic Data [78.85123603488664]
  We look at the problem of estimating the velocity of road vehicles from a camera mounted on a moving car. We propose a two-step approach where first an off-the-shelf tracker is used to extract vehicle bounding boxes and then a small neural network is used to regress the vehicle velocity.
  arXiv  Detail & Related papers  (2021-09-16T13:10:27Z)
• IntentNet: Learning to Predict Intention from Raw Sensor Data [86.74403297781039]
  In this paper, we develop a one-stage detector and forecaster that exploits both 3D point clouds produced by a LiDAR sensor as well as dynamic maps of the environment. Our multi-task model achieves better accuracy than the respective separate modules while saving computation, which is critical to reducing reaction time in self-driving applications.
  arXiv  Detail & Related papers  (2021-01-20T00:31:52Z)
• Driver Drowsiness Classification Based on Eye Blink and Head Movement Features Using the k-NN Algorithm [8.356765961526955]
  This work is to extend the driver drowsiness detection in vehicles using signals of a driver monitoring camera. For this purpose, 35 features related to the driver's eye blinking behavior and head movements are extracted in driving simulator experiments. A concluding analysis of the best performing feature sets yields valuable insights about the influence of drowsiness on the driver's blink behavior and head movements.
  arXiv  Detail & Related papers  (2020-09-28T12:37:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.