N-DriverMotion: Driver motion learning and prediction using an event-based camera and directly trained spiking neural networks

• URL: http://arxiv.org/abs/2408.13379v1
• Date: Fri, 23 Aug 2024 21:25:16 GMT
• Title: N-DriverMotion: Driver motion learning and prediction using an event-based camera and directly trained spiking neural networks
• Authors: Hyo Jong Chung, Byungkon Kang, Yoonseok Yang,
• Abstract summary: This paper presents a novel system for learning and predicting driver motions and an event-based high-resolution (1280x720) dataset. The proposed neuromorphic vision system achieves comparable accuracy, 94.04%, in recognizing driver motions with the CSNN architecture.
• Score: 2.3941497253612085
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Driver motion recognition is a principal factor in ensuring the safety of driving systems. This paper presents a novel system for learning and predicting driver motions and an event-based high-resolution (1280x720) dataset, N-DriverMotion, newly collected to train on a neuromorphic vision system. The system comprises an event-based camera that generates the first high-resolution driver motion dataset representing spike inputs and efficient spiking neural networks (SNNs) that are effective in training and predicting the driver's gestures. The event dataset consists of 13 driver motion categories classified by direction (front, side), illumination (bright, moderate, dark), and participant. A novel simplified four-layer convolutional spiking neural network (CSNN) that we proposed was directly trained using the high-resolution dataset without any time-consuming preprocessing. This enables efficient adaptation to on-device SNNs for real-time inference on high-resolution event-based streams. Compared with recent gesture recognition systems adopting neural networks for vision processing, the proposed neuromorphic vision system achieves comparable accuracy, 94.04\%, in recognizing driver motions with the CSNN architecture. Our proposed CSNN and the dataset can be used to develop safer and more efficient driver monitoring systems for autonomous vehicles or edge devices requiring an efficient neural network architecture.

Related papers

• Spiking-DD: Neuromorphic Event Camera based Driver Distraction Detection with Spiking Neural Network [0.09999629695552192]
  Event camera-based driver monitoring is emerging as a pivotal area of research. To the best of our knowledge, this study is the first to utilize event camera data with spiking neural networks for driver distraction.
  arXiv  Detail & Related papers  (2024-07-30T08:23:47Z)
• Fluid Batching: Exit-Aware Preemptive Serving of Early-Exit Neural Networks on Edge NPUs [74.83613252825754]
  "smart ecosystems" are being formed where sensing happens concurrently rather than standalone. This is shifting the on-device inference paradigm towards deploying neural processing units (NPUs) at the edge. We propose a novel early-exit scheduling that allows preemption at run time to account for the dynamicity introduced by the arrival and exiting processes.
  arXiv  Detail & Related papers  (2022-09-27T15:04:01Z)
• Federated Deep Learning Meets Autonomous Vehicle Perception: Design and Verification [168.67190934250868]
  Federated learning empowered connected autonomous vehicle (FLCAV) has been proposed. FLCAV preserves privacy while reducing communication and annotation costs. It is challenging to determine the network resources and road sensor poses for multi-stage training.
  arXiv  Detail & Related papers  (2022-06-03T23:55:45Z)
• Efficient Federated Learning with Spike Neural Networks for Traffic Sign Recognition [70.306089187104]
  We introduce powerful Spike Neural Networks (SNNs) into traffic sign recognition for energy-efficient and fast model training. Numerical results indicate that the proposed federated SNN outperforms traditional federated convolutional neural networks in terms of accuracy, noise immunity, and energy efficiency as well.
  arXiv  Detail & Related papers  (2022-05-28T03:11:48Z)
• Object Detection with Spiking Neural Networks on Automotive Event Data [0.0]
  We propose to train spiking neural networks (SNNs) directly on data coming from event cameras to design fast and efficient automotive embedded applications. In this paper, we conducted experiments on two automotive event datasets, establishing new state-of-the-art classification results for spiking neural networks.
  arXiv  Detail & Related papers  (2022-05-09T14:39:47Z)
• Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision [64.71260357476602]
  Event-based vision sensors encode local pixel-wise brightness changes in streams of events rather than image frames. Recent progress in object recognition from event-based sensors has come from conversions of deep neural networks. We propose a hybrid architecture for end-to-end training of deep neural networks for event-based pattern recognition and object detection.
  arXiv  Detail & Related papers  (2021-12-06T23:45:58Z)
• Driving Style Representation in Convolutional Recurrent Neural Network Model of Driver Identification [8.007800530105191]
  We present a deep-neural-network architecture, we term D-CRNN, for building high-fidelity representations for driving style. Using CNN, we capture semantic patterns of driver behavior from trajectories. We then find temporal dependencies between these semantic patterns using RNN to encode driving style.
  arXiv  Detail & Related papers  (2021-02-11T04:33:43Z)
• Risk-Averse MPC via Visual-Inertial Input and Recurrent Networks for Online Collision Avoidance [95.86944752753564]
  We propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates. The robustness of our methods is validated on complex quadruped robot dynamics and can be generally applied to most robotic platforms.
  arXiv  Detail & Related papers  (2020-07-28T07:34:30Z)
• Temporal Pulses Driven Spiking Neural Network for Fast Object Recognition in Autonomous Driving [65.36115045035903]
  We propose an approach to address the object recognition problem directly with raw temporal pulses utilizing the spiking neural network (SNN) Being evaluated on various datasets, our proposed method has shown comparable performance as the state-of-the-art methods, while achieving remarkable time efficiency.
  arXiv  Detail & Related papers  (2020-01-24T22:58:55Z)
• Driver Safety Development Real Time Driver Drowsiness Detection System Based on Convolutional Neural Network [1.7188280334580195]
  This paper focuses on the challenge of driver safety on the road and presents a novel system for drowsiness detection. To detect the falling sleep state of the driver as the sign of drowsiness, Convolutional Neural Networks (CNN) are used.
  arXiv  Detail & Related papers  (2020-01-15T05:38:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.