Resource-Efficient Gesture Recognition using Low-Resolution Thermal Camera via Spiking Neural Networks and Sparse Segmentation

• URL: http://arxiv.org/abs/2401.06563v1
• Date: Fri, 12 Jan 2024 13:20:01 GMT
• Title: Resource-Efficient Gesture Recognition using Low-Resolution Thermal Camera via Spiking Neural Networks and Sparse Segmentation
• Authors: Ali Safa, Wout Mommen, Lars Keuninckx
• Abstract summary: This work proposes a novel approach for hand gesture recognition using an inexpensive, low-resolution (24 x 32) thermal sensor. Compared to the use of standard RGB cameras, the proposed system is insensitive to lighting variations. This paper shows that the innovative use of the recently proposed Monostable Multivibrator (MMV) neural networks as a new class of SNN achieves more than one order of magnitude smaller memory and compute complexity.
• Score: 1.7758299835471887
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work proposes a novel approach for hand gesture recognition using an inexpensive, low-resolution (24 x 32) thermal sensor processed by a Spiking Neural Network (SNN) followed by Sparse Segmentation and feature-based gesture classification via Robust Principal Component Analysis (R-PCA). Compared to the use of standard RGB cameras, the proposed system is insensitive to lighting variations while being significantly less expensive compared to high-frequency radars, time-of-flight cameras and high-resolution thermal sensors previously used in literature. Crucially, this paper shows that the innovative use of the recently proposed Monostable Multivibrator (MMV) neural networks as a new class of SNN achieves more than one order of magnitude smaller memory and compute complexity compared to deep learning approaches, while reaching a top gesture recognition accuracy of 93.9% using a 5-class thermal camera dataset acquired in a car cabin, within an automotive context. Our dataset is released for helping future research.

Related papers

• Multi-Sensor Fusion for UAV Classification Based on Feature Maps of Image and Radar Data [4.392337343771302]
  We propose a system that fuses already processed multi-sensor data into a new Deep Neural Network to increase its classification accuracy towards UAV detection. The model fuses high-level features extracted from individual object detection and classification models associated with thermal, optronic, and radar data.
  arXiv  Detail & Related papers  (2024-10-21T15:12:37Z)
• Semantic Segmentation of Radar Detections using Convolutions on Point Clouds [59.45414406974091]
  We introduce a deep-learning based method to convolve radar detections into point clouds. We adapt this algorithm to radar-specific properties through distance-dependent clustering and pre-processing of input point clouds. Our network outperforms state-of-the-art approaches that are based on PointNet++ on the task of semantic segmentation of radar point clouds.
  arXiv  Detail & Related papers  (2023-05-22T07:09:35Z)
• Improved Static Hand Gesture Classification on Deep Convolutional Neural Networks using Novel Sterile Training Technique [2.534406146337704]
  Non-contact hand pose and static gesture recognition have received considerable attention in many applications. This article presents an efficient data collection approach and a novel technique for deep CNN training by introducing sterile'' images. Applying the proposed data collection and training methods yields an increase in classification rate of static hand gestures from $85%$ to $93%$.
  arXiv  Detail & Related papers  (2023-05-03T11:10:50Z)
• Neural Architectural Nonlinear Pre-Processing for mmWave Radar-based Human Gesture Perception [10.826849062116748]
  This paper utilizes two deep learning models, U-Net and EfficientNet, to detect hand gestures and remove noise in a millimeter-wave (mmWave) radar image. A novel pre-processing approach to denoise images before entering the first deep learning model stage increases the accuracy of classification.
  arXiv  Detail & Related papers  (2022-11-07T12:42:13Z)
• Braille Letter Reading: A Benchmark for Spatio-Temporal Pattern Recognition on Neuromorphic Hardware [50.380319968947035]
  Recent deep learning approaches have reached accuracy in such tasks, but their implementation on conventional embedded solutions is still computationally very and energy expensive. We propose a new benchmark for computing tactile pattern recognition at the edge through letters reading. We trained and compared feed-forward and recurrent spiking neural networks (SNNs) offline using back-propagation through time with surrogate gradients, then we deployed them on the Intel Loihimorphic chip for efficient inference. Our results show that the LSTM outperforms the recurrent SNN in terms of accuracy by 14%. However, the recurrent SNN on Loihi is 237 times more energy
  arXiv  Detail & Related papers  (2022-05-30T14:30:45Z)
• Neural Architecture Search for Efficient Uncalibrated Deep Photometric Stereo [105.05232615226602]
  We leverage differentiable neural architecture search (NAS) strategy to find uncalibrated PS architecture automatically. Experiments on the DiLiGenT dataset show that the automatically searched neural architectures performance compares favorably with the state-of-the-art uncalibrated PS methods.
  arXiv  Detail & Related papers  (2021-10-11T21:22:17Z)
• Infrared Small-Dim Target Detection with Transformer under Complex Backgrounds [155.388487263872]
  We propose a new infrared small-dim target detection method with the transformer. We adopt the self-attention mechanism of the transformer to learn the interaction information of image features in a larger range. We also design a feature enhancement module to learn more features of small-dim targets.
  arXiv  Detail & Related papers  (2021-09-29T12:23:41Z)
• SignalNet: A Low Resolution Sinusoid Decomposition and Estimation Network [79.04274563889548]
  We propose SignalNet, a neural network architecture that detects the number of sinusoids and estimates their parameters from quantized in-phase and quadrature samples. We introduce a worst-case learning threshold for comparing the results of our network relative to the underlying data distributions. In simulation, we find that our algorithm is always able to surpass the threshold for three-bit data but often cannot exceed the threshold for one-bit data.
  arXiv  Detail & Related papers  (2021-06-10T04:21:20Z)
• Learning Frequency-aware Dynamic Network for Efficient Super-Resolution [56.98668484450857]
  This paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain. In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden. Experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures.
  arXiv  Detail & Related papers  (2021-03-15T12:54:26Z)
• RAMP-CNN: A Novel Neural Network for Enhanced Automotive Radar Object Recognition [10.006245521984697]
  We propose a radar multiple-perspectives convolutional neural network (RAMP-CNN) that extracts the location and class of objects. To bypass the complexity of 4D convolutional neural networks (NN), we propose to combine several lower-dimension NN models within our RAMP-CNN model. The proposed RAMP-CNN model achieves better average recall and average precision than prior works in all testing scenarios.
  arXiv  Detail & Related papers  (2020-11-13T19:12:12Z)
• HyNNA: Improved Performance for Neuromorphic Vision Sensor based Surveillance using Hybrid Neural Network Architecture [7.293414498855147]
  We improve on a recently proposed hybrid event-frame approach by using morphological image processing algorithms for region proposal. We also address the low-power requirement for object detection and classification by exploring various convolutional neural network (CNN) architectures. Specifically, we compare the results obtained from our object detection framework against the state-of-the-art low-power NVS surveillance system and show an improved accuracy of 82.16% from 63.1%.
  arXiv  Detail & Related papers  (2020-03-19T07:18:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.