Tesla-Rapture: A Lightweight Gesture Recognition System from mmWave Radar Point Clouds

• URL: http://arxiv.org/abs/2109.06448v1
• Date: Tue, 14 Sep 2021 05:25:17 GMT
• Title: Tesla-Rapture: A Lightweight Gesture Recognition System from mmWave Radar Point Clouds
• Authors: Dariush Salami, Ramin Hasibi, Sameera Palipana, Petar Popovski, Tom Michoel, and Stephan Sigg
• Abstract summary: Tesla-Rapture is a gesture recognition interface for point clouds generated by mmWave Radars. We develop Tesla, a Message Passing Neural Network (MPNN) graph convolution approach for mmWave radar point clouds. We publish the source code, the trained models, and the implementation of the model for embedded devices.
• Score: 28.304829106423988
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Tesla-Rapture, a gesture recognition interface for point clouds generated by mmWave Radars. State of the art gesture recognition models are either too resource consuming or not sufficiently accurate for integration into real-life scenarios using wearable or constrained equipment such as IoT devices (e.g. Raspberry PI), XR hardware (e.g. HoloLens), or smart-phones. To tackle this issue, we developed Tesla, a Message Passing Neural Network (MPNN) graph convolution approach for mmWave radar point clouds. The model outperforms the state of the art on two datasets in terms of accuracy while reducing the computational complexity and, hence, the execution time. In particular, the approach, is able to predict a gesture almost 8 times faster than the most accurate competitor. Our performance evaluation in different scenarios (environments, angles, distances) shows that Tesla generalizes well and improves the accuracy up to 20% in challenging scenarios like a through-wall setting and sensing at extreme angles. Utilizing Tesla, we develop Tesla-Rapture, a real-time implementation using a mmWave Radar on a Raspberry PI 4 and evaluate its accuracy and time-complexity. We also publish the source code, the trained models, and the implementation of the model for embedded devices.

Related papers

• Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion [74.84019379368807]
  We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline. Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
  arXiv  Detail & Related papers  (2023-07-31T09:53:50Z)
• UnLoc: A Universal Localization Method for Autonomous Vehicles using LiDAR, Radar and/or Camera Input [51.150605800173366]
  UnLoc is a novel unified neural modeling approach for localization with multi-sensor input in all weather conditions. Our method is extensively evaluated on Oxford Radar RobotCar, ApolloSouthBay and Perth-WA datasets.
  arXiv  Detail & Related papers  (2023-07-03T04:10:55Z)
• RadarFormer: Lightweight and Accurate Real-Time Radar Object Detection Model [13.214257841152033]
  Radar-centric data sets do not get a lot of attention in the development of deep learning techniques for radar perception. We propose a transformers-based model, named RadarFormer, that utilizes state-of-the-art developments in vision deep learning. Our model also introduces a channel-chirp-time merging module that reduces the size and complexity of our models by more than 10 times without compromising accuracy.
  arXiv  Detail & Related papers  (2023-04-17T17:07:35Z)
• Hand gesture recognition using 802.11ad mmWave sensor in the mobile device [2.5476515662939563]
  We explore the feasibility of AI assisted hand-gesture recognition using 802.11ad 60GHz (mmWave) technology in smartphones. We built a prototype system, where radar sensing and communication waveform can coexist by time-division duplex (TDD) It can gather sensing data and predict gestures within 100 milliseconds.
  arXiv  Detail & Related papers  (2022-11-14T03:36:17Z)
• NVRadarNet: Real-Time Radar Obstacle and Free Space Detection for Autonomous Driving [57.03126447713602]
  We present a deep neural network (DNN) that detects dynamic obstacles and drivable free space using automotive RADAR sensors. The network runs faster than real time on an embedded GPU and shows good generalization across geographic regions.
  arXiv  Detail & Related papers  (2022-09-29T01:30:34Z)
• Braille Letter Reading: A Benchmark for Spatio-Temporal Pattern Recognition on Neuromorphic Hardware [50.380319968947035]
  Recent deep learning approaches have reached accuracy in such tasks, but their implementation on conventional embedded solutions is still computationally very and energy expensive. We propose a new benchmark for computing tactile pattern recognition at the edge through letters reading. We trained and compared feed-forward and recurrent spiking neural networks (SNNs) offline using back-propagation through time with surrogate gradients, then we deployed them on the Intel Loihimorphic chip for efficient inference. Our results show that the LSTM outperforms the recurrent SNN in terms of accuracy by 14%. However, the recurrent SNN on Loihi is 237 times more energy
  arXiv  Detail & Related papers  (2022-05-30T14:30:45Z)
• Deep Learning for Real Time Satellite Pose Estimation on Low Power Edge TPU [58.720142291102135]
  In this paper we propose a pose estimation software exploiting neural network architectures. We show how low power machine learning accelerators could enable Artificial Intelligence exploitation in space.
  arXiv  Detail & Related papers  (2022-04-07T08:53:18Z)
• Taking ROCKET on an Efficiency Mission: Multivariate Time Series Classification with LightWaveS [3.5786621294068373]
  We present LightWaveS, a framework for accurate multivariate time series classification. It employs just 2.5% of the ROCKET features, while achieving accuracy comparable to recent deep learning models. We show that we achieve speedup ranging from 9x to 65x compared to ROCKET during inference on an edge device.
  arXiv  Detail & Related papers  (2022-04-04T10:52:20Z)
• Cross-modal Learning of Graph Representations using Radar Point Cloud for Long-Range Gesture Recognition [6.9545038359818445]
  We propose a novel architecture for a long-range (1m - 2m) gesture recognition solution. We use a point cloud-based cross-learning approach from camera point cloud to 60-GHz FMCW radar point cloud. In the experimental results section, we demonstrate our model's overall accuracy of 98.4% for five gestures and its generalization capability.
  arXiv  Detail & Related papers  (2022-03-31T14:34:36Z)
• StrObe: Streaming Object Detection from LiDAR Packets [73.27333924964306]
  Rolling shutter LiDARs emitted as a stream of packets, each covering a sector of the 360deg coverage. Modern perception algorithms wait for the full sweep to be built before processing the data, which introduces an additional latency. In this paper we propose StrObe, a novel approach that minimizes latency by ingesting LiDAR packets and emitting a stream of detections without waiting for the full sweep to be built.
  arXiv  Detail & Related papers  (2020-11-12T14:57:44Z)
• RaLL: End-to-end Radar Localization on Lidar Map Using Differentiable Measurement Model [14.155337185792279]
  We propose an end-to-end deep learning framework for Radar Localization on Lidar Map (RaLL) RaLL exploits the mature lidar mapping technique, thus reducing the cost of radar mapping. Our proposed system achieves superior performance over $90km$ driving, even in generalization scenarios where the model training is in UK.
  arXiv  Detail & Related papers  (2020-09-15T13:13:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.