Gesture Recognition with mmWave Wi-Fi Access Points: Lessons Learned

• URL: http://arxiv.org/abs/2306.17062v1
• Date: Thu, 29 Jun 2023 16:10:07 GMT
• Title: Gesture Recognition with mmWave Wi-Fi Access Points: Lessons Learned
• Authors: Nabeel Nisar Bhat, Rafael Berkvens, Jeroen Famaey
• Abstract summary: We explore mmWave (60 GHz) Wi-Fi signals for gesture recognition/pose estimation. For this reason, we extract beam signal-to-noise ratios (SNRs) from periodic beam training employed by IEEE 802.11ad devices. A deep neural network (DNN) achieves promising results on the beam SNR task with state-of-the-art 96.7% accuracy in a single environment.
• Score: 3.5711957833616235
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, channel state information (CSI) at sub-6 GHz has been widely exploited for Wi-Fi sensing, particularly for activity and gesture recognition. In this work, we instead explore mmWave (60 GHz) Wi-Fi signals for gesture recognition/pose estimation. Our focus is on the mmWave Wi-Fi signals so that they can be used not only for high data rate communication but also for improved sensing e.g., for extended reality (XR) applications. For this reason, we extract spatial beam signal-to-noise ratios (SNRs) from the periodic beam training employed by IEEE 802.11ad devices. We consider a set of 10 gestures/poses motivated by XR applications. We conduct experiments in two environments and with three people.As a comparison, we also collect CSI from IEEE 802.11ac devices. To extract features from the CSI and the beam SNR, we leverage a deep neural network (DNN). The DNN classifier achieves promising results on the beam SNR task with state-of-the-art 96.7% accuracy in a single environment, even with a limited dataset. We also investigate the robustness of the beam SNR against CSI across different environments. Our experiments reveal that features from the CSI generalize without additional re-training, while those from beam SNRs do not. Therefore, re-training is required in the latter case.

Related papers

• Faster Region-Based CNN Spectrum Sensing and Signal Identification in Cluttered RF Environments [0.7734726150561088]
  We optimize a faster region-based convolutional neural network (FRCNN) for 1-dimensional (1D) signal processing and electromagnetic spectrum sensing. Results show that our method has better localization performance, and is faster than the 2D equivalent.
  arXiv  Detail & Related papers  (2023-02-20T09:35:13Z)
• Exposing the CSI: A Systematic Investigation of CSI-based Wi-Fi Sensing Capabilities and Limitations [16.819111460629397]
  This work aims to shed light on the impact of Wi-Fi 6 features on the sensing performance and to create a benchmark for future research on Wi-Fi sensing. We perform an extensive CSI data collection campaign involving 3 individuals, 3 environments, and 12 activities, using Wi-Fi 6 signals. An anonymized ground truth obtained through video recording accompanies our 80-GB dataset, which contains almost two hours of CSI data from three collectors.
  arXiv  Detail & Related papers  (2023-02-02T10:21:00Z)
• DensePose From WiFi [86.61881052177228]
  We develop a deep neural network that maps the phase and amplitude of WiFi signals to UV coordinates within 24 human regions. Our model can estimate the dense pose of multiple subjects, with comparable performance to image-based approaches.
  arXiv  Detail & Related papers  (2022-12-31T16:48:43Z)
• Hand gesture recognition using 802.11ad mmWave sensor in the mobile device [2.5476515662939563]
  We explore the feasibility of AI assisted hand-gesture recognition using 802.11ad 60GHz (mmWave) technology in smartphones. We built a prototype system, where radar sensing and communication waveform can coexist by time-division duplex (TDD) It can gather sensing data and predict gestures within 100 milliseconds.
  arXiv  Detail & Related papers  (2022-11-14T03:36:17Z)
• NVRadarNet: Real-Time Radar Obstacle and Free Space Detection for Autonomous Driving [57.03126447713602]
  We present a deep neural network (DNN) that detects dynamic obstacles and drivable free space using automotive RADAR sensors. The network runs faster than real time on an embedded GPU and shows good generalization across geographic regions.
  arXiv  Detail & Related papers  (2022-09-29T01:30:34Z)
• A Wireless-Vision Dataset for Privacy Preserving Human Activity Recognition [53.41825941088989]
  A new WiFi-based and video-based neural network (WiNN) is proposed to improve the robustness of activity recognition. Our results show that WiVi data set satisfies the primary demand and all three branches in the proposed pipeline keep more than $80%$ of activity recognition accuracy.
  arXiv  Detail & Related papers  (2022-05-24T10:49:11Z)
• A CNN Approach for 5G mmWave Positioning Using Beamformed CSI Measurements [0.5685944521394608]
  We utilize the power of AI by training a Convolutional Neural Network (CNN) using 5G New Radio (NR) fingerprints. We collect ray-tracing-based 5G NR CSI from an urban area. The CSI data of the signals from one Base Station (BS) is collected at the reference points with known positions to train a CNN. The results prove that our trained network for a specific urban environment can estimate the UE position with a minimum mean error of 0.98 m.
  arXiv  Detail & Related papers  (2022-04-30T14:33:04Z)
• Oriented R-CNN for Object Detection [61.78746189807462]
  This work proposes an effective and simple oriented object detection framework, termed Oriented R-CNN. In the first stage, we propose an oriented Region Proposal Network (oriented RPN) that directly generates high-quality oriented proposals in a nearly cost-free manner. The second stage is oriented R-CNN head for refining oriented Regions of Interest (oriented RoIs) and recognizing them.
  arXiv  Detail & Related papers  (2021-08-12T12:47:43Z)
• Deep Networks for Direction-of-Arrival Estimation in Low SNR [89.45026632977456]
  We introduce a Convolutional Neural Network (CNN) that is trained from mutli-channel data of the true array manifold matrix. We train a CNN in the low-SNR regime to predict DoAs across all SNRs. Our robust solution can be applied in several fields, ranging from wireless array sensors to acoustic microphones or sonars.
  arXiv  Detail & Related papers  (2020-11-17T12:52:18Z)
• Near-chip Dynamic Vision Filtering for Low-Bandwidth Pedestrian Detection [99.94079901071163]
  This paper presents a novel end-to-end system for pedestrian detection using Dynamic Vision Sensors (DVSs) We target applications where multiple sensors transmit data to a local processing unit, which executes a detection algorithm. Our detector is able to perform a detection every 450 ms, with an overall testing F1 score of 83%.
  arXiv  Detail & Related papers  (2020-04-03T17:36:26Z)
• Centimeter-Level Indoor Localization using Channel State Information with Recurrent Neural Networks [12.193558591962754]
  This paper proposes the neural network method to estimate the centimeter-level indoor positioning with real CSI data collected from linear antennas. It utilizes an amplitude of channel response or a correlation matrix as the input, which can highly reduce the data size and suppress the noise. Also, it makes use of the consistency in the user motion trajectory via Recurrent Neural Network (RNN) and signal-noise ratio (SNR) information, which can further improve the estimation accuracy.
  arXiv  Detail & Related papers  (2020-02-04T17:10:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.