GaitFi: Robust Device-Free Human Identification via WiFi and Vision Multimodal Learning

• URL: http://arxiv.org/abs/2208.14326v1
• Date: Tue, 30 Aug 2022 15:07:43 GMT
• Title: GaitFi: Robust Device-Free Human Identification via WiFi and Vision Multimodal Learning
• Authors: Lang Deng, Jianfei Yang, Shenghai Yuan, Han Zou, Chris Xiaoxuan Lu, Lihua Xie
• Abstract summary: We propose a novel multimodal gait recognition method, namely GaitFi, which leverages WiFi signals and videos for human identification. In GaitFi, Channel State Information (CSI) that reflects the multi-path propagation of WiFi is collected to capture human gaits, while videos are captured by cameras. To learn robust gait information, we propose a Lightweight Residual Convolution Network (LRCN) as the backbone network, and further propose the two-stream GaitFi. Experiments are conducted in the real world, which demonstrates that the GaitFi outperforms state-of-the-art gait recognition
• Score: 33.89340087471202
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As an important biomarker for human identification, human gait can be collected at a distance by passive sensors without subject cooperation, which plays an essential role in crime prevention, security detection and other human identification applications. At present, most research works are based on cameras and computer vision techniques to perform gait recognition. However, vision-based methods are not reliable when confronting poor illuminations, leading to degrading performances. In this paper, we propose a novel multimodal gait recognition method, namely GaitFi, which leverages WiFi signals and videos for human identification. In GaitFi, Channel State Information (CSI) that reflects the multi-path propagation of WiFi is collected to capture human gaits, while videos are captured by cameras. To learn robust gait information, we propose a Lightweight Residual Convolution Network (LRCN) as the backbone network, and further propose the two-stream GaitFi by integrating WiFi and vision features for the gait retrieval task. The GaitFi is trained by the triplet loss and classification loss on different levels of features. Extensive experiments are conducted in the real world, which demonstrates that the GaitFi outperforms state-of-the-art gait recognition methods based on single WiFi or camera, achieving 94.2% for human identification tasks of 12 subjects.

Related papers

• ViFi-ReID: A Two-Stream Vision-WiFi Multimodal Approach for Person Re-identification [3.3743041904085125]
  Person re-identification (ReID) plays a vital role in safety inspections, personnel counting, and more. Most current ReID approaches primarily extract features from images, which are easily affected by objective conditions. We leverage widely available routers as sensing devices by capturing gait information from pedestrians through the Channel State Information (CSI) in WiFi signals.
  arXiv  Detail & Related papers  (2024-10-13T15:34:11Z)
• Time-Frequency Analysis of Variable-Length WiFi CSI Signals for Person Re-Identification [3.3743041904085125]
  Person re-identification (ReID) plays an important role in security detection and people counting. This letter introduces a method using WiFi Channel State Information (CSI), leveraging the multipath propagation characteristics of WiFi signals as a basis for distinguishing different pedestrian features. We propose a two-stream network structure capable of processing variable-length data, which analyzes the amplitude in the time domain and the phase in the frequency domain of WiFi signals.
  arXiv  Detail & Related papers  (2024-07-12T07:10:47Z)
• MaskFi: Unsupervised Learning of WiFi and Vision Representations for Multimodal Human Activity Recognition [32.89577715124546]
  We propose a novel unsupervised multimodal HAR solution, MaskFi, that leverages only unlabeled video and WiFi activity data for model training. Benefiting from our unsupervised learning procedure, the network requires only a small amount of annotated data for finetuning and can adapt to the new environment with better performance.
  arXiv  Detail & Related papers  (2024-02-29T15:27:55Z)
• DensePose From WiFi [86.61881052177228]
  We develop a deep neural network that maps the phase and amplitude of WiFi signals to UV coordinates within 24 human regions. Our model can estimate the dense pose of multiple subjects, with comparable performance to image-based approaches.
  arXiv  Detail & Related papers  (2022-12-31T16:48:43Z)
• Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition [56.91538445510214]
  We present a learning algorithm for human activity recognition in videos. Our approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras. We conduct extensive experiments on the UAV Human dataset and the NEC Drone dataset.
  arXiv  Detail & Related papers  (2022-09-15T22:16:52Z)
• A Wireless-Vision Dataset for Privacy Preserving Human Activity Recognition [53.41825941088989]
  A new WiFi-based and video-based neural network (WiNN) is proposed to improve the robustness of activity recognition. Our results show that WiVi data set satisfies the primary demand and all three branches in the proposed pipeline keep more than $80%$ of activity recognition accuracy.
  arXiv  Detail & Related papers  (2022-05-24T10:49:11Z)
• RealGait: Gait Recognition for Person Re-Identification [79.67088297584762]
  We construct a new gait dataset by extracting silhouettes from an existing video person re-identification challenge which consists of 1,404 persons walking in an unconstrained manner. Our results suggest that recognizing people by their gait in real surveillance scenarios is feasible and the underlying gait pattern is probably the true reason why video person re-idenfification works in practice.
  arXiv  Detail & Related papers  (2022-01-13T06:30:56Z)
• WildGait: Learning Gait Representations from Raw Surveillance Streams [1.90365714903665]
  Existing methods for gait recognition require cooperative gait scenarios, in which a single person is walking multiple times in a straight line in front of a camera. We propose a novel weakly supervised learning framework, WildGait, which consists of training a Spatio-Temporal Graph Convolutional Network on a large number of automatically annotated skeleton sequences. Our results show that, with fine-tuning, we surpass in terms of recognition accuracy the current state-of-the-art pose-based gait recognition solutions.
  arXiv  Detail & Related papers  (2021-05-12T09:11:32Z)
• Self-Supervised WiFi-Based Activity Recognition [3.4473723375416188]
  We extract fine-grained physical layer information from WiFi devices for passive activity recognition in indoor environments. We propose the use of self-supervised contrastive learning to improve activity recognition performance. We observe a 17.7% increase in macro averaged F1 score on the task of WiFi based activity recognition.
  arXiv  Detail & Related papers  (2021-04-19T06:40:21Z)
• Vision Meets Wireless Positioning: Effective Person Re-identification with Recurrent Context Propagation [120.18969251405485]
  Existing person re-identification methods rely on the visual sensor to capture the pedestrians. Mobile phone can be sensed by WiFi and cellular networks in the form of a wireless positioning signal. We propose a novel recurrent context propagation module that enables information to propagate between visual data and wireless positioning data.
  arXiv  Detail & Related papers  (2020-08-10T14:19:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.