Cross Vision-RF Gait Re-identification with Low-cost RGB-D Cameras and mmWave Radars

• URL: http://arxiv.org/abs/2207.07896v1
• Date: Sat, 16 Jul 2022 10:34:25 GMT
• Title: Cross Vision-RF Gait Re-identification with Low-cost RGB-D Cameras and mmWave Radars
• Authors: Dongjiang Cao, Ruofeng Liu, Hao Li, Shuai Wang, Wenchao Jiang, Chris Xiaoxuan Lu
• Abstract summary: This work studies the problem of cross-modal human re-identification (ReID) We propose the first-of-its-kind vision-RF system for cross-modal multi-person ReID at the same time. Our proposed system is able to achieve 92.5% top-1 accuracy and 97.5% top-5 accuracy out of 56 volunteers.
• Score: 15.662787088335618
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human identification is a key requirement for many applications in everyday life, such as personalized services, automatic surveillance, continuous authentication, and contact tracing during pandemics, etc. This work studies the problem of cross-modal human re-identification (ReID), in response to the regular human movements across camera-allowed regions (e.g., streets) and camera-restricted regions (e.g., offices) deployed with heterogeneous sensors. By leveraging the emerging low-cost RGB-D cameras and mmWave radars, we propose the first-of-its-kind vision-RF system for cross-modal multi-person ReID at the same time. Firstly, to address the fundamental inter-modality discrepancy, we propose a novel signature synthesis algorithm based on the observed specular reflection model of a human body. Secondly, an effective cross-modal deep metric learning model is introduced to deal with interference caused by unsynchronized data across radars and cameras. Through extensive experiments in both indoor and outdoor environments, we demonstrate that our proposed system is able to achieve ~92.5% top-1 accuracy and ~97.5% top-5 accuracy out of 56 volunteers. We also show that our proposed system is able to robustly reidentify subjects even when multiple subjects are present in the sensors' field of view.

Related papers

• Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar [62.51065633674272]
  We introduce Radar Fields - a neural scene reconstruction method designed for active radar imagers. Our approach unites an explicit, physics-informed sensor model with an implicit neural geometry and reflectance model to directly synthesize raw radar measurements. We validate the effectiveness of the method across diverse outdoor scenarios, including urban scenes with dense vehicles and infrastructure.
  arXiv  Detail & Related papers  (2024-05-07T20:44:48Z)
• Physical-Layer Semantic-Aware Network for Zero-Shot Wireless Sensing [74.12670841657038]
  Device-free wireless sensing has recently attracted significant interest due to its potential to support a wide range of immersive human-machine interactive applications. Data heterogeneity in wireless signals and data privacy regulation of distributed sensing have been considered as the major challenges that hinder the wide applications of wireless sensing in large area networking systems. We propose a novel zero-shot wireless sensing solution that allows models constructed in one or a limited number of locations to be directly transferred to other locations without any labeled data.
  arXiv  Detail & Related papers  (2023-12-08T13:50:30Z)
• Ultra-Range Gesture Recognition using a Web-Camera in Human-Robot Interaction [2.240453048130742]
  Vision-based methods for gesture recognition have been shown to be effective only up to a user-camera distance of seven meters. We propose a novel URGR termed Graph Vision Transformer (GViT) which takes the enhanced image as input. Evaluation of the proposed framework over diverse test data yields a high recognition rate of 98.1%.
  arXiv  Detail & Related papers  (2023-11-26T17:27:26Z)
• DensePose From WiFi [86.61881052177228]
  We develop a deep neural network that maps the phase and amplitude of WiFi signals to UV coordinates within 24 human regions. Our model can estimate the dense pose of multiple subjects, with comparable performance to image-based approaches.
  arXiv  Detail & Related papers  (2022-12-31T16:48:43Z)
• HDNet: Hierarchical Dynamic Network for Gait Recognition using Millimeter-Wave Radar [13.19744551082316]
  We propose a Hierarchical Dynamic Network (HDNet) for gait recognition using mmWave radar. To prove the superiority of our methods, we perform extensive experiments on two public mmWave radar-based gait recognition datasets.
  arXiv  Detail & Related papers  (2022-11-01T07:34:22Z)
• HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar [30.51398364813315]
  This paper introduces a novel human pose estimation benchmark, Human Pose with Millimeter Wave Radar (HuPR) This dataset is created using cross-calibrated mmWave radar sensors and a monocular RGB camera for cross-modality training of radar-based human pose estimation.
  arXiv  Detail & Related papers  (2022-10-22T22:28:40Z)
• Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition [56.91538445510214]
  We present a learning algorithm for human activity recognition in videos. Our approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras. We conduct extensive experiments on the UAV Human dataset and the NEC Drone dataset.
  arXiv  Detail & Related papers  (2022-09-15T22:16:52Z)
• Fail-Safe Human Detection for Drones Using a Multi-Modal Curriculum Learning Approach [1.094245191265935]
  We present KUL-UAVSAFE, a first-of-its-kind dataset for the study of safety-critical people detection by drones. We propose a CNN architecture with cross-fusion highways and introduce a curriculum learning strategy for multi-modal data.
  arXiv  Detail & Related papers  (2021-09-28T12:34:13Z)
• Cross-Camera Feature Prediction for Intra-Camera Supervised Person Re-identification across Distant Scenes [70.30052164401178]
  Person re-identification (Re-ID) aims to match person images across non-overlapping camera views. ICS-DS Re-ID uses cross-camera unpaired data with intra-camera identity labels for training. Cross-camera feature prediction method to mine cross-camera self supervision information. Joint learning of global-level and local-level features forms a global-local cross-camera feature prediction scheme.
  arXiv  Detail & Related papers  (2021-07-29T11:27:50Z)
• Complex-valued Convolutional Neural Networks for Enhanced Radar Signal Denoising and Interference Mitigation [73.0103413636673]
  We propose the use of Complex-Valued Convolutional Neural Networks (CVCNNs) to address the issue of mutual interference between radar sensors. CVCNNs increase data efficiency, speeds up network training and substantially improves the conservation of phase information during interference removal.
  arXiv  Detail & Related papers  (2021-04-29T10:06:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.