Unsupervised Learning for Human Sensing Using Radio Signals

• URL: http://arxiv.org/abs/2207.02370v1
• Date: Wed, 6 Jul 2022 00:28:18 GMT
• Title: Unsupervised Learning for Human Sensing Using Radio Signals
• Authors: Tianhong Li, Lijie Fan, Yuan Yuan, Dina Katabi
• Abstract summary: This paper explores the feasibility of adapting RGB-based unsupervised representation learning to RF signals. We show that while contrastive learning has emerged as the main technique for unsupervised representation learning from images and videos, such methods produce poor performance when applied to sensing humans using RF signals. In contrast, predictive unsupervised learning methods learn high-quality representations that can be used for multiple downstream RF-based sensing tasks.
• Score: 29.118868792782937
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There is a growing literature demonstrating the feasibility of using Radio Frequency (RF) signals to enable key computer vision tasks in the presence of occlusions and poor lighting. It leverages that RF signals traverse walls and occlusions to deliver through-wall pose estimation, action recognition, scene captioning, and human re-identification. However, unlike RGB datasets which can be labeled by human workers, labeling RF signals is a daunting task because such signals are not human interpretable. Yet, it is fairly easy to collect unlabelled RF signals. It would be highly beneficial to use such unlabeled RF data to learn useful representations in an unsupervised manner. Thus, in this paper, we explore the feasibility of adapting RGB-based unsupervised representation learning to RF signals. We show that while contrastive learning has emerged as the main technique for unsupervised representation learning from images and videos, such methods produce poor performance when applied to sensing humans using RF signals. In contrast, predictive unsupervised learning methods learn high-quality representations that can be used for multiple downstream RF-based sensing tasks. Our empirical results show that this approach outperforms state-of-the-art RF-based human sensing on various tasks, opening the possibility of unsupervised representation learning from this novel modality.

Related papers

• NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning [63.39461847093663]
  We propose NeRF-VPT, an innovative method for novel view synthesis to address these challenges. Our proposed NeRF-VPT employs a cascading view prompt tuning paradigm, wherein RGB information gained from preceding rendering outcomes serves as instructive visual prompts for subsequent rendering stages. NeRF-VPT only requires sampling RGB data from previous stage renderings as priors at each training stage, without relying on extra guidance or complex techniques.
  arXiv  Detail & Related papers  (2024-03-02T22:08:10Z)
• MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields [49.68916478541697]
  We develop a Memory-Efficient Incremental Learning algorithm for NeRF (MEIL-NeRF) MEIL-NeRF takes inspiration from NeRF itself in that a neural network can serve as a memory that provides the pixel RGB values, given rays as queries. As a result, MEIL-NeRF demonstrates constant memory consumption and competitive performance.
  arXiv  Detail & Related papers  (2022-12-16T08:04:56Z)
• Disentangled Representation Learning for RF Fingerprint Extraction under Unknown Channel Statistics [77.13542705329328]
  We propose a framework of disentangled representation learning(DRL) that first learns to factor the input signals into a device-relevant component and a device-irrelevant component via adversarial learning. The implicit data augmentation in the proposed framework imposes a regularization on the RFF extractor to avoid the possible overfitting of device-irrelevant channel statistics. Experiments validate that the proposed approach, referred to as DR-RFF, outperforms conventional methods in terms of generalizability to unknown complicated propagation environments.
  arXiv  Detail & Related papers  (2022-08-04T15:46:48Z)
• Self-Supervised RF Signal Representation Learning for NextG Signal Classification with Deep Learning [5.624291722263331]
  Self-supervised learning enables the learning of useful representations from Radio Frequency (RF) signals themselves. We show that the sample efficiency (the number of labeled samples required to achieve a certain accuracy performance) of AMR can be significantly increased by learning signal representations with self-supervised learning.
  arXiv  Detail & Related papers  (2022-07-07T02:07:03Z)
• NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields [54.27264716713327]
  We show that a Neural Radiance Fields (NeRF) representation of a scene can be used to train dense object descriptors. We use an optimized NeRF to extract dense correspondences between multiple views of an object, and then use these correspondences as training data for learning a view-invariant representation of the object. Dense correspondence models supervised with our method significantly outperform off-the-shelf learned descriptors by 106%.
  arXiv  Detail & Related papers  (2022-03-03T18:49:57Z)
• RFGAN: RF-Based Human Synthesis [9.709890321556204]
  This paper aims to generate fine-grained optical human images by introducing a novel cross-modal RFGAN model. To the best of our knowledge, this is the first work to generate optical images based on RF signals.
  arXiv  Detail & Related papers  (2021-12-07T14:34:35Z)
• NeRF-SR: High-Quality Neural Radiance Fields using Super-Sampling [82.99453001445478]
  We present NeRF-SR, a solution for high-resolution (HR) novel view synthesis with mostly low-resolution (LR) inputs. Our method is built upon Neural Radiance Fields (NeRF) that predicts per-point density and color with a multi-layer perceptron.
  arXiv  Detail & Related papers  (2021-12-03T07:33:47Z)
• Spectro-Temporal RF Identification using Deep Learning [3.8137985834223507]
  WRIST is a Wideband, Real-time RF Identification system with Spectro-Temporal detection, framework and system. Our resulting deep learning model is capable to detect, classify, and precisely locate RF emissions in time and frequency. WRIST detector achieves 90 mean Average Precision even in extremely congested environment in the wild.
  arXiv  Detail & Related papers  (2021-07-11T19:02:07Z)
• FedRec: Federated Learning of Universal Receivers over Fading Channels [92.15358738530037]
  We propose a neural network-based symbol detection technique for downlink fading channels. Multiple users collaborate to jointly learn a universal data-driven detector, hence the name FedRec. The performance of the resulting receiver is shown to approach the MAP performance in diverse channel conditions without requiring knowledge of the fading statistics.
  arXiv  Detail & Related papers  (2020-11-14T11:29:55Z)
• Learning Longterm Representations for Person Re-Identification Using Radio Signals [26.900504973871122]
  Person Re-Identification (ReID) aims to recognize a person-of-interest across different places and times. We introduce RF-ReID, a novel approach that harnesses radio frequency (RF) signals for longterm person ReID. RF-ReID outperforms state-of-the-art RGB-based ReID approaches for long term person ReID.
  arXiv  Detail & Related papers  (2020-04-02T15:50:42Z)
• Harvesting Ambient RF for Presence Detection Through Deep Learning [12.535149305258171]
  This paper explores the use of ambient radio frequency (RF) signals for human presence detection through deep learning. Using WiFi signal as an example, we demonstrate that the channel state information (CSI) obtained at the receiver contains rich information about the propagation environment. A convolutional neural network (CNN) properly trained with both magnitude and phase information is then designed to achieve reliable presence detection.
  arXiv  Detail & Related papers  (2020-02-13T20:35:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.