RFGAN: RF-Based Human Synthesis

• URL: http://arxiv.org/abs/2112.03727v1
• Date: Tue, 7 Dec 2021 14:34:35 GMT
• Title: RFGAN: RF-Based Human Synthesis
• Authors: Cong Yu, Zhi Wu, Dongheng Zhang, Zhi Lu, Yang Hu, Yan Chen
• Abstract summary: This paper aims to generate fine-grained optical human images by introducing a novel cross-modal RFGAN model. To the best of our knowledge, this is the first work to generate optical images based on RF signals.
• Score: 9.709890321556204
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper demonstrates human synthesis based on the Radio Frequency (RF) signals, which leverages the fact that RF signals can record human movements with the signal reflections off the human body. Different from existing RF sensing works that can only perceive humans roughly, this paper aims to generate fine-grained optical human images by introducing a novel cross-modal RFGAN model. Specifically, we first build a radio system equipped with horizontal and vertical antenna arrays to transceive RF signals. Since the reflected RF signals are processed as obscure signal projection heatmaps on the horizontal and vertical planes, we design a RF-Extractor with RNN in RFGAN for RF heatmap encoding and combining to obtain the human activity information. Then we inject the information extracted by the RF-Extractor and RNN as the condition into GAN using the proposed RF-based adaptive normalizations. Finally, we train the whole model in an end-to-end manner. To evaluate our proposed model, we create two cross-modal datasets (RF-Walk & RF-Activity) that contain thousands of optical human activity frames and corresponding RF signals. Experimental results show that the RFGAN can generate target human activity frames using RF signals. To the best of our knowledge, this is the first work to generate optical images based on RF signals.

Related papers

• RF Challenge: The Data-Driven Radio Frequency Signal Separation Challenge [66.33067693672696]
  This paper addresses the critical problem of interference rejection in radio-frequency (RF) signals using a novel, data-driven approach. First, we present an insightful signal model that serves as a foundation for developing and analyzing interference rejection algorithms. Second, we introduce the RF Challenge, a publicly available dataset featuring diverse RF signals along with code templates. Third, we propose novel AI-based rejection algorithms, specifically architectures like UNet and WaveNet, and evaluate their performance across eight different signal mixture types.
  arXiv  Detail & Related papers  (2024-09-13T13:53:41Z)
• RF-Diffusion: Radio Signal Generation via Time-Frequency Diffusion [15.175370227353406]
  We introduce a novel Time-Frequency Diffusion theory to enhance the original diffusion model, enabling it to tap into the information within the time, frequency, and complex-valued domains of RF signals. RF-Diffusion is a versatile solution to generate diverse, high-quality, and time-series RF data. We also showcase the versatility of RF-Diffusion in boosting Wi-Fi sensing systems and performing channel estimation in 5G networks.
  arXiv  Detail & Related papers  (2024-04-14T04:56:05Z)
• Prompt2NeRF-PIL: Fast NeRF Generation via Pretrained Implicit Latent [61.56387277538849]
  This paper explores promptable NeRF generation for direct conditioning and fast generation of NeRF parameters for the underlying 3D scenes. Prompt2NeRF-PIL is capable of generating a variety of 3D objects with a single forward pass. We will show that our approach speeds up the text-to-NeRF model DreamFusion and the 3D reconstruction speed of the image-to-NeRF method Zero-1-to-3 by 3 to 5 times.
  arXiv  Detail & Related papers  (2023-12-05T08:32:46Z)
• DReg-NeRF: Deep Registration for Neural Radiance Fields [66.69049158826677]
  We propose DReg-NeRF to solve the NeRF registration problem on object-centric annotated scenes without human intervention. Our proposed method beats the SOTA point cloud registration methods by a large margin.
  arXiv  Detail & Related papers  (2023-08-18T08:37:49Z)
• Synthetic Wave-Geometric Impulse Responses for Improved Speech Dereverberation [69.1351513309953]
  We show that accurately simulating the low-frequency components of Room Impulse Responses (RIRs) is important to achieving good dereverberation. We demonstrate that speech dereverberation models trained on hybrid synthetic RIRs outperform models trained on RIRs generated by prior geometric ray tracing methods.
  arXiv  Detail & Related papers  (2022-12-10T20:15:23Z)
• Classification of multi-frequency RF signals by extreme learning, using magnetic tunnel junctions as neurons and synapses [46.000685134136525]
  We show that magnetic tunnel junctions can process RF inputs with multiple frequencies in parallel. Using a backpropagation-free method called extreme learning, we classify noisy images encoded by RF signals. These results are a key step for embedded radiofrequency artificial intelligence.
  arXiv  Detail & Related papers  (2022-11-02T14:09:42Z)
• Unsupervised Learning for Human Sensing Using Radio Signals [29.118868792782937]
  This paper explores the feasibility of adapting RGB-based unsupervised representation learning to RF signals. We show that while contrastive learning has emerged as the main technique for unsupervised representation learning from images and videos, such methods produce poor performance when applied to sensing humans using RF signals. In contrast, predictive unsupervised learning methods learn high-quality representations that can be used for multiple downstream RF-based sensing tasks.
  arXiv  Detail & Related papers  (2022-07-06T00:28:18Z)
• RF Signal Classification with Synthetic Training Data and its Real-World Performance [0.0]
  This paper investigates the impact of different RF signal impairments modeled in synthetic training data with respect to the real-world performance. It achieves an accuracy of up to 95 % in real-world operation by using carefully designed synthetic training data only.
  arXiv  Detail & Related papers  (2022-06-26T20:47:33Z)
• RF Signal Transformation and Classification using Deep Neural Networks [11.200006239443416]
  Deep neural networks (DNNs) designed for computer vision and natural language processing tasks cannot be directly applied to the radio frequency (RF) datasets. We propose to convert the raw RF data to data types that are suitable for off-the-shelf DNNs by introducing a convolutional transform technique. In addition, we propose a simple 5-layer convolutional neural network architecture (CONV-5) that can operate with raw RF I/Q data without any transformation.
  arXiv  Detail & Related papers  (2022-04-06T05:01:59Z)
• NeRF-SR: High-Quality Neural Radiance Fields using Super-Sampling [82.99453001445478]
  We present NeRF-SR, a solution for high-resolution (HR) novel view synthesis with mostly low-resolution (LR) inputs. Our method is built upon Neural Radiance Fields (NeRF) that predicts per-point density and color with a multi-layer perceptron.
  arXiv  Detail & Related papers  (2021-12-03T07:33:47Z)
• iNeRF: Inverting Neural Radiance Fields for Pose Estimation [68.91325516370013]
  We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural RadianceField (NeRF) NeRFs have been shown to be remarkably effective for the task of view synthesis.
  arXiv  Detail & Related papers  (2020-12-10T18:36:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.