RFGAN: RF-Based Human Synthesis

• URL: http://arxiv.org/abs/2112.03727v1
• Date: Tue, 7 Dec 2021 14:34:35 GMT
• Title: RFGAN: RF-Based Human Synthesis
• Abstract summary: This paper aims to generate fine-grained optical human images by introducing a novel cross-modal RFGAN model. To the best of our knowledge, this is the first work to generate optical images based on RF signals.
• Score: 9.709890321556204
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper demonstrates human synthesis based on the Radio Frequency (RF) signals, which leverages the fact that RF signals can record human movements with the signal reflections off the human body. Different from existing RF sensing works that can only perceive humans roughly, this paper aims to generate fine-grained optical human images by introducing a novel cross-modal RFGAN model. Specifically, we first build a radio system equipped with horizontal and vertical antenna arrays to transceive RF signals. Since the reflected RF signals are processed as obscure signal projection heatmaps on the horizontal and vertical planes, we design a RF-Extractor with RNN in RFGAN for RF heatmap encoding and combining to obtain the human activity information. Then we inject the information extracted by the RF-Extractor and RNN as the condition into GAN using the proposed RF-based adaptive normalizations. Finally, we train the whole model in an end-to-end manner. To evaluate our proposed model, we create two cross-modal datasets (RF-Walk & RF-Activity) that contain thousands of optical human activity frames and corresponding RF signals. Experimental results show that the RFGAN can generate target human activity frames using RF signals. To the best of our knowledge, this is the first work to generate optical images based on RF signals.

Related papers

• CORF-GS: Real-Time Wireless Radiance Field Reconstruction via Coupled Optical-RF Gaussian Splatting [65.63543177449922]
  CORF-GS is a real-time WRF reconstruction framework that processes sequential and radio frequency (RF) waves.<n>It constructs a unified Gaussian representation for optical and RF with shared geometry and modality-specific appearance.<n>It achieves state-of-the-art RF spectrum quality and reduces the reconstruction time by $6.4times$ compared with existing WRF methods.
  arXiv  Detail & Related papers  (2026-07-28T10:56:52Z)
• GeRaF: Neural Geometry Reconstruction from Radio Frequency Signals [7.144634906657132]
  GeRaF is the first method to use neural implicit learning for near-range 3D geometry reconstruction from radio frequency (RF) signals.<n>Unlike RGB or LiDAR-based methods, RF sensing can see through occlusion but suffers from low resolution and noise due to its lensless imaging nature.
  arXiv  Detail & Related papers  (2026-05-27T20:58:36Z)
• Semantically Annotated Multimodal Dataset for RF Interpretation and Prediction [4.872063946977515]
  Current limitations in wireless modeling and radio frequency (RF)-based AI are driven by a lack of high-quality, measurement-based datasets that connect RF signals to their physical environments.<n>We propose a new class of multimodal datasets that combines RF measurements with auxiliary modalities like high-resolution cameras and lidar to bridge the gap between RF signals and their physical causes.
  arXiv  Detail & Related papers  (2026-04-01T22:18:01Z)
• RF-GPT: Teaching AI to See the Wireless World [48.294819966466044]
  Large language models (LLMs) and multimodal models have become powerful general-purpose reasoning systems.<n>RF-GPT is a radio-frequency language model (RFLM) that utilizes the visual encoders of multimodals to process and understand RF spectrograms.<n>A text-only LLM then converts these captions into RF-grounded instruction-answer pairs, yielding roughly 12,000 RF scenes and 25 million examples.
  arXiv  Detail & Related papers  (2026-02-16T15:24:56Z)
• Diffusion^2: Turning 3D Environments into Radio Frequency Heatmaps [12.678929276882299]
  We introduce Diffusion2, a diffusion-based approach that uses 3D point clouds to model the propagation of RF signals across a wide range of frequencies.<n>We show that Diffusion2 accurately estimates the behavior of RF signals in various frequency bands and environmental conditions, with an error margin of just 1.9 dB and 27x faster than existing methods.
  arXiv  Detail & Related papers  (2025-10-02T17:50:22Z)
• Data-driven RF Tomography via Cross-modal Sensing and Continual Learning [11.021948836550829]
  We propose a data-driven radio frequency tomography (DRIFT) framework to reconstruct cross section images of underground root tubers.<n>First, we design a cross-modal sensing system with RF and visual sensors, and propose to train an RF tomography deep neural network (DNN) model.<n> Experimental results show that our approach achieves an average equivalent diameter error of 2.29 cm, 23.2% improvement upon the state-of-the-art approach.
  arXiv  Detail & Related papers  (2025-08-04T09:48:50Z)
• RINN: One Sample Radio Frequency Imaging based on Physics Informed Neural Network [9.812746486699323]
  Radio frequency (RF) imaging technology is expected to bring new possibilities for embodied intelligence and multimodal sensing. In this paper, we combine the ideas of PINN to design the RINN network, using physical constraints instead of true value comparison constraints. Our numerical evaluation results show that RINN's imaging results based on phaseless data are good, with indicators such as RRMSE (0.11) performing similarly well.
  arXiv  Detail & Related papers  (2025-04-19T15:19:12Z)
• Rydberg Atomic Quantum Receivers for Classical Wireless Communications and Sensing: Their Models and Performance [78.76421728334013]
  Rydberg atomic quantum receivers (RAQRs) are an eminent solution for detecting the electric field of radio frequency (RF) signals. We introduce the superheterodyne version of RAQRs to the wireless community by presenting an end-to-end reception scheme. We then develop a corresponding equivalent baseband signal model relying on a realistic reception flow.
  arXiv  Detail & Related papers  (2024-12-07T06:25:54Z)
• RF Challenge: The Data-Driven Radio Frequency Signal Separation Challenge [66.33067693672696]
  This paper addresses the critical problem of interference rejection in radio-frequency (RF) signals using a novel, data-driven approach. First, we present an insightful signal model that serves as a foundation for developing and analyzing interference rejection algorithms. Second, we introduce the RF Challenge, a publicly available dataset featuring diverse RF signals along with code templates. Third, we propose novel AI-based rejection algorithms, specifically architectures like UNet and WaveNet, and evaluate their performance across eight different signal mixture types.
  arXiv  Detail & Related papers  (2024-09-13T13:53:41Z)
• RF-Diffusion: Radio Signal Generation via Time-Frequency Diffusion [15.175370227353406]
  We introduce a novel Time-Frequency Diffusion theory to enhance the original diffusion model, enabling it to tap into the information within the time, frequency, and complex-valued domains of RF signals. RF-Diffusion is a versatile solution to generate diverse, high-quality, and time-series RF data. We also showcase the versatility of RF-Diffusion in boosting Wi-Fi sensing systems and performing channel estimation in 5G networks.
  arXiv  Detail & Related papers  (2024-04-14T04:56:05Z)
• Prompt2NeRF-PIL: Fast NeRF Generation via Pretrained Implicit Latent [61.56387277538849]
  This paper explores promptable NeRF generation for direct conditioning and fast generation of NeRF parameters for the underlying 3D scenes. Prompt2NeRF-PIL is capable of generating a variety of 3D objects with a single forward pass. We will show that our approach speeds up the text-to-NeRF model DreamFusion and the 3D reconstruction speed of the image-to-NeRF method Zero-1-to-3 by 3 to 5 times.
  arXiv  Detail & Related papers  (2023-12-05T08:32:46Z)
• DReg-NeRF: Deep Registration for Neural Radiance Fields [66.69049158826677]
  We propose DReg-NeRF to solve the NeRF registration problem on object-centric annotated scenes without human intervention. Our proposed method beats the SOTA point cloud registration methods by a large margin.
  arXiv  Detail & Related papers  (2023-08-18T08:37:49Z)
• Synthetic Wave-Geometric Impulse Responses for Improved Speech Dereverberation [69.1351513309953]
  We show that accurately simulating the low-frequency components of Room Impulse Responses (RIRs) is important to achieving good dereverberation. We demonstrate that speech dereverberation models trained on hybrid synthetic RIRs outperform models trained on RIRs generated by prior geometric ray tracing methods.
  arXiv  Detail & Related papers  (2022-12-10T20:15:23Z)
• Classification of multi-frequency RF signals by extreme learning, using magnetic tunnel junctions as neurons and synapses [46.000685134136525]
  We show that magnetic tunnel junctions can process RF inputs with multiple frequencies in parallel. Using a backpropagation-free method called extreme learning, we classify noisy images encoded by RF signals. These results are a key step for embedded radiofrequency artificial intelligence.
  arXiv  Detail & Related papers  (2022-11-02T14:09:42Z)
• Unsupervised Learning for Human Sensing Using Radio Signals [29.118868792782937]
  This paper explores the feasibility of adapting RGB-based unsupervised representation learning to RF signals. We show that while contrastive learning has emerged as the main technique for unsupervised representation learning from images and videos, such methods produce poor performance when applied to sensing humans using RF signals. In contrast, predictive unsupervised learning methods learn high-quality representations that can be used for multiple downstream RF-based sensing tasks.
  arXiv  Detail & Related papers  (2022-07-06T00:28:18Z)
• RF Signal Classification with Synthetic Training Data and its Real-World Performance [0.0]
  This paper investigates the impact of different RF signal impairments modeled in synthetic training data with respect to the real-world performance. It achieves an accuracy of up to 95 % in real-world operation by using carefully designed synthetic training data only.
  arXiv  Detail & Related papers  (2022-06-26T20:47:33Z)
• RF Signal Transformation and Classification using Deep Neural Networks [11.200006239443416]
  Deep neural networks (DNNs) designed for computer vision and natural language processing tasks cannot be directly applied to the radio frequency (RF) datasets. We propose to convert the raw RF data to data types that are suitable for off-the-shelf DNNs by introducing a convolutional transform technique. In addition, we propose a simple 5-layer convolutional neural network architecture (CONV-5) that can operate with raw RF I/Q data without any transformation.
  arXiv  Detail & Related papers  (2022-04-06T05:01:59Z)
• NeRF-SR: High-Quality Neural Radiance Fields using Super-Sampling [82.99453001445478]
  We present NeRF-SR, a solution for high-resolution (HR) novel view synthesis with mostly low-resolution (LR) inputs. Our method is built upon Neural Radiance Fields (NeRF) that predicts per-point density and color with a multi-layer perceptron.
  arXiv  Detail & Related papers  (2021-12-03T07:33:47Z)
• iNeRF: Inverting Neural Radiance Fields for Pose Estimation [68.91325516370013]
  We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural RadianceField (NeRF) NeRFs have been shown to be remarkably effective for the task of view synthesis.
  arXiv  Detail & Related papers  (2020-12-10T18:36:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.