Semantically Annotated Multimodal Dataset for RF Interpretation and Prediction

• URL: http://arxiv.org/abs/2604.01433v1
• Date: Wed, 01 Apr 2026 22:18:01 GMT
• Title: Semantically Annotated Multimodal Dataset for RF Interpretation and Prediction
• Abstract summary: Current limitations in wireless modeling and radio frequency (RF)-based AI are driven by a lack of high-quality, measurement-based datasets that connect RF signals to their physical environments.<n>We propose a new class of multimodal datasets that combines RF measurements with auxiliary modalities like high-resolution cameras and lidar to bridge the gap between RF signals and their physical causes.
• Score: 4.872063946977515
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current limitations in wireless modeling and radio frequency (RF)-based AI are primarily driven by a lack of high-quality, measurement-based datasets that connect RF signals to their physical environments. RF heatmaps, the typical form of such data, are high-dimensional and complex but lack the geometric and semantic context needed for interpretation, constraining the development of supervised machine learning models. To address this bottleneck, we propose a new class of multimodal datasets that combines RF measurements with auxiliary modalities like high-resolution cameras and lidar to bridge the gap between RF signals and their physical causes. The proposed data collection will span diverse indoor and outdoor environments, featuring both static and dynamic scenarios, including human activities ranging from walking to subtle gestures. By achieving precise spatial and temporal co-registration and creating digital replicas for voxel-level annotation, this dataset will enable transformative AI research. Key tasks include the forward problem of predicting RF heatmaps from visual data to revolutionize wireless system design, and the inverse problem of inferring scene semantics from RF signals, creating a new form of RF-based perception.

Related papers

• Implicit Neural Representations: A Signal Processing Perspective [54.57279006229212]
  Implicit neural representations (INRs) mark a fundamental shift in signal modeling, moving from discrete sampled data to continuous functional representations.<n>This article examines the evolution of INRs from a signal processing perspective, emphasizing spectral behavior, sampling theory, and multiscale representation.<n>We highlight the utility of INRs across a broad range of applications, including inverse problems in medical and radar imaging, compression, and 3D scene representation.
  arXiv  Detail & Related papers  (2026-04-16T14:12:06Z)
• RF-GPT: Teaching AI to See the Wireless World [48.294819966466044]
  Large language models (LLMs) and multimodal models have become powerful general-purpose reasoning systems.<n>RF-GPT is a radio-frequency language model (RFLM) that utilizes the visual encoders of multimodals to process and understand RF spectrograms.<n>A text-only LLM then converts these captions into RF-grounded instruction-answer pairs, yielding roughly 12,000 RF scenes and 25 million examples.
  arXiv  Detail & Related papers  (2026-02-16T15:24:56Z)
• RF-MatID: Dataset and Benchmark for Radio Frequency Material Identification [27.227118724475137]
  We present RF-MatID, the first open-source, large-scale, wide-band, and geometry-diverse RF dataset for fine-grained material identification.<n>RF-MatID includes 16 fine-grained categories grouped into 5 superclasses, spanning a broad frequency range from 4 to 43.5 GHz, and comprises 142k samples in both frequency- and time-domain representations.
  arXiv  Detail & Related papers  (2026-01-28T08:43:48Z)
• Data-driven RF Tomography via Cross-modal Sensing and Continual Learning [11.021948836550829]
  We propose a data-driven radio frequency tomography (DRIFT) framework to reconstruct cross section images of underground root tubers.<n>First, we design a cross-modal sensing system with RF and visual sensors, and propose to train an RF tomography deep neural network (DNN) model.<n> Experimental results show that our approach achieves an average equivalent diameter error of 2.29 cm, 23.2% improvement upon the state-of-the-art approach.
  arXiv  Detail & Related papers  (2025-08-04T09:48:50Z)
• RF Challenge: The Data-Driven Radio Frequency Signal Separation Challenge [66.33067693672696]
  We address the critical problem of interference rejection in radio-frequency (RF) signals using a data-driven approach that leverages deep-learning methods.<n>A primary contribution of this paper is the introduction of the RF Challenge, which is a publicly available, diverse RF signal dataset.
  arXiv  Detail & Related papers  (2024-09-13T13:53:41Z)
• Deep-Learned Compression for Radio-Frequency Signal Classification [0.49109372384514843]
  Next-generation cellular concepts rely on the processing of large quantities of radio-frequency (RF) samples. We propose a deep learned compression model, HQARF, to compress the complex-valued samples of RF signals. We are assessing the effects of HQARF on the performance of an AI model trained to infer the modulation class of the RF signal.
  arXiv  Detail & Related papers  (2024-03-05T17:42:39Z)
• Physical-Layer Semantic-Aware Network for Zero-Shot Wireless Sensing [74.12670841657038]
  Device-free wireless sensing has recently attracted significant interest due to its potential to support a wide range of immersive human-machine interactive applications. Data heterogeneity in wireless signals and data privacy regulation of distributed sensing have been considered as the major challenges that hinder the wide applications of wireless sensing in large area networking systems. We propose a novel zero-shot wireless sensing solution that allows models constructed in one or a limited number of locations to be directly transferred to other locations without any labeled data.
  arXiv  Detail & Related papers  (2023-12-08T13:50:30Z)
• Three-Way Deep Neural Network for Radio Frequency Map Generation and Source Localization [67.93423427193055]
  Monitoring wireless spectrum over spatial, temporal, and frequency domains will become a critical feature in beyond-5G and 6G communication technologies. In this paper, we present a Generative Adversarial Network (GAN) machine learning model to interpolate irregularly distributed measurements across the spatial domain.
  arXiv  Detail & Related papers  (2021-11-23T22:25:10Z)
• ChaRRNets: Channel Robust Representation Networks for RF Fingerprinting [0.0]
  We present complex-valued Convolutional Neural Networks (CNNs) for RF fingerprinting. We focus on the problem of fingerprinting wireless IoT devices in-the-wild using Deep Learning (DL) techniques.
  arXiv  Detail & Related papers  (2021-05-08T03:03:21Z)
• Deep Neural Network Feature Designs for RF Data-Driven Wireless Device Classification [9.05607520128194]
  We present novel feature design approaches that exploit the distinct structures of the RF communication signals and the spectrum emissions caused by transmitter hardware impairments. Our proposed DNN feature designs substantially improve classification robustness in terms of scalability, accuracy, signature anti-cloning, and insensitivity to environment perturbations.
  arXiv  Detail & Related papers  (2021-03-02T20:19:05Z)
• A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems [82.2513703281725]
  Federated learning is a privacy-preserving approach to train a global model at a central server by collaborating with wireless devices. We present a compressive sensing approach for federated learning over massive multiple-input multiple-output communication systems.
  arXiv  Detail & Related papers  (2020-03-18T05:56:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.