Enabling Visual Recognition at Radio Frequency

• URL: http://arxiv.org/abs/2405.19516v1
• Date: Wed, 29 May 2024 20:52:59 GMT
• Title: Enabling Visual Recognition at Radio Frequency
• Abstract summary: PanoRadar is a novel RF imaging system that brings RF resolution close to that of LiDAR. Results enable, for the first time, a variety of visual recognition tasks at radio frequency. Our results demonstrate PanoRadar's robust performance across 12 buildings.
• Score: 13.399148413043411
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces PanoRadar, a novel RF imaging system that brings RF resolution close to that of LiDAR, while providing resilience against conditions challenging for optical signals. Our LiDAR-comparable 3D imaging results enable, for the first time, a variety of visual recognition tasks at radio frequency, including surface normal estimation, semantic segmentation, and object detection. PanoRadar utilizes a rotating single-chip mmWave radar, along with a combination of novel signal processing and machine learning algorithms, to create high-resolution 3D images of the surroundings. Our system accurately estimates robot motion, allowing for coherent imaging through a dense grid of synthetic antennas. It also exploits the high azimuth resolution to enhance elevation resolution using learning-based methods. Furthermore, PanoRadar tackles 3D learning via 2D convolutions and addresses challenges due to the unique characteristics of RF signals. Our results demonstrate PanoRadar's robust performance across 12 buildings.

Related papers

• CORF-GS: Real-Time Wireless Radiance Field Reconstruction via Coupled Optical-RF Gaussian Splatting [65.63543177449922]
  CORF-GS is a real-time WRF reconstruction framework that processes sequential and radio frequency (RF) waves.<n>It constructs a unified Gaussian representation for optical and RF with shared geometry and modality-specific appearance.<n>It achieves state-of-the-art RF spectrum quality and reduces the reconstruction time by $6.4times$ compared with existing WRF methods.
  arXiv  Detail & Related papers  (2026-07-28T10:56:52Z)
• GeRaF: Neural Geometry Reconstruction from Radio Frequency Signals [7.144634906657132]
  GeRaF is the first method to use neural implicit learning for near-range 3D geometry reconstruction from radio frequency (RF) signals.<n>Unlike RGB or LiDAR-based methods, RF sensing can see through occlusion but suffers from low resolution and noise due to its lensless imaging nature.
  arXiv  Detail & Related papers  (2026-05-27T20:58:36Z)
• 4D Radar Gaussian Modeling and Scan Matching with RCS [80.74019565026542]
  Radar Cross Section (RCS) information is often overlooked and not usually used in modeling and scan matching processes.<n>We propose incorporating the physical behavior of RCS in the model, in order to further enrich the summarized information about the scene.
  arXiv  Detail & Related papers  (2026-04-16T10:58:29Z)
• RadarXFormer: Robust Object Detection via Cross-Dimension Fusion of 4D Radar Spectra and Images for Autonomous Driving [28.09294325639777]
  Camera- and LiDAR-based perception systems suffer from performance degradation under adverse weather and lighting conditions.<n>Radar-vision fusion provides a promising alternative by combining the environmental robustness and cost efficiency of millimeter-wave (mmWave) radar with the rich semantic information captured by cameras.<n>This paper proposes RadarXFormer, a 3D object detection framework that enables efficient cross-modal fusion between 4D radar spectra and RGB images.
  arXiv  Detail & Related papers  (2026-03-16T04:56:37Z)
• RadioGen3D: 3D Radio Map Generation via Adversarial Learning on Large-Scale Synthetic Data [62.63849426834315]
  Radio maps are essential for efficient radio resource management in future 6G and low-altitude networks.<n>Deep learning (DL) techniques have emerged as an efficient alternative to conventional ray-tracing for radio map estimation.<n>We present the RadioGen3D framework to capture essential 3D signal propagation characteristics and antenna polarization effects.
  arXiv  Detail & Related papers  (2026-02-21T07:50:05Z)
• Bridging Visual and Wireless Sensing: A Unified Radiation Field for 3D Radio Map Construction [14.26926951448715]
  Next-generation wireless networks require high-fidelity environmental intelligence.<n>3D radio maps have emerged as a critical tool for this purpose.<n>We propose URF-GS, a unified radio-optical radiation field representation framework.
  arXiv  Detail & Related papers  (2026-01-27T05:35:50Z)
• RadarGen: Automotive Radar Point Cloud Generation from Cameras [64.69976771710057]
  We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery.<n>RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form.<n>We show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data.
  arXiv  Detail & Related papers  (2025-12-19T18:57:33Z)
• Radar2Shape: 3D Shape Reconstruction from High-Frequency Radar using Multiresolution Signed Distance Functions [11.138924354650582]
  We present Radar2Shape, a denoising diffusion model that handles a partially observable radar signal for 3D reconstruction.<n>We demonstrate that Radar2Shape can successfully reconstruct arbitrary 3D shapes even from partially-observed radar signals.
  arXiv  Detail & Related papers  (2025-11-21T18:40:03Z)
• Diffusion^2: Turning 3D Environments into Radio Frequency Heatmaps [12.678929276882299]
  We introduce Diffusion2, a diffusion-based approach that uses 3D point clouds to model the propagation of RF signals across a wide range of frequencies.<n>We show that Diffusion2 accurately estimates the behavior of RF signals in various frequency bands and environmental conditions, with an error margin of just 1.9 dB and 27x faster than existing methods.
  arXiv  Detail & Related papers  (2025-10-02T17:50:22Z)
• Simulating Automotive Radar with Lidar and Camera Inputs [14.196071603770251]
  Low-cost millimeter automotive radar has received more and more attention due to its ability to handle adverse weather and lighting conditions in autonomous driving. We report a new method that is able to simulate 4D millimeter wave radar signals using camera image, light detection and ranging (lidar) point cloud, and ego-velocity.
  arXiv  Detail & Related papers  (2025-03-11T05:59:43Z)
• RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection [68.99784784185019]
  Poor lighting or adverse weather conditions degrade camera performance. Radar suffers from noise and positional ambiguity. We propose RobuRCDet, a robust object detection model in BEV.
  arXiv  Detail & Related papers  (2025-02-18T17:17:38Z)
• Automotive Elevation Mapping with Interferometric Synthetic Aperture Radar [0.0]
  Synthetic Aperture Radar (SAR) is a class of techniques to improve azimuth resolution and sensitivity for radar. Interferometric SAR can be used to extract elevation from the variations in phase measurements in SAR images. We show that a typical, low-resolution radar array mounted on a vehicle can be used to accurately localize detections in 3D space for both urban and agricultural environments.
  arXiv  Detail & Related papers  (2025-01-14T23:57:35Z)
• SuperNeRF-GAN: A Universal 3D-Consistent Super-Resolution Framework for Efficient and Enhanced 3D-Aware Image Synthesis [59.73403876485574]
  We propose SuperNeRF-GAN, a universal framework for 3D-consistent super-resolution. A key highlight of SuperNeRF-GAN is its seamless integration with NeRF-based 3D-aware image synthesis methods. Experimental results demonstrate the superior efficiency, 3D-consistency, and quality of our approach.
  arXiv  Detail & Related papers  (2025-01-12T10:31:33Z)
• Active Reconfigurable Intelligent Surface Empowered Synthetic Aperture Radar Imaging [18.482494583284627]
  Synthetic Aperture Radar (SAR) utilizes the movement of the radar antenna over a specific area interest to achieve higher resolution imaging. We present a range-Doppler (RD) imaging algorithm to obtain imaging results for the proposed ARIS-empowered SAR system.
  arXiv  Detail & Related papers  (2024-09-18T06:33:11Z)
• Redefining Automotive Radar Imaging: A Domain-Informed 1D Deep Learning Approach for High-Resolution and Efficient Performance [6.784861785632841]
  Our study redefines radar imaging super-resolution as a one-dimensional (1D) signal super-resolution spectra estimation problem. Our tailored deep learning network for automotive radar imaging exhibits remarkable scalability, parameter efficiency and fast inference speed. Our SR-SPECNet sets a new benchmark in producing high-resolution radar range-azimuth images.
  arXiv  Detail & Related papers  (2024-06-11T16:07:08Z)
• G3R: Generating Rich and Fine-grained mmWave Radar Data from 2D Videos for Generalized Gesture Recognition [19.95047010486547]
  We develop a software pipeline that exploits wealthy 2D videos to generate realistic radar data. It addresses the challenge of simulating diversified and fine-grained reflection properties of user gestures. We implement and evaluate G3R using 2D videos from public data sources and self-collected real-world radar data.
  arXiv  Detail & Related papers  (2024-04-23T11:22:59Z)
• CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs [65.80187860906115]
  We propose a novel approach to improve NeRF's performance with sparse inputs. We first adopt a voxel-based ray sampling strategy to ensure that the sampled rays intersect with a certain voxel in 3D space. We then randomly sample additional points within the voxel and apply a Transformer to infer the properties of other points on each ray, which are then incorporated into the volume rendering.
  arXiv  Detail & Related papers  (2024-03-25T15:56:17Z)
• DART: Implicit Doppler Tomography for Radar Novel View Synthesis [9.26298115522881]
  DART is a Neural Radiance Field-inspired method which uses radar-specific physics to create a reflectance and transmittance-based rendering pipeline for range-Doppler images. In comparison to state-of-the-art baselines, DART synthesizes superior radar range-Doppler images from novel views across all datasets.
  arXiv  Detail & Related papers  (2024-03-06T17:54:50Z)
• Multi-Space Neural Radiance Fields [74.46513422075438]
  Existing Neural Radiance Fields (NeRF) methods suffer from the existence of reflective objects. We propose a multi-space neural radiance field (MS-NeRF) that represents the scene using a group of feature fields in parallel sub-spaces. Our approach significantly outperforms the existing single-space NeRF methods for rendering high-quality scenes.
  arXiv  Detail & Related papers  (2023-05-07T13:11:07Z)
• NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusion [107.67277084886929]
  Novel view synthesis from a single image requires inferring occluded regions of objects and scenes whilst simultaneously maintaining semantic and physical consistency with the input. We propose NerfDiff, which addresses this issue by distilling the knowledge of a 3D-aware conditional diffusion model (CDM) into NeRF through synthesizing and refining a set of virtual views at test time. We further propose a novel NeRF-guided distillation algorithm that simultaneously generates 3D consistent virtual views from the CDM samples, and finetunes the NeRF based on the improved virtual views.
  arXiv  Detail & Related papers  (2023-02-20T17:12:00Z)
• NeRF-SR: High-Quality Neural Radiance Fields using Super-Sampling [82.99453001445478]
  We present NeRF-SR, a solution for high-resolution (HR) novel view synthesis with mostly low-resolution (LR) inputs. Our method is built upon Neural Radiance Fields (NeRF) that predicts per-point density and color with a multi-layer perceptron.
  arXiv  Detail & Related papers  (2021-12-03T07:33:47Z)
• Urban Radiance Fields [77.43604458481637]
  We perform 3D reconstruction and novel view synthesis from data captured by scanning platforms commonly deployed for world mapping in urban outdoor environments. Our approach extends Neural Radiance Fields, which has been demonstrated to synthesize realistic novel images for small scenes in controlled settings. Each of these three extensions provides significant performance improvements in experiments on Street View data.
  arXiv  Detail & Related papers  (2021-11-29T15:58:16Z)
• Rethinking of Radar's Role: A Camera-Radar Dataset and Systematic Annotator via Coordinate Alignment [38.24705460170415]
  We propose a new dataset, named CRUW, with a systematic annotator and performance evaluation system. CRUW aims to classify and localize the objects in 3D purely from radar's radio frequency (RF) images. To the best of our knowledge, CRUW is the first public large-scale dataset with a systematic annotation and evaluation system.
  arXiv  Detail & Related papers  (2021-05-11T17:13:45Z)
• MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo [52.329580781898116]
  We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis. Unlike prior works on neural radiance fields that consider per-scene optimization on densely captured images, we propose a generic deep neural network that can reconstruct radiance fields from only three nearby input views via fast network inference.
  arXiv  Detail & Related papers  (2021-03-29T13:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.