mmIR: Frequency-Space Inverse Rendering for 3D Millimeter-Wave Radar ADC Synthesis

• URL: http://arxiv.org/abs/2608.28913v1
• Date: Fri, 28 Aug 2026 22:21:47 GMT
• Title: mmIR: Frequency-Space Inverse Rendering for 3D Millimeter-Wave Radar ADC Synthesis
• Abstract summary: We present mmIR, an open-source differentiable frequency-modulated continuouswave radar inverse that fits a physics-based forward model to real captures and re-renders from dense virtual apertures to synthesize high-resolution 3D radar data.<n>Because radar resolution is too coarse to recover geometry directly, mmIR performs LiDAR-assisted inverse rendering using LiDAR-derived meshes as a geometric scaffold.<n>On seven outdoor and six indoor ColoRadar scenes, mmIR achieves 0.914 mean Pearson correlation on range-azimuth maps versus 0.307 for Sionna-RT.
• Score: 18.777328493392762
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-resolution 3D radar data is scarce. Commodity mmWave sensors use small antenna arrays that limit angular resolution to several degrees, and existing datasets provide only 2D range-azimuth maps or sparse point clouds rather than raw analog-to-digital converter (ADC) signals. Hardware scaling is expensive, synthetic-aperture scanning is impractical at fleet scale, and learned synthesis methods are bottlenecked by the very data shortage they aim to address. We present mmIR, an open-source differentiable frequency-modulated continuous-wave (FMCW) radar inverse renderer that fits a physics-based forward model to real captures and re-renders from dense virtual apertures to synthesize high-resolution 3D radar data. Because radar resolution is too coarse to recover geometry directly, mmIR performs LiDAR-assisted inverse rendering: using LiDAR-derived meshes as a geometric scaffold, mmIR optimizes per-vertex International Telecommunication Union (ITU) physics materials, vertex normals, and antenna beam patterns through end-to-end automatic differentiation of a phase-coherent multiple-input multiple-output (MIMO) forward model with multi-bounce propagation, polarization, and free-space diffraction. On seven outdoor and six indoor ColoRadar scenes, mmIR achieves 0.914 mean Pearson correlation on range-azimuth maps versus 0.307 for Sionna-RT. Scenes trained on a cascaded imaging radar transfer to a co-located single-chip radar without re-training (0.554 correlation), and dense virtual arrays (100x100 elements) produce single-frame 3D occupancy validated against LiDAR. Project page: https://mmwave-inverse-rendering.github.io/

Related papers

• 3D Point Splatting for mmWave Radar Novel View Synthesis [18.777328493392762]
  novel view synthesis (NVS) for millimeter-wave (mmWave) radar requires acompute that is physically faithful, complex-valued, and multi-viewpoint-tractable.<n>No prior method achieves these three properties simultaneously.<n>Differentiable Monte Carlo (MC) ray tracers implement the radar forward model directly with explicit material modeling and complex outputs.<n>We propose 3D Point Point Splatting, the first differentiable point for radar, derived directly from the standard solid-angle form of the radar.
  arXiv  Detail & Related papers  (2026-09-10T17:52:17Z)
• DeeperRadar: End-to-End MIMO Radar Design and Multi-Modal Fusion for Autonomous Vehicle Perception [5.385471397428483]
  DeeperRadar is a framework that co-designs radar sensing and 3D detection for autonomous mobility.<n>It learns a sparse acquisition pattern end-to-end with the fusion model.<n>It discovers sparse, task-aware radar configurations that match or exceed full-array baselines.
  arXiv  Detail & Related papers  (2026-07-19T17:26:34Z)
• RadarTwin: Scene-Specific mmWave Radar Simulation and Learning for Mobile Indoor Perception [2.0255467447384015]
  RadarTwin is a framework for generating deployment-specific radar training data before real data collection.<n>We show that simulated and real radar share the same object-discriminative shape and material features.<n>A representation trained on simulation alone recognizes real objects at 2.5 times chance with no real radar labels, and a few labeled examples raise this to 95.3% on a 12-way recognition task.
  arXiv  Detail & Related papers  (2026-06-24T05:50:26Z)
• RadarSim: Simulating Single-Chip Radar via Multimodal Neural Fields [49.41438378775056]
  Radars are an ideal complement to cameras, with cameras offering fine angular resolution and radars providing metric depth and robustness under adverse weather.<n>RadarSim produces sharper geometry and Doppler range frames than radar-only reconstructions.
  arXiv  Detail & Related papers  (2026-05-25T21:03:40Z)
• RadioGen3D: 3D Radio Map Generation via Adversarial Learning on Large-Scale Synthetic Data [62.63849426834315]
  Radio maps are essential for efficient radio resource management in future 6G and low-altitude networks.<n>Deep learning (DL) techniques have emerged as an efficient alternative to conventional ray-tracing for radio map estimation.<n>We present the RadioGen3D framework to capture essential 3D signal propagation characteristics and antenna polarization effects.
  arXiv  Detail & Related papers  (2026-02-21T07:50:05Z)
• Sim2Radar: Toward Bridging the Radar Sim-to-Real Gap with VLM-Guided Scene Reconstruction [2.3510064024442374]
  Sim2Radar is an end-to-end framework that synthesizes training radar data directly from single-view RGB images.<n>Sim2Radar reconstructs a material-aware 3D scene by combining monocular depth estimation, segmentation, and vision-language reasoning.<n> Evaluated on real-world indoor scenes, Sim2Radar improves downstream 3D radar perception via transfer learning.
  arXiv  Detail & Related papers  (2026-02-10T10:56:47Z)
• RadarGen: Automotive Radar Point Cloud Generation from Cameras [64.69976771710057]
  We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery.<n>RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form.<n>We show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data.
  arXiv  Detail & Related papers  (2025-12-19T18:57:33Z)
• RadioDiff-3D: A 3D$\times$3D Radio Map Dataset and Generative Diffusion Based Benchmark for 6G Environment-Aware Communication [76.6171399066216]
  UrbanRadio3D is a large-scale, high-resolution 3D RM dataset constructed via ray tracing in realistic urban environments.<n>RadioDiff-3D is a diffusion-model-based generative framework utilizing the 3D convolutional architecture.<n>This work provides a foundational dataset and benchmark for future research in 3D environment-aware communication.
  arXiv  Detail & Related papers  (2025-07-16T11:54:08Z)
• Targetless 6DoF Calibration of LiDAR and 2D Scanning Radar Based on Cylindrical Occupancy [8.895838973148452]
  LiRaCo is a targetless calibration approach for the extrinsic 6DoF calibration of LiDAR and Radar sensors.<n>LiRaCo leverages a spatial occupancy consistency between LiDAR point clouds and Radar scans in a common cylindrical representation.<n>A cost function involving extrinsic calibration parameters is formulated based on the spatial overlap of 3D grids and LiDAR points.
  arXiv  Detail & Related papers  (2025-03-21T10:09:04Z)
• V2X-R: Cooperative LiDAR-4D Radar Fusion with Denoising Diffusion for 3D Object Detection [64.93675471780209]
  We present V2X-R, the first simulated V2X dataset incorporating LiDAR, camera, and 4D radar.<n>V2X-R contains 12,079 scenarios with 37,727 frames of LiDAR and 4D radar point clouds, 150,908 images, and 170,859 annotated 3D vehicle bounding boxes.<n>We propose a novel cooperative LiDAR-4D radar fusion pipeline for 3D object detection and implement it with various fusion strategies.
  arXiv  Detail & Related papers  (2024-11-13T07:41:47Z)
• Differentiable Radio Frequency Ray Tracing for Millimeter-Wave Sensing [29.352303349003165]
  We propose DiffSBR, a differentiable framework for mmWave-based 3D reconstruction. DiffSBR incorporates a differentiable ray tracing engine to simulate radar point clouds from virtual 3D models. Experiments using various radar hardware validate DiffSBR's capability for fine-grained 3D reconstruction.
  arXiv  Detail & Related papers  (2023-11-22T06:13:39Z)
• Bi-LRFusion: Bi-Directional LiDAR-Radar Fusion for 3D Dynamic Object Detection [78.59426158981108]
  We introduce a bi-directional LiDAR-Radar fusion framework, termed Bi-LRFusion, to tackle the challenges and improve 3D detection for dynamic objects. We conduct extensive experiments on nuScenes and ORR datasets, and show that our Bi-LRFusion achieves state-of-the-art performance for detecting dynamic objects.
  arXiv  Detail & Related papers  (2023-06-02T10:57:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.