HybridSim: A Physics-Learning Hybrid Digital Twin for mmWave Human Sensing

• URL: http://arxiv.org/abs/2607.15806v1
• Date: Fri, 17 Jul 2026 10:20:31 GMT
• Title: HybridSim: A Physics-Learning Hybrid Digital Twin for mmWave Human Sensing
• Abstract summary: HybridSim is a physics-learning hybrid simulator that synthesizes mmWave radar signals from dynamic human meshes under a fixed indoor room configuration.<n>We show improved agreement with a physically based reference and consistent gains on downstream radar-based human sensing tasks when using HybridSim for site-specific data augmentation.
• Score: 16.25180983132509
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-fidelity simulation of mmWave radar signals for dynamic human motion is valuable for developing radar-based human sensing models; yet collecting accurately labeled measurements for a specific deployment site remains expensive. We present HybridSim, a physics-learning hybrid simulator that synthesizes mmWave radar signals from dynamic human meshes under a fixed indoor room configuration, explicitly decoupling propagation into two components. To parameterize the human subject, we use a tri-plane representation to extract human features and a Graph Convolutional Network to stabilize optimization and mitigate gradient instability. The direct signal path is modeled via an inverse-rendering formulation with a microfacet BRDF to capture primary surface reflections. In parallel, the indirect path is approximated by combining 3D Gaussian Splatting with a virtual-receiver geometry to fit and reproduce site-specific multipath interference patterns, achieving substantially lower computational cost than explicit full ray tracing. Experiments in a fixed-room setting show improved agreement with a physically based reference and consistent gains on downstream radar-based human sensing tasks when using HybridSim for site-specific data augmentation.

Related papers

• mmSimPrior: Learning Simulation Priors for Data-Efficient and Generalizable Real-World Radar-based Human Motion Reconstruction [12.731621912265986]
  We present mmSimPrior, a simulation-pretrained framework that factorizes transferable knowledge into signal, motion, and radar-to-motion mapping priors.<n>Experiments on mmSimPrior-Real and RT-Pose demonstrate consistent gains.
  arXiv  Detail & Related papers  (2026-07-25T00:56:44Z)
• A Two-Stage Motion-Aware Framework for mmWave-based Human Mesh Recovery [13.982684122303896]
  We propose a two-stage framework for radar-based human body reconstruction.<n>First, we introduce a human reflection extraction module that performs coarse-to-fine localization.<n>Second, we design a motion-aware mesh recovery network that reconstructs the human body.
  arXiv  Detail & Related papers  (2026-05-08T22:34:03Z)
• HSImul3R: Physics-in-the-Loop Reconstruction of Simulation-Ready Human-Scene Interactions [69.43316445143408]
  We present HSImul3R, a unified framework for simulation-ready 3D reconstruction of human-scene interactions.<n>Existing methods suffer from a perception-simulation gap, leading to instability in physics engines and failure in embodied AI applications.
  arXiv  Detail & Related papers  (2026-03-16T17:58:33Z)
• Robust Mesh Saliency GT Acquisition in VR via View Cone Sampling and Geometric Smoothing [59.12032628787018]
  3D mesh saliency ground truth is essential for human-centric visual modeling in virtual reality (VR)<n>Current VR eye-tracking pipelines rely on single ray sampling and Euclidean smoothing, triggering texture attention and signal leakage across gaps.<n>This paper proposes a robust framework to address these limitations.
  arXiv  Detail & Related papers  (2026-01-06T05:20:12Z)
• UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Pass [83.7071371474926]
  UniSH is a unified, feed-forward framework for joint metric-scale 3D scene and human reconstruction.<n>Our framework bridges strong, disparate priors from scene reconstruction and HMR.<n>Our model achieves state-of-the-art performance on human-centric scene reconstruction.
  arXiv  Detail & Related papers  (2026-01-03T16:06:27Z)
• RadarGen: Automotive Radar Point Cloud Generation from Cameras [64.69976771710057]
  We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery.<n>RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form.<n>We show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data.
  arXiv  Detail & Related papers  (2025-12-19T18:57:33Z)
• RadHARSimulator V2: Video to Doppler Generator [1.3706331473063882]
  A simulator that directly generates Doppler spectra from recorded video footage is presented in this paper.<n>Both computer vision and radar modules are included in the simulator.<n>A hybrid parallel serial neural network architecture is proposed for radar-based HAR.
  arXiv  Detail & Related papers  (2025-11-12T06:15:53Z)
• Resource-Efficient Beam Prediction in mmWave Communications with Multimodal Realistic Simulation Framework [57.994965436344195]
  Beamforming is a key technology in millimeter-wave (mmWave) communications that improves signal transmission by optimizing directionality and intensity.<n> multimodal sensing-aided beam prediction has gained significant attention, using various sensing data to predict user locations or network conditions.<n>Despite its promising potential, the adoption of multimodal sensing-aided beam prediction is hindered by high computational complexity, high costs, and limited datasets.
  arXiv  Detail & Related papers  (2025-04-07T15:38:25Z)
• A Lightweight Human Pose Estimation Approach for Edge Computing-Enabled Metaverse with Compressive Sensing [23.884862152830184]
  The ability to estimate 3D movements of users over edge computing-enabled networks, such as 5G/6G networks, is a key enabler for the new era of extended reality (XR) and Metaverse applications. Recent advancements in deep learning have shown advantages over optimization techniques for estimating 3D human poses given spare measurements from sensor signals. We propose a novel approach for redundancy removal and lightweight transmission of IMU signals over noisy wireless environments.
  arXiv  Detail & Related papers  (2024-08-26T02:57:23Z)
• Differentiable and Learnable Wireless Simulation with Geometric Transformers [13.538643388955768]
  Wi-GATr is a fully-learnable neural simulation surrogate designed to predict the channel observations based on scene primitives. We evaluate our approach on a range of tasks and find that Wi-GATr is accurate, fast, sample-efficient, and robust to symmetry-induced transformations.
  arXiv  Detail & Related papers  (2024-06-21T09:14:11Z)
• Differentiable Radio Frequency Ray Tracing for Millimeter-Wave Sensing [29.352303349003165]
  We propose DiffSBR, a differentiable framework for mmWave-based 3D reconstruction. DiffSBR incorporates a differentiable ray tracing engine to simulate radar point clouds from virtual 3D models. Experiments using various radar hardware validate DiffSBR's capability for fine-grained 3D reconstruction.
  arXiv  Detail & Related papers  (2023-11-22T06:13:39Z)
• Synthetic Wave-Geometric Impulse Responses for Improved Speech Dereverberation [69.1351513309953]
  We show that accurately simulating the low-frequency components of Room Impulse Responses (RIRs) is important to achieving good dereverberation. We demonstrate that speech dereverberation models trained on hybrid synthetic RIRs outperform models trained on RIRs generated by prior geometric ray tracing methods.
  arXiv  Detail & Related papers  (2022-12-10T20:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.