A Two-Stage Motion-Aware Framework for mmWave-based Human Mesh Recovery

• URL: http://arxiv.org/abs/2605.08530v1
• Date: Fri, 08 May 2026 22:34:03 GMT
• Title: A Two-Stage Motion-Aware Framework for mmWave-based Human Mesh Recovery
• Abstract summary: We propose a two-stage framework for radar-based human body reconstruction.<n>First, we introduce a human reflection extraction module that performs coarse-to-fine localization.<n>Second, we design a motion-aware mesh recovery network that reconstructs the human body.
• Score: 13.982684122303896
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Millimeter-wave (mmWave) radar has emerged as a promising sensing modality for human perception due to its robustness under challenging environmental conditions and strong privacy-preserving properties. However, recovering accurate 3D human body meshes from radar observations remains difficult due to severe signal clutter and the inherently partial nature of radar measurements. Previous works typically adopt end-to-end frameworks that directly regress human body parameters from raw radar data, without decoupling signal interpretation from geometric reasoning or exploiting temporal motion cues, limiting learning performance. To address this, we propose a two-stage framework for radar-based human body reconstruction. First, we introduce a human reflection extraction module that performs coarse-to-fine localization and voxel-wise segmentation to produce a confidence-weighted radar volume encoding voxel-level human likelihood. Second, we design a motion-aware mesh recovery network that reconstructs the human body by jointly modeling per-frame geometry and inter-frame dynamics using a dual-branch architecture. Extensive experiments demonstrate that the proposed method outperforms existing approaches while maintaining computational efficiency.

Related papers

• HybridSim: A Physics-Learning Hybrid Digital Twin for mmWave Human Sensing [16.25180983132509]
  HybridSim is a physics-learning hybrid simulator that synthesizes mmWave radar signals from dynamic human meshes under a fixed indoor room configuration.<n>We show improved agreement with a physically based reference and consistent gains on downstream radar-based human sensing tasks when using HybridSim for site-specific data augmentation.
  arXiv  Detail & Related papers  (2026-07-17T10:20:31Z)
• Towards 3D heart mesh generation using contactless radar imaging and physics-informed neural network [31.20753776114221]
  Millimeter-wave (mmWave) radar and its Synthetic Aperture Radar (SAR) mode offer a privacy-preserving and portable point-of-care clinical applications.<n>We propose SAR2Mesh, a novel framework that reformulates the task as a coarse-to-fine mesh deformation process.<n>We introduce a geometry-aware feature projection module to extract multi-view features via 3D-to-2D sampling, and a physics-informed radar loss to enforce consistency between the predicted geometry and raw radar echoes.
  arXiv  Detail & Related papers  (2026-05-25T16:20:52Z)
• HSImul3R: Physics-in-the-Loop Reconstruction of Simulation-Ready Human-Scene Interactions [69.43316445143408]
  We present HSImul3R, a unified framework for simulation-ready 3D reconstruction of human-scene interactions.<n>Existing methods suffer from a perception-simulation gap, leading to instability in physics engines and failure in embodied AI applications.
  arXiv  Detail & Related papers  (2026-03-16T17:58:33Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• Towards Geometry-Aware and Motion-Guided Video Human Mesh Recovery [60.51998732898099]
  We introduce HMRMamba, a new paradigm for 3D Human Mesh Recovery.<n>It pioneers the use of Structured State Space Models for their efficiency and long-range modeling prowess.<n>Our framework is distinguished by two core contributions. First, the Geometry-Aware Lifting Module, featuring a novel dual-scan Mamba architecture.
  arXiv  Detail & Related papers  (2026-01-29T08:05:02Z)
• UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Pass [83.7071371474926]
  UniSH is a unified, feed-forward framework for joint metric-scale 3D scene and human reconstruction.<n>Our framework bridges strong, disparate priors from scene reconstruction and HMR.<n>Our model achieves state-of-the-art performance on human-centric scene reconstruction.
  arXiv  Detail & Related papers  (2026-01-03T16:06:27Z)
• World-Coordinate Human Motion Retargeting via SAM 3D Body [0.3735787582927221]
  We propose a framework that leverages SAM 3D Body as a frozen perception backbone.<n>We retarget the reconstructed motion to the Unitree G1 humanoid using a kinematics-aware two-stage inverse kinematics pipeline.<n>Results on real monocular videos show that our method has stable world trajectories and reliable robot representation.
  arXiv  Detail & Related papers  (2025-12-25T08:30:04Z)
• RadarLLM: Empowering Large Language Models to Understand Human Motion from Millimeter-wave Point Cloud Sequence [10.115852646162843]
  We present Radar-LLM, the first framework that leverages large language models (LLMs) for human understanding using millimeter-wave radar as the sensing modality.<n>To address data scarcity, we introduce a physics-aware pipeline synthesis that generates realistic radar-text pairs from motion-text datasets.<n>Radar-LLM achieves state-of-the-art performance across both synthetic and real-world benchmarks, enabling accurate translation of millimeter-wave signals to natural language descriptions.
  arXiv  Detail & Related papers  (2025-04-14T04:18:25Z)
• ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer [58.49950218437718]
  We present ReCoM, an efficient framework for generating high-fidelity and generalizable human body motions synchronized with speech.<n>The core innovation lies in the Recurrent Embedded Transformer (RET), which integrates Dynamic Embedding Regularization (DER) into a Vision Transformer (ViT) core architecture.<n>To enhance model robustness, we incorporate the proposed DER strategy, which equips the model with dual capabilities of noise resistance and cross-domain generalization.
  arXiv  Detail & Related papers  (2025-03-27T16:39:40Z)
• Diffusion Model is a Good Pose Estimator from 3D RF-Vision [32.72703340013302]
  Human pose estimation (HPE) from Radio Frequency vision (RF-vision) performs human sensing using RF signals. mmWave radar has emerged as a promising RF-vision sensor, providing radar point clouds by processing RF signals. This work proposes mmDiff, a novel diffusion-based pose estimator tailored for noisy radar data.
  arXiv  Detail & Related papers  (2024-03-24T15:39:52Z)
• Machine learning for phase-resolved reconstruction of nonlinear ocean wave surface elevations from sparse remote sensing data [37.69303106863453]
  We propose a novel approach for phase-resolved wave surface reconstruction using neural networks. Our approach utilizes synthetic yet highly realistic training data on uniform one-dimensional grids.
  arXiv  Detail & Related papers  (2023-05-18T12:30:26Z)
• Physics-informed inference of aerial animal movements from weather radar data [0.0]
  We tackle the problem of reconstructing movement patterns from available radar data. A convolutional decoder maps the inferred latent system states back to the physical space in which the known radar observation model can be applied. Experiments on synthetic radar data show promising results in terms of reconstruction quality and data-efficiency.
  arXiv  Detail & Related papers  (2022-11-08T20:20:52Z)
• HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar [30.51398364813315]
  This paper introduces a novel human pose estimation benchmark, Human Pose with Millimeter Wave Radar (HuPR) This dataset is created using cross-calibrated mmWave radar sensors and a monocular RGB camera for cross-modality training of radar-based human pose estimation.
  arXiv  Detail & Related papers  (2022-10-22T22:28:40Z)
• Risk-Averse MPC via Visual-Inertial Input and Recurrent Networks for Online Collision Avoidance [95.86944752753564]
  We propose an online path planning architecture that extends the model predictive control (MPC) formulation to consider future location uncertainties. Our algorithm combines an object detection pipeline with a recurrent neural network (RNN) which infers the covariance of state estimates. The robustness of our methods is validated on complex quadruped robot dynamics and can be generally applied to most robotic platforms.
  arXiv  Detail & Related papers  (2020-07-28T07:34:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.