Make Tracking Easy: Neural Motion Retargeting for Humanoid Whole-body Control

• URL: http://arxiv.org/abs/2603.22201v1
• Date: Mon, 23 Mar 2026 16:59:00 GMT
• Title: Make Tracking Easy: Neural Motion Retargeting for Humanoid Whole-body Control
• Abstract summary: We propose a Neural Motion Retargeting framework to group heterogeneous movements into latent motifs.<n>This strategy significantly reduces the computational massively parallel learning experts.<n> references accelerate the convergence of whole-body control policies.
• Score: 43.01629060165596
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humanoid robots require diverse motor skills to integrate into complex environments, but bridging the kinematic and dynamic embodiment gap from human data remains a major bottleneck. We demonstrate through Hessian analysis that traditional optimization-based retargeting is inherently non-convex and prone to local optima, leading to physical artifacts like joint jumps and self-penetration. To address this, we reformulate the targeting problem as learning data distribution rather than optimizing optimal solutions, where we propose NMR, a Neural Motion Retargeting framework that transforms static geometric mapping into a dynamics-aware learned process. We first propose Clustered-Expert Physics Refinement (CEPR), a hierarchical data pipeline that leverages VAE-based motion clustering to group heterogeneous movements into latent motifs. This strategy significantly reduces the computational overhead of massively parallel reinforcement learning experts, which project and repair noisy human demonstrations onto the robot's feasible motion manifold. The resulting high-fidelity data supervises a non-autoregressive CNN-Transformer architecture that reasons over global temporal context to suppress reconstruction noise and bypass geometric traps. Experiments on the Unitree G1 humanoid across diverse dynamic tasks (e.g., martial arts, dancing) show that NMR eliminates joint jumps and significantly reduces self-collisions compared to state-of-the-art baselines. Furthermore, NMR-generated references accelerate the convergence of downstream whole-body control policies, establishing a scalable path for bridging the human-robot embodiment gap.

Related papers

• Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos [6.238518905285812]
  Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots.<n>We propose Direct Dynamic Retargeting ( DDR), a novel single-stage framework that generates high-fidelity, dynamically feasible trajectories directly from expert videos.<n>Our experiments demonstrate that bypassing the geometric bias allows DDR to outperform state-of-the-art baselines in demonstration tracking accuracy.
  arXiv  Detail & Related papers  (2026-05-22T15:33:40Z)
• ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting [8.302518310051232]
  We propose a bilevel optimization framework that adapts reference motions to a robot's morphology.<n>By integrating directly with physics simulation, we produce physically plausible motions that facilitate robust imitation learning.
  arXiv  Detail & Related papers  (2026-05-07T17:20:15Z)
• SPARK: Skeleton-Parameter Aligned Retargeting on Humanoid Robots with Kinodynamic Trajectory Optimization [9.553570391005257]
  Human motion provides rich priors for training general-purpose humanoid control policies, but raw demonstrations are often incompatible with a robot's kinematics and dynamics.<n>We present a two-stage pipeline for generating natural and dynamically feasible motion references from task-space human data.
  arXiv  Detail & Related papers  (2026-03-12T03:03:14Z)
• Kinodynamic Motion Retargeting for Humanoid Locomotion via Multi-Contact Whole-Body Trajectory Optimization [6.47799404542057]
  We present the KinoDynamic Motion Retargeting (KDMR) framework, a novel approach for humanoid locomotion.<n>Our method automatically detects heel-toe contact events to accurately replicate human-like contact patterns.<n> Experimental results demonstrate that KDMR significantly outperforms purely kinematic methods.
  arXiv  Detail & Related papers  (2026-03-10T17:51:17Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• AdaMorph: Unified Motion Retargeting via Embodiment-Aware Adaptive Transformers [49.796014934503184]
  We present a unified neural framework that enables a single model to adapt human motion to diverse robot morphologies.<n> Experimental results on 12 distinct humanoid robots demonstrate that AdaMorph effectively control across heterogeneous topologies.
  arXiv  Detail & Related papers  (2026-01-12T07:39:38Z)
• UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Pass [83.7071371474926]
  UniSH is a unified, feed-forward framework for joint metric-scale 3D scene and human reconstruction.<n>Our framework bridges strong, disparate priors from scene reconstruction and HMR.<n>Our model achieves state-of-the-art performance on human-centric scene reconstruction.
  arXiv  Detail & Related papers  (2026-01-03T16:06:27Z)
• OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction [76.44108003274955]
  A dominant paradigm for teaching humanoid robots complex skills is to retarget human motions as kinematic references to train reinforcement learning policies.<n>We introduce OmniRetarget, an interaction-preserving data generation engine based on an interaction mesh.<n>By minimizing the Laplacian deformation between the human and robot meshes, OmniRetarget generates kinematically feasible trajectories.
  arXiv  Detail & Related papers  (2025-09-30T17:59:02Z)
• ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer [58.49950218437718]
  We present ReCoM, an efficient framework for generating high-fidelity and generalizable human body motions synchronized with speech.<n>The core innovation lies in the Recurrent Embedded Transformer (RET), which integrates Dynamic Embedding Regularization (DER) into a Vision Transformer (ViT) core architecture.<n>To enhance model robustness, we incorporate the proposed DER strategy, which equips the model with dual capabilities of noise resistance and cross-domain generalization.
  arXiv  Detail & Related papers  (2025-03-27T16:39:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.