Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos

• URL: http://arxiv.org/abs/2605.23762v1
• Date: Fri, 22 May 2026 15:33:40 GMT
• Title: Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos
• Abstract summary: Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots.<n>We propose Direct Dynamic Retargeting ( DDR), a novel single-stage framework that generates high-fidelity, dynamically feasible trajectories directly from expert videos.<n>Our experiments demonstrate that bypassing the geometric bias allows DDR to outperform state-of-the-art baselines in demonstration tracking accuracy.
• Score: 6.238518905285812
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots. However, translating human motion to humanoids requires overcoming significant morphological mismatches. Standard approaches rely on Geometric Retargeting or Indirect Dynamic Retargeting pipelines. We identify that these intermediate kinematic projections introduce a geometric bias, restricting the search space and yielding suboptimal dynamic behaviors. In this paper, we propose Direct Dynamic Retargeting (DDR), a novel single-stage framework that generates high-fidelity, dynamically feasible trajectories directly from expert videos. By formulating the problem in the task space and leveraging a sampling-based Model Predictive Control solver within a physics simulator, DDR natively optimizes over complex contact sequences while mitigating input drift. Our experiments demonstrate that bypassing the geometric bias allows DDR to outperform state-of-the-art baselines in demonstration tracking accuracy. Furthermore, we establish that providing such physically viable references to RL agents accelerates training convergence and enhances the final execution of agile and balancing behaviors. Source code will be made publicly available.

Related papers

• Native Video-Action Pretraining for Generalizable Robot Control [77.59831077077176]
  LingBot-VA 2.0 is a video-action foundation model built from the ground up for embodiment.<n>To meet the demands of high-frequency inference, our model employs a sparse MoE backbone.<n>Real-world deployment validates LingBot-VA 2.0 as a robust foundation model.
  arXiv  Detail & Related papers  (2026-07-09T16:15:43Z)
• IOI: Decoupling Kinematics and Physics for Interactive World Models [46.3330122411516]
  We propose IOI, a hybrid interactive world model integrating kinematics with learned physical dynamics.<n> IOI introduces explicit kinematic guidance, computing kinematics sequences for accurate motion trajectories.<n> experiments on the RoboTwin benchmark validate IOI across kinematic fidelity, out-of-distribution, and policy evaluation.
  arXiv  Detail & Related papers  (2026-06-22T13:09:34Z)
• ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting [8.302518310051232]
  We propose a bilevel optimization framework that adapts reference motions to a robot's morphology.<n>By integrating directly with physics simulation, we produce physically plausible motions that facilitate robust imitation learning.
  arXiv  Detail & Related papers  (2026-05-07T17:20:15Z)
• Make Tracking Easy: Neural Motion Retargeting for Humanoid Whole-body Control [43.01629060165596]
  We propose a Neural Motion Retargeting framework to group heterogeneous movements into latent motifs.<n>This strategy significantly reduces the computational massively parallel learning experts.<n> references accelerate the convergence of whole-body control policies.
  arXiv  Detail & Related papers  (2026-03-23T16:59:00Z)
• RoboForge: Physically Optimized Text-guided Whole-Body Locomotion for Humanoids [20.796118584632904]
  We present a unified latent-driven framework that bridges natural language and whole-body locomotion.<n>Our framework provides a practical path toward deployable text-guided humanoid intelligence.
  arXiv  Detail & Related papers  (2026-03-18T17:02:56Z)
• HSImul3R: Physics-in-the-Loop Reconstruction of Simulation-Ready Human-Scene Interactions [69.43316445143408]
  We present HSImul3R, a unified framework for simulation-ready 3D reconstruction of human-scene interactions.<n>Existing methods suffer from a perception-simulation gap, leading to instability in physics engines and failure in embodied AI applications.
  arXiv  Detail & Related papers  (2026-03-16T17:58:33Z)
• SPARK: Skeleton-Parameter Aligned Retargeting on Humanoid Robots with Kinodynamic Trajectory Optimization [9.553570391005257]
  Human motion provides rich priors for training general-purpose humanoid control policies, but raw demonstrations are often incompatible with a robot's kinematics and dynamics.<n>We present a two-stage pipeline for generating natural and dynamically feasible motion references from task-space human data.
  arXiv  Detail & Related papers  (2026-03-12T03:03:14Z)
• Kinodynamic Motion Retargeting for Humanoid Locomotion via Multi-Contact Whole-Body Trajectory Optimization [6.47799404542057]
  We present the KinoDynamic Motion Retargeting (KDMR) framework, a novel approach for humanoid locomotion.<n>Our method automatically detects heel-toe contact events to accurately replicate human-like contact patterns.<n> Experimental results demonstrate that KDMR significantly outperforms purely kinematic methods.
  arXiv  Detail & Related papers  (2026-03-10T17:51:17Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• Generalizable Geometric Prior and Recurrent Spiking Feature Learning for Humanoid Robot Manipulation [90.90219129619344]
  This paper presents a novel R-prior-S, Recurrent Geometric-priormodal Policy with Spiking features.<n>To ground high-level reasoning in physical reality, we leverage lightweight 2D geometric inductive biases.<n>For the data efficiency issue in robotic action generation, we introduce a Recursive Adaptive Spiking Network.
  arXiv  Detail & Related papers  (2026-01-13T23:36:30Z)
• AdaMorph: Unified Motion Retargeting via Embodiment-Aware Adaptive Transformers [49.796014934503184]
  We present a unified neural framework that enables a single model to adapt human motion to diverse robot morphologies.<n> Experimental results on 12 distinct humanoid robots demonstrate that AdaMorph effectively control across heterogeneous topologies.
  arXiv  Detail & Related papers  (2026-01-12T07:39:38Z)
• Physics-Grounded Motion Forecasting via Equation Discovery for Trajectory-Guided Image-to-Video Generation [54.42523027597904]
  We introduce a novel framework that integrates symbolic regression and trajectory-guided image-to-video (I2V) models for physics-grounded video forecasting.<n>Our approach extracts motion trajectories from input videos, uses a retrieval-based pre-training mechanism to enhance symbolic regression, and discovers equations of motion to forecast physically accurate future trajectories.
  arXiv  Detail & Related papers  (2025-07-09T13:28:42Z)
• Dynamic Manipulation of Deformable Objects in 3D: Simulation, Benchmark and Learning Strategy [88.8665000676562]
  Prior methods often simplify the problem to low-speed or 2D settings, limiting their applicability to real-world 3D tasks.<n>To mitigate data scarcity, we introduce a novel simulation framework and benchmark grounded in reduced-order dynamics.<n>We propose Dynamics Informed Diffusion Policy (DIDP), a framework that integrates imitation pretraining with physics-informed test-time adaptation.
  arXiv  Detail & Related papers  (2025-05-23T03:28:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.