AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Control

• URL: http://arxiv.org/abs/2505.03738v1
• Date: Tue, 06 May 2025 17:59:51 GMT
• Title: AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Control
• Abstract summary: We propose a framework that integrates sim-to-real reinforcement learning with trajectory optimization for real-time, adaptive whole-body control.<n>We show that AMO's consistent performance supports autonomous task execution via imitation learning.
• Score: 14.403489342466049
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humanoid robots derive much of their dexterity from hyper-dexterous whole-body movements, enabling tasks that require a large operational workspace: such as picking objects off the ground. However, achieving these capabilities on real humanoids remains challenging due to their high degrees of freedom (DoF) and nonlinear dynamics. We propose Adaptive Motion Optimization (AMO), a framework that integrates sim-to-real reinforcement learning (RL) with trajectory optimization for real-time, adaptive whole-body control. To mitigate distribution bias in motion imitation RL, we construct a hybrid AMO dataset and train a network capable of robust, on-demand adaptation to potentially O.O.D. commands. We validate AMO in simulation and on a 29-DoF Unitree G1 humanoid robot, demonstrating superior stability and an expanded workspace compared to strong baselines. Finally, we show that AMO's consistent performance supports autonomous task execution via imitation learning, underscoring the system's versatility and robustness.

Related papers

• Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL [11.048264771623836]
  Reinforcement Learning (RL) is essential for robot autonomy, but scaling it to complex tasks is time-consuming and labor-intensive.<n>We show how we can train an off-policy RL agent using purely sparse task rewards, drastically reducing the time required to learn new skills and eliminating the need for manual tuning.<n>We validate the robustness of this framework by successfully deploying complex loco-manipulation skills across different morphologies, including an arm-equipped Spot quadruped and a G1 humanoid.
  arXiv  Detail & Related papers  (2026-08-12T13:48:56Z)
• X-OP: Cross-Morphology Whole-Body Teleoperation via MPC Retargeting [3.134050607080983]
  Whole-body teleoperation is essential for scalable robot data collection in loco-manipulation tasks.<n>We propose a hierarchical whole-body teleoperation framework driven by a single XR device.<n>We show higher success rates on whole-body control tasks for both a humanoid and a mobile manipulator.
  arXiv  Detail & Related papers  (2026-06-06T01:50:59Z)
• HybridMimic: Hybrid RL-Centroidal Control for Humanoid Motion Mimicking [3.446278503007937]
  HybridMimic is a framework in which a learned policy dynamically modulates a centroidal-model-based controller.<n>It reduces the average base position tracking error by 13% compared to a state-of-the-art RL baseline.
  arXiv  Detail & Related papers  (2026-03-06T17:58:02Z)
• ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation [55.467742403416175]
  We introduce a physics-driven neural algorithm that translates large-scale motion capture to humanoid embodiments.<n>We learn a unified multimodal controller that supports both dense references and sparse task specifications.<n>Results show that ULTRA generalizes to autonomous, goal-conditioned whole-body loco-manipulation from egocentric perception.
  arXiv  Detail & Related papers  (2026-03-03T18:59:29Z)
• D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping [66.22412592525369]
  We introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine.<n>We show that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values.<n>Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping.
  arXiv  Detail & Related papers  (2026-03-01T15:32:04Z)
• AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic Manipulation [12.592383721590402]
  We introduce a unified framework, World-Model-Driven Diffusion Policy with Online Adaptive Learning (AdaWorldPolicy)<n>Our core insight is that world models provide strong supervision signals, enabling online adaptive learning in dynamic environments.<n>AdaWorldPolicy achieves state-of-the-art performance, with dynamical adaptive capacity to out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2026-02-23T17:12:25Z)
• AdaMorph: Unified Motion Retargeting via Embodiment-Aware Adaptive Transformers [49.796014934503184]
  We present a unified neural framework that enables a single model to adapt human motion to diverse robot morphologies.<n> Experimental results on 12 distinct humanoid robots demonstrate that AdaMorph effectively control across heterogeneous topologies.
  arXiv  Detail & Related papers  (2026-01-12T07:39:38Z)
• ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learning [59.64325421657381]
  Humanoid whole-body loco-manipulation promises transformative capabilities for daily service and warehouse tasks.<n>We introduce ResMimic, a two-stage residual learning framework for precise and expressive humanoid control from human motion data.<n>Results show substantial gains in task success, training efficiency, and robustness over strong baselines.
  arXiv  Detail & Related papers  (2025-10-06T17:47:02Z)
• KungfuBot2: Learning Versatile Motion Skills for Humanoid Whole-Body Control [30.738592041595933]
  We present VMS, a unified whole-body controller that enables humanoid robots to learn diverse and dynamic behaviors within a single policy.<n>Our framework integrates a hybrid tracking objective that balances local motion fidelity with global trajectory consistency.<n>We validate VMS specialization extensively in both simulation and real-world experiments, demonstrating accurate imitation of dynamic skills, stable performance over minute-long sequences, and strong generalization to unseen motions.
  arXiv  Detail & Related papers  (2025-09-20T11:31:14Z)
• Pre-Trained Video Generative Models as World Simulators [59.546627730477454]
  We propose Dynamic World Simulation (DWS) to transform pre-trained video generative models into controllable world simulators.<n>To achieve precise alignment between conditioned actions and generated visual changes, we introduce a lightweight, universal action-conditioned module.<n> Experiments demonstrate that DWS can be versatilely applied to both diffusion and autoregressive transformer models.
  arXiv  Detail & Related papers  (2025-02-10T14:49:09Z)
• ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills [46.16771391136412]
  ASAP is a two-stage framework designed to tackle the dynamics mismatch and enable agile humanoid whole-body skills.<n>In the first stage, we pre-train motion tracking policies in simulation using retargeted human motion data.<n>In the second stage, we deploy the policies in the real world and collect real-world data to train a delta (residual) action model.
  arXiv  Detail & Related papers  (2025-02-03T08:22:46Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer.<n>By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• RobotDiffuse: Motion Planning for Redundant Manipulator based on Diffusion Model [13.110235244912474]
  Redundant manipulators offer enhanced kinematic performance and versatility.<n>Motion planning for these manipulators is challenging due to increased DOFs and complex, dynamic environments.<n>This paper introduces RobotDiffuse, a diffusion model-based approach for motion planning in redundant manipulators.
  arXiv  Detail & Related papers  (2024-12-27T07:34:54Z)
• DrEureka: Language Model Guided Sim-To-Real Transfer [64.14314476811806]
  Transferring policies learned in simulation to the real world is a promising strategy for acquiring robot skills at scale. In this paper, we investigate using Large Language Models (LLMs) to automate and accelerate sim-to-real design. Our approach is capable of solving novel robot tasks, such as quadruped balancing and walking atop a yoga ball.
  arXiv  Detail & Related papers  (2024-06-04T04:53:05Z)
• Task and Domain Adaptive Reinforcement Learning for Robot Control [0.34137115855910755]
  We present a novel adaptive agent to dynamically adapt policy in response to different tasks and environmental conditions. The agent is trained using a custom, highly parallelized simulator built on IsaacGym. We perform zero-shot transfer to fly the blimp in the real world to solve various tasks.
  arXiv  Detail & Related papers  (2024-04-29T14:02:02Z)
• Combining model-predictive control and predictive reinforcement learning for stable quadrupedal robot locomotion [0.0]
  We study how this can be achieved by a combination of model-predictive and predictive reinforcement learning controllers. In this work, we combine both control methods to address the quadrupedal robot stable gate generation problem.
  arXiv  Detail & Related papers  (2023-07-15T09:22:37Z)
• Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robots [14.760426243769308]
  In this work, we demonstrate robust sim-to-real transfer of control policies on the BALLU robots via system identification. Rather than relying on standard supervised learning formulations, we utilize deep reinforcement learning to train an external force policy. We analyze the improved simulation fidelity by comparing the simulation trajectories against the real-world ones.
  arXiv  Detail & Related papers  (2023-03-16T18:49:05Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)
• OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation [50.59541802645156]
  Operational Space Control (OSC) has been used as an effective task-space controller for manipulation. We propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of OSC that compensates for modeling errors. We evaluate our method on a variety of simulated manipulation problems, and find substantial improvements over an array of controller baselines.
  arXiv  Detail & Related papers  (2021-10-02T01:21:38Z)
• ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation [99.2543521972137]
  ReLMoGen is a framework that combines a learned policy to predict subgoals and a motion generator to plan and execute the motion needed to reach these subgoals. Our method is benchmarked on a diverse set of seven robotics tasks in photo-realistic simulation environments. ReLMoGen shows outstanding transferability between different motion generators at test time, indicating a great potential to transfer to real robots.
  arXiv  Detail & Related papers  (2020-08-18T08:05:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.