Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning

• URL: http://arxiv.org/abs/2408.14472v1
• Date: Mon, 26 Aug 2024 17:59:03 GMT
• Title: Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning
• Authors: Xinyang Gu, Yen-Jen Wang, Xiang Zhu, Chengming Shi, Yanjiang Guo, Yichen Liu, Jianyu Chen,
• Abstract summary: We introduce Denoising World Model Learning (DWL), an end-to-end reinforcement learning framework for humanoid locomotion control. DWL demonstrates the world's first humanoid robot to master real-world challenging terrains such as snowy and inclined land in the wild, up and down stairs, and extremely uneven terrains.
• Score: 11.648198428063415
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humanoid robots, with their human-like skeletal structure, are especially suited for tasks in human-centric environments. However, this structure is accompanied by additional challenges in locomotion controller design, especially in complex real-world environments. As a result, existing humanoid robots are limited to relatively simple terrains, either with model-based control or model-free reinforcement learning. In this work, we introduce Denoising World Model Learning (DWL), an end-to-end reinforcement learning framework for humanoid locomotion control, which demonstrates the world's first humanoid robot to master real-world challenging terrains such as snowy and inclined land in the wild, up and down stairs, and extremely uneven terrains. All scenarios run the same learned neural network with zero-shot sim-to-real transfer, indicating the superior robustness and generalization capability of the proposed method.

Related papers

• GenTe: Generative Real-world Terrains for General Legged Robot Locomotion Control [3.5594486521440323]
  GenTe is a framework for generating physically realistic and adaptable terrains to train generalizable locomotion policies. By leveraging function-calling techniques and reasoning capabilities of Vision-Language Models, GenTe generates complex, contextually relevant terrains. Experiments demonstrate improved generalization and robustness in bipedal robot locomotion.
  arXiv  Detail & Related papers  (2025-04-14T09:01:44Z)
• Humanoid Whole-Body Locomotion on Narrow Terrain via Dynamic Balance and Reinforcement Learning [54.26816599309778]
  We propose a novel whole-body locomotion algorithm based on dynamic balance and Reinforcement Learning (RL) Specifically, we introduce a dynamic balance mechanism by leveraging an extended measure of Zero-Moment Point (ZMP)-driven rewards and task-driven rewards in a whole-body actor-critic framework. Experiments conducted on a full-sized Unitree H1-2 robot verify the ability of our method to maintain balance on extremely narrow terrains.
  arXiv  Detail & Related papers  (2025-02-24T14:53:45Z)
• Learning Humanoid Locomotion with World Model Reconstruction [8.27902320260747]
  We introduce World Model Reconstruction (WMR), an end-to-end learning-based approach for blind humanoid locomotion. We train an estimator to explicitly reconstruct the world state and utilize it to enhance the locomotion policy. The robot successfully completed a 3.2 km hike without any human assistance.
  arXiv  Detail & Related papers  (2025-02-22T13:57:56Z)
• Learning Getting-Up Policies for Real-World Humanoid Robots [16.654440490255]
  This paper develops a learning framework to produce controllers that enable humanoid robots to get up from varying configurations on varying terrains. This is one of the first successful demonstrations of learned getting-up policies for human-sized humanoid robots in the real world.
  arXiv  Detail & Related papers  (2025-02-17T18:59:06Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer. By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• Learning Humanoid Locomotion over Challenging Terrain [84.35038297708485]
  We present a learning-based approach for blind humanoid locomotion capable of traversing challenging natural and man-made terrains. Our model is first pre-trained on a dataset of flat-ground trajectories with sequence modeling, and then fine-tuned on uneven terrain using reinforcement learning. We evaluate our model on a real humanoid robot across a variety of terrains, including rough, deformable, and sloped surfaces.
  arXiv  Detail & Related papers  (2024-10-04T17:57:09Z)
• Hierarchical World Models as Visual Whole-Body Humanoid Controllers [44.67878600054106]
  Whole-body control for humanoids is challenging due to the high-dimensional nature of the problem, and the inherent instability of a bipedal morphology. We propose a hierarchical world model in which a high-level agent generates commands based on visual observations for a low-level agent to execute, both of which are trained with rewards. Our approach produces highly performant control policies in 8 tasks with a simulated 56-DoF humanoid, while synthesizing motions that are broadly preferred by humans.
  arXiv  Detail & Related papers  (2024-05-28T17:57:23Z)
• Hierarchical generative modelling for autonomous robots [8.023920215148486]
  We show how a humanoid robot can autonomously complete a complex task that requires a holistic use of locomotion, manipulation, and grasping. Specifically, we demonstrate the ability of a humanoid robot that can retrieve and transport a box, open and walk through a door to reach the destination, approach and kick a football, while showing robust performance in presence of body damage and ground irregularities.
  arXiv  Detail & Related papers  (2023-08-15T13:51:03Z)
• Learning Human-to-Robot Handovers from Point Clouds [63.18127198174958]
  We propose the first framework to learn control policies for vision-based human-to-robot handovers. We show significant performance gains over baselines on a simulation benchmark, sim-to-sim transfer and sim-to-real transfer.
  arXiv  Detail & Related papers  (2023-03-30T17:58:36Z)
• Real-World Humanoid Locomotion with Reinforcement Learning [92.85934954371099]
  We present a fully learning-based approach for real-world humanoid locomotion. Our controller can walk over various outdoor terrains, is robust to external disturbances, and can adapt in context.
  arXiv  Detail & Related papers  (2023-03-06T18:59:09Z)
• Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance [71.36749876465618]
  We describe a system for vision-based dexterous manipulation that provides a "programming-free" approach for users to define new tasks. Our system includes a framework for users to define a final task and intermediate sub-tasks with image examples. experimental results with a four-finger robotic hand learning multi-stage object manipulation tasks directly in the real world.
  arXiv  Detail & Related papers  (2022-12-19T22:50:40Z)
• Learning Quadrupedal Locomotion over Challenging Terrain [68.51539602703662]
  Legged locomotion can dramatically expand the operational domains of robotics. Conventional controllers for legged locomotion are based on elaborate state machines that explicitly trigger the execution of motion primitives and reflexes. Here we present a radically robust controller for legged locomotion in challenging natural environments.
  arXiv  Detail & Related papers  (2020-10-21T19:11:20Z)
• Visual Navigation Among Humans with Optimal Control as a Supervisor [72.5188978268463]
  We propose an approach that combines learning-based perception with model-based optimal control to navigate among humans. Our approach is enabled by our novel data-generation tool, HumANav. We demonstrate that the learned navigation policies can anticipate and react to humans without explicitly predicting future human motion.
  arXiv  Detail & Related papers  (2020-03-20T16:13:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.