Humanoid Whole-Body Locomotion on Narrow Terrain via Dynamic Balance and Reinforcement Learning

• URL: http://arxiv.org/abs/2502.17219v2
• Date: Fri, 28 Feb 2025 07:04:13 GMT
• Title: Humanoid Whole-Body Locomotion on Narrow Terrain via Dynamic Balance and Reinforcement Learning
• Authors: Weiji Xie, Chenjia Bai, Jiyuan Shi, Junkai Yang, Yunfei Ge, Weinan Zhang, Xuelong Li,
• Abstract summary: We propose a novel whole-body locomotion algorithm based on dynamic balance and Reinforcement Learning (RL)<n> Specifically, we introduce a dynamic balance mechanism by leveraging an extended measure of Zero-Moment Point (ZMP)-driven rewards and task-driven rewards in a whole-body actor-critic framework.<n> Experiments conducted on a full-sized Unitree H1-2 robot verify the ability of our method to maintain balance on extremely narrow terrains.
• Score: 54.26816599309778
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humans possess delicate dynamic balance mechanisms that enable them to maintain stability across diverse terrains and under extreme conditions. However, despite significant advances recently, existing locomotion algorithms for humanoid robots are still struggle to traverse extreme environments, especially in cases that lack external perception (e.g., vision or LiDAR). This is because current methods often rely on gait-based or perception-condition rewards, lacking effective mechanisms to handle unobservable obstacles and sudden balance loss. To address this challenge, we propose a novel whole-body locomotion algorithm based on dynamic balance and Reinforcement Learning (RL) that enables humanoid robots to traverse extreme terrains, particularly narrow pathways and unexpected obstacles, using only proprioception. Specifically, we introduce a dynamic balance mechanism by leveraging an extended measure of Zero-Moment Point (ZMP)-driven rewards and task-driven rewards in a whole-body actor-critic framework, aiming to achieve coordinated actions of the upper and lower limbs for robust locomotion. Experiments conducted on a full-sized Unitree H1-2 robot verify the ability of our method to maintain balance on extremely narrow terrains and under external disturbances, demonstrating its effectiveness in enhancing the robot's adaptability to complex environments. The videos are given at https://whole-body-loco.github.io.

Related papers

• Coordinating Spinal and Limb Dynamics for Enhanced Sprawling Robot Mobility [0.047116288835793156]
  A flexible spine enables undulation of the body through a wavelike motion along the spine, aiding navigation over uneven terrains and obstacles. Environmental uncertainties, such as surface irregularities and variations in friction, can significantly disrupt body-limb coordination. Deep reinforcement learning offers a promising framework for handling non-deterministic environments. We comparatively examine learning-based control strategies and biologically inspired gait design methods on a salamander-like robot.
  arXiv  Detail & Related papers  (2025-04-18T23:08:48Z)
• StyleLoco: Generative Adversarial Distillation for Natural Humanoid Robot Locomotion [31.30409161905949]
  StyleLoco is a novel framework for learning humanoid locomotion. It combines the agility of reinforcement learning with the natural fluidity of human-like movements. We demonstrate that StyleLoco enables humanoid robots to perform diverse locomotion tasks.
  arXiv  Detail & Related papers  (2025-03-19T10:27:44Z)
• PALo: Learning Posture-Aware Locomotion for Quadruped Robots [29.582249837902427]
  We propose an end-to-end deep reinforcement learning framework for posture-aware locomotion named PALo. PALo handles simultaneous linear and angular velocity tracking and real-time adjustments of body height, pitch, and roll angles. PALo achieves agile posture-aware locomotion control in simulated environments and successfully transfers to real-world settings without fine-tuning.
  arXiv  Detail & Related papers  (2025-03-06T14:13:59Z)
• Learning Humanoid Standing-up Control across Diverse Postures [27.79222176982376]
  We present HoST (Humanoid Standing-up Control), a reinforcement learning framework that learns standing-up control from scratch.<n>HoST effectively learns posture-adaptive motions by leveraging a multi-critic architecture and curriculum-based training on diverse simulated terrains.<n>Our experimental results demonstrate that the controllers achieve smooth, stable, and robust standing-up motions across a wide range of laboratory and outdoor environments.
  arXiv  Detail & Related papers  (2025-02-12T13:10:09Z)
• Learning Bipedal Walking for Humanoid Robots in Challenging Environments with Obstacle Avoidance [0.3481985817302898]
  Deep reinforcement learning has seen successful implementations on humanoid robots to achieve dynamic walking. In this paper, we aim to achieve bipedal locomotion in an environment where obstacles are present using a policy-based reinforcement learning.
  arXiv  Detail & Related papers  (2024-09-25T07:02:04Z)
• Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control [106.32794844077534]
  This paper presents a study on using deep reinforcement learning to create dynamic locomotion controllers for bipedal robots. We develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. This work pushes the limits of agility for bipedal robots through extensive real-world experiments.
  arXiv  Detail & Related papers  (2024-01-30T10:48:43Z)
• Natural and Robust Walking using Reinforcement Learning without Demonstrations in High-Dimensional Musculoskeletal Models [29.592874007260342]
  Humans excel at robust bipedal walking in complex natural environments. It is still not fully understood how the nervous system resolves the musculoskeletal redundancy to solve the multi-objective control problem.
  arXiv  Detail & Related papers  (2023-09-06T13:20:31Z)
• VAE-Loco: Versatile Quadruped Locomotion by Learning a Disentangled Gait Representation [78.92147339883137]
  We show that it is pivotal in increasing controller robustness by learning a latent space capturing the key stance phases constituting a particular gait. We demonstrate that specific properties of the drive signal map directly to gait parameters such as cadence, footstep height and full stance duration. The use of a generative model facilitates the detection and mitigation of disturbances to provide a versatile and robust planning framework.
  arXiv  Detail & Related papers  (2022-05-02T19:49:53Z)
• Next Steps: Learning a Disentangled Gait Representation for Versatile Quadruped Locomotion [69.87112582900363]
  Current planners are unable to vary key gait parameters continuously while the robot is in motion. In this work we address this limitation by learning a latent space capturing the key stance phases constituting a particular gait. We demonstrate that specific properties of the drive signal map directly to gait parameters such as cadence, foot step height and full stance duration.
  arXiv  Detail & Related papers  (2021-12-09T10:02:02Z)
• Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots [121.42930679076574]
  We present a model-free reinforcement learning framework for training robust locomotion policies in simulation. domain randomization is used to encourage the policies to learn behaviors that are robust across variations in system dynamics. We demonstrate this on versatile walking behaviors such as tracking a target walking velocity, walking height, and turning yaw.
  arXiv  Detail & Related papers  (2021-03-26T07:14:01Z)
• Learning Quadrupedal Locomotion over Challenging Terrain [68.51539602703662]
  Legged locomotion can dramatically expand the operational domains of robotics. Conventional controllers for legged locomotion are based on elaborate state machines that explicitly trigger the execution of motion primitives and reflexes. Here we present a radically robust controller for legged locomotion in challenging natural environments.
  arXiv  Detail & Related papers  (2020-10-21T19:11:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.