Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2103.06484v1
• Date: Thu, 11 Mar 2021 06:13:09 GMT
• Title: Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning
• Authors: Guillaume Bellegarda and Quan Nguyen
• Abstract summary: In this paper, we explore learning foot positions in Cartesian space for a task of running as fast as possible subject to environmental disturbances. Compared with other action spaces, we observe less needed reward shaping, much improved sample efficiency, and the emergence of natural gaits such as galloping and bounding. Policies can be learned in only a few million time steps, even for challenging tasks of running over rough terrain with loads of over 100% of the nominal quadruped mass.
• Score: 7.264355680723856
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep reinforcement learning has emerged as a popular and powerful way to develop locomotion controllers for quadruped robots. Common approaches have largely focused on learning actions directly in joint space, or learning to modify and offset foot positions produced by trajectory generators. Both approaches typically require careful reward shaping and training for millions of time steps, and with trajectory generators introduce human bias into the resulting control policies. In this paper, we instead explore learning foot positions in Cartesian space, which we track with impedance control, for a task of running as fast as possible subject to environmental disturbances. Compared with other action spaces, we observe less needed reward shaping, much improved sample efficiency, the emergence of natural gaits such as galloping and bounding, and ease of sim-to-sim transfer. Policies can be learned in only a few million time steps, even for challenging tasks of running over rough terrain with loads of over 100% of the nominal quadruped mass. Training occurs in PyBullet, and we perform a sim-to-sim transfer to Gazebo, where our quadruped is able to run at over 4 m/s without a load, and 3.5 m/s with a 10 kg load, which is over 83% of the nominal quadruped mass. Video results can be found at https://youtu.be/roE1vxpEWfw.

Related papers

• Impedance Matching: Enabling an RL-Based Running Jump in a Quadruped Robot [7.516046071926082]
  We propose a new framework to mitigate the gap between simulated and real robots. Our framework offers a structured guideline for parameter selection and the range for dynamics randomization in simulation. Results are, to the best of our knowledge, one of the highest and longest running jumps demonstrated by an RL-based control policy in a real quadruped robot.
  arXiv  Detail & Related papers  (2024-04-23T14:52:09Z)
• Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control [106.32794844077534]
  This paper presents a study on using deep reinforcement learning to create dynamic locomotion controllers for bipedal robots. We develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. This work pushes the limits of agility for bipedal robots through extensive real-world experiments.
  arXiv  Detail & Related papers  (2024-01-30T10:48:43Z)
• Barkour: Benchmarking Animal-level Agility with Quadruped Robots [70.97471756305463]
  We introduce the Barkour benchmark, an obstacle course to quantify agility for legged robots. Inspired by dog agility competitions, it consists of diverse obstacles and a time based scoring mechanism. We present two methods for tackling the benchmark.
  arXiv  Detail & Related papers  (2023-05-24T02:49:43Z)
• Legged Locomotion in Challenging Terrains using Egocentric Vision [70.37554680771322]
  We present the first end-to-end locomotion system capable of traversing stairs, curbs, stepping stones, and gaps. We show this result on a medium-sized quadruped robot using a single front-facing depth camera.
  arXiv  Detail & Related papers  (2022-11-14T18:59:58Z)
• Learning a Single Near-hover Position Controller for Vastly Different Quadcopters [56.37274861303324]
  This paper proposes an adaptive near-hover position controller for quadcopters. It can be deployed to quadcopters of very different mass, size and motor constants. It also shows rapid adaptation to unknown disturbances during runtime.
  arXiv  Detail & Related papers  (2022-09-19T17:55:05Z)
• A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning [86.06110576808824]
  Deep reinforcement learning is a promising approach to learning policies in uncontrolled environments. Recent advancements in machine learning algorithms and libraries combined with a carefully tuned robot controller lead to learning quadruped in only 20 minutes in the real world.
  arXiv  Detail & Related papers  (2022-08-16T17:37:36Z)
• Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning [2.930703970709558]
  We present and study a training set-up that achieves fast policy generation for real-world robotic tasks by using massive parallelism on a single workstation GPU. We analyze and discuss the impact of different training algorithm components in the massively parallel regime on the final policy performance and training times. We present a novel game-inspired curriculum that is well suited for training with thousands of simulated robots in parallel.
  arXiv  Detail & Related papers  (2021-09-24T14:04:19Z)
• Learning Quadruped Locomotion Policies using Logical Rules [2.008081703108095]
  We aim to enable easy gait specification and efficient policy learning for quadruped robots. Our approach is called RM-based Locomotion Learning(RMLL), and supports adjusting gait frequency at execution time. We demonstrate these learned policies with a real quadruped robot.
  arXiv  Detail & Related papers  (2021-07-23T00:37:32Z)
• Quadruped Locomotion on Non-Rigid Terrain using Reinforcement Learning [10.729374293332281]
  We present a novel reinforcement learning framework for learning locomotion on non-rigid dynamic terrains. A trained robot with 55cm base length can walk on terrain that can sink up to 5cm. We show the effectiveness of our method by training the robot with various terrain conditions.
  arXiv  Detail & Related papers  (2021-07-07T00:34:23Z)
• Fast and Efficient Locomotion via Learned Gait Transitions [35.86279693549959]
  We focus on the problem of developing efficient controllers for quadrupedal robots. We devise a hierarchical learning framework, in which distinctive locomotion gaits and natural gait transitions emerge automatically. We show that the learned hierarchical controller consumes much less energy across a wide range of locomotion speed than baseline controllers.
  arXiv  Detail & Related papers  (2021-04-09T23:53:28Z)
• Learning Quadrupedal Locomotion over Challenging Terrain [68.51539602703662]
  Legged locomotion can dramatically expand the operational domains of robotics. Conventional controllers for legged locomotion are based on elaborate state machines that explicitly trigger the execution of motion primitives and reflexes. Here we present a radically robust controller for legged locomotion in challenging natural environments.
  arXiv  Detail & Related papers  (2020-10-21T19:11:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.