SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience

• URL: http://arxiv.org/abs/2409.13678v1
• Date: Fri, 20 Sep 2024 17:39:20 GMT
• Title: SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience
• Authors: Elliot Chane-Sane, Joseph Amigo, Thomas Flayols, Ludovic Righetti, Nicolas Mansard,
• Abstract summary: Parkour poses a significant challenge for legged robots, requiring navigation through complex environments with agility and precision based on limited sensory inputs. We introduce a novel method for training end-to-end visual policies, from depth pixels to robot control commands, to achieve agile and safe quadruped locomotion. We demonstrate the effectiveness of our method on a real Solo-12 robot, showcasing its capability to perform a variety of parkour skills such as walking, climbing, leaping, and crawling.
• Score: 19.817578964184147
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Parkour poses a significant challenge for legged robots, requiring navigation through complex environments with agility and precision based on limited sensory inputs. In this work, we introduce a novel method for training end-to-end visual policies, from depth pixels to robot control commands, to achieve agile and safe quadruped locomotion. We formulate robot parkour as a constrained reinforcement learning (RL) problem designed to maximize the emergence of agile skills within the robot's physical limits while ensuring safety. We first train a policy without vision using privileged information about the robot's surroundings. We then generate experience from this privileged policy to warm-start a sample efficient off-policy RL algorithm from depth images. This allows the robot to adapt behaviors from this privileged experience to visual locomotion while circumventing the high computational costs of RL directly from pixels. We demonstrate the effectiveness of our method on a real Solo-12 robot, showcasing its capability to perform a variety of parkour skills such as walking, climbing, leaping, and crawling.

Related papers

• Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control [106.32794844077534]
  This paper presents a study on using deep reinforcement learning to create dynamic locomotion controllers for bipedal robots. We develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. This work pushes the limits of agility for bipedal robots through extensive real-world experiments.
  arXiv  Detail & Related papers  (2024-01-30T10:48:43Z)
• Extreme Parkour with Legged Robots [43.041181063455255]
  We show how a single neural net policy operating directly from a camera image can overcome imprecise sensing and actuation. We show our robot can perform a high jump on obstacles 2x its height, long jump across gaps 2x its length, do a handstand and run across tilted ramps.
  arXiv  Detail & Related papers  (2023-09-25T17:59:55Z)
• Robot Parkour Learning [70.56172796132368]
  Parkour is a grand challenge for legged locomotion that requires robots to overcome various obstacles rapidly. We develop a reinforcement learning method inspired by direct collocation to generate parkour skills. We distill these skills into a single vision-based parkour policy and transfer it to a quadrupedal robot using its egocentric depth camera.
  arXiv  Detail & Related papers  (2023-09-11T17:59:17Z)
• Barkour: Benchmarking Animal-level Agility with Quadruped Robots [70.97471756305463]
  We introduce the Barkour benchmark, an obstacle course to quantify agility for legged robots. Inspired by dog agility competitions, it consists of diverse obstacles and a time based scoring mechanism. We present two methods for tackling the benchmark.
  arXiv  Detail & Related papers  (2023-05-24T02:49:43Z)
• Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion [34.33972863987201]
  We train quadruped robots to use the front legs to climb walls, press buttons, and perform object interaction in the real world. These skills are trained in simulation using curriculum and transferred to the real world using our proposed sim2real variant. We evaluate our method in both simulation and real-world showing successful executions of both short as well as long-range tasks.
  arXiv  Detail & Related papers  (2023-03-20T17:59:58Z)
• Learning Agile Locomotion via Adversarial Training [59.03007947334165]
  In this paper, we present a multi-agent learning system, in which a quadruped robot (protagonist) learns to chase another robot (adversary) while the latter learns to escape. We find that this adversarial training process not only encourages agile behaviors but also effectively alleviates the laborious environment design effort. In contrast to prior works that used only one adversary, we find that training an ensemble of adversaries, each of which specializes in a different escaping strategy, is essential for the protagonist to master agility.
  arXiv  Detail & Related papers  (2020-08-03T01:20:37Z)
• Learning Agile Robotic Locomotion Skills by Imitating Animals [72.36395376558984]
  Reproducing the diverse and agile locomotion skills of animals has been a longstanding challenge in robotics. We present an imitation learning system that enables legged robots to learn agile locomotion skills by imitating real-world animals.
  arXiv  Detail & Related papers  (2020-04-02T02:56:16Z)
• Learning to Walk in the Real World with Minimal Human Effort [80.7342153519654]
  We develop a system for learning legged locomotion policies with deep RL in the real world with minimal human effort. Our system can automatically and efficiently learn locomotion skills on a Minitaur robot with little human intervention.
  arXiv  Detail & Related papers  (2020-02-20T03:36:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.