Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity

• URL: http://arxiv.org/abs/2210.09918v1
• Date: Tue, 18 Oct 2022 15:02:41 GMT
• Title: Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity
• Authors: Maxime Allard, Sim\'on C. Smith, Konstantinos Chatzilygeroudis, Bryan Lim, Antoine Cully
• Abstract summary: We introduce the Hierarchical Trial and Error algorithm, which uses a hierarchical behavioural repertoire to learn diverse skills. We show that the hierarchical decomposition of skills enables the robot to learn more complex behaviours while keeping the learning of the repertoire tractable.
• Score: 3.899855581265355
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In real-world environments, robots need to be resilient to damages and robust to unforeseen scenarios. Quality-Diversity (QD) algorithms have been successfully used to make robots adapt to damages in seconds by leveraging a diverse set of learned skills. A high diversity of skills increases the chances of a robot to succeed at overcoming new situations since there are more potential alternatives to solve a new task.However, finding and storing a large behavioural diversity of multiple skills often leads to an increase in computational complexity. Furthermore, robot planning in a large skill space is an additional challenge that arises with an increased number of skills. Hierarchical structures can help reducing this search and storage complexity by breaking down skills into primitive skills. In this paper, we introduce the Hierarchical Trial and Error algorithm, which uses a hierarchical behavioural repertoire to learn diverse skills and leverages them to make the robot adapt quickly in the physical world. We show that the hierarchical decomposition of skills enables the robot to learn more complex behaviours while keeping the learning of the repertoire tractable. Experiments with a hexapod robot show that our method solves a maze navigation tasks with 20% less actions in simulation, and 43% less actions in the physical world, for the most challenging scenarios than the best baselines while having 78% less complete failures.

Related papers

• A roadmap for AI in robotics [55.87087746398059]
  We are witnessing growing excitement in robotics at the prospect of leveraging the potential of AI to tackle some of the outstanding barriers to the full deployment of robots in our daily lives.<n>This article offers an assessment of what AI for robotics has achieved since the 1990s and proposes a short- and medium-term research roadmap listing challenges and promises.
  arXiv  Detail & Related papers  (2025-07-26T15:18:28Z)
• SIME: Enhancing Policy Self-Improvement with Modal-level Exploration [49.86173021151849]
  We identify the key to successful self-improvement: modal-level exploration and data selection.<n>By incorporating a modal-level exploration mechanism during policy execution, the robot can produce more diverse and multi-modal interactions.<n>We successfully demonstrate effective robot self-improvement on both simulation benchmarks and real-world experiments.
  arXiv  Detail & Related papers  (2025-05-02T17:13:03Z)
• Towards Conscious Service Robots [21.66931637743555]
  Real-world robotics face challenges like variability, high-dimensional state spaces, non-linear dependencies, and partial observability. Unlike current machine learning models, humans adapt quickly to changes and new tasks due to a cognitive architecture that enables systematic generalization and meta-cognition. Next generation of service robots will handle novel situations and monitor themselves to avoid risks and mitigate errors.
  arXiv  Detail & Related papers  (2025-01-25T12:32:52Z)
• A Retrospective on the Robot Air Hockey Challenge: Benchmarking Robust, Reliable, and Safe Learning Techniques for Real-world Robotics [53.33976793493801]
  We organized the Robot Air Hockey Challenge at the NeurIPS 2023 conference. We focus on practical challenges in robotics, such as the sim-to-real gap, low-level control issues, safety problems, real-time requirements, and the limited availability of real-world data. Results show that solutions combining learning-based approaches with prior knowledge outperform those relying solely on data when real-world deployment is challenging.
  arXiv  Detail & Related papers  (2024-11-08T17:20:47Z)
• $π_0$: A Vision-Language-Action Flow Model for General Robot Control [77.32743739202543]
  We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people, and its ability to acquire new skills via fine-tuning.
  arXiv  Detail & Related papers  (2024-10-31T17:22:30Z)
• HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation [50.616995671367704]
  We present a high-dimensional, simulated robot learning benchmark, HumanoidBench, featuring a humanoid robot equipped with dexterous hands. Our findings reveal that state-of-the-art reinforcement learning algorithms struggle with most tasks, whereas a hierarchical learning approach achieves superior performance when supported by robust low-level policies.
  arXiv  Detail & Related papers  (2024-03-15T17:45:44Z)
• RSG: Fast Learning Adaptive Skills for Quadruped Robots by Skill Graph [26.861541495975686]
  We propose a novel framework, named Robot Skill Graph (RSG), for organizing massive fundamental skills of robots. RSG is composed of massive dynamic behavioral skills instead of static knowledge in Knowledge Graph (KG) Extensive experimental results demonstrate that RSG can provide rational skill inference upon new tasks and environments.
  arXiv  Detail & Related papers  (2023-11-10T11:59:41Z)
• RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot [56.130215236125224]
  A key challenge in robotic manipulation in open domains is how to acquire diverse and generalizable skills for robots. Recent research in one-shot imitation learning has shown promise in transferring trained policies to new tasks based on demonstrations. This paper aims to unlock the potential for an agent to generalize to hundreds of real-world skills with multi-modal perception.
  arXiv  Detail & Related papers  (2023-07-02T15:33:31Z)
• Exploring the effects of robotic design on learning and neural control [0.0]
  dissertation focuses on the development of robotic bodies, rather than neural controllers. I have discovered that robots can be designed such that they overcome many of the current pitfalls encountered by neural controllers in multitask settings.
  arXiv  Detail & Related papers  (2023-06-06T15:17:34Z)
• Discovering Unsupervised Behaviours from Full-State Trajectories [1.827510863075184]
  We propose an analysis of Autonomous Robots Realising their Abilities; a Quality-Diversity algorithm that autonomously finds behavioural characterisations. We evaluate this approach on a simulated robotic environment, where the robot has to autonomously discover its abilities from its full-state trajectories. More specifically, the analysed approach autonomously finds policies that make the robot move to diverse positions, but also utilise its legs in diverse ways, and even perform half-rolls.
  arXiv  Detail & Related papers  (2022-11-22T16:57:52Z)
• Hierarchical Quality-Diversity for Online Damage Recovery [1.376408511310322]
  We introduce the Hierarchical Trial and Error algorithm, which uses a hierarchical behavioural repertoire to learn diverse skills. We show that the hierarchical decomposition of skills enables the robot to learn more complex behaviours while keeping the learning of the repertoire tractable.
  arXiv  Detail & Related papers  (2022-04-12T11:44:01Z)
• Lifelong Robotic Reinforcement Learning by Retaining Experiences [61.79346922421323]
  Many multi-task reinforcement learning efforts assume the robot can collect data from all tasks at all times. In this work, we study a practical sequential multi-task RL problem motivated by the practical constraints of physical robotic systems. We derive an approach that effectively leverages the data and policies learned for previous tasks to cumulatively grow the robot's skill-set.
  arXiv  Detail & Related papers  (2021-09-19T18:00:51Z)
• Discovering Generalizable Skills via Automated Generation of Diverse Tasks [82.16392072211337]
  We propose a method to discover generalizable skills via automated generation of a diverse set of tasks. As opposed to prior work on unsupervised discovery of skills, our method pairs each skill with a unique task produced by a trainable task generator. A task discriminator defined on the robot behaviors in the generated tasks is jointly trained to estimate the evidence lower bound of the diversity objective. The learned skills can then be composed in a hierarchical reinforcement learning algorithm to solve unseen target tasks.
  arXiv  Detail & Related papers  (2021-06-26T03:41:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.