Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms

• URL: http://arxiv.org/abs/2103.03697v1
• Date: Fri, 5 Mar 2021 14:16:20 GMT
• Title: Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms
• Authors: Ali Ghadirzadeh, Xi Chen, Petra Poklukar, Chelsea Finn, M{\aa}rten Bj\"orkman and Danica Kragic
• Abstract summary: Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform. We formulate it as a few-shot meta-learning problem where the goal is to find a model that captures the common structure shared across different robotic platforms. We experimentally evaluate our framework on a simulated reaching and a real-robot picking task using 400 simulated robots.
• Score: 60.59764170868101
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform. A policy trained with expensive data is rendered useless after making even a minor change to the robot hardware. In this paper, we address the challenging problem of adapting a policy, trained to perform a task, to a novel robotic hardware platform given only few demonstrations of robot motion trajectories on the target robot. We formulate it as a few-shot meta-learning problem where the goal is to find a meta-model that captures the common structure shared across different robotic platforms such that data-efficient adaptation can be performed. We achieve such adaptation by introducing a learning framework consisting of a probabilistic gradient-based meta-learning algorithm that models the uncertainty arising from the few-shot setting with a low-dimensional latent variable. We experimentally evaluate our framework on a simulated reaching and a real-robot picking task using 400 simulated robots generated by varying the physical parameters of an existing set of robotic platforms. Our results show that the proposed method can successfully adapt a trained policy to different robotic platforms with novel physical parameters and the superiority of our meta-learning algorithm compared to state-of-the-art methods for the introduced few-shot policy adaptation problem.

Related papers

• Solving Multi-Goal Robotic Tasks with Decision Transformer [0.0]
  We introduce a novel adaptation of the decision transformer architecture for offline multi-goal reinforcement learning in robotics. Our approach integrates goal-specific information into the decision transformer, allowing it to handle complex tasks in an offline setting.
  arXiv  Detail & Related papers  (2024-10-08T20:35:30Z)
• Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments [26.66666135624716]
  We present Robot Utility Models (RUMs), a framework for training and deploying zero-shot robot policies. RUMs can generalize to new environments without any finetuning. We train five utility models for opening cabinet doors, opening drawers, picking up napkins, picking up paper bags, and reorienting fallen objects.
  arXiv  Detail & Related papers  (2024-09-09T17:59:50Z)
• EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learning [36.0274770291531]
  We propose Equibot, a robust, data-efficient, and generalizable approach for robot manipulation task learning. Our approach combines SIM(3)-equivariant neural network architectures with diffusion models. We show that our method can easily generalize to novel objects and scenes after learning from just 5 minutes of human demonstrations in each task.
  arXiv  Detail & Related papers  (2024-07-01T17:09:43Z)
• RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation [68.70755196744533]
  RoboGen is a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation. Our work attempts to extract the extensive and versatile knowledge embedded in large-scale models and transfer them to the field of robotics.
  arXiv  Detail & Related papers  (2023-11-02T17:59:21Z)
• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning [58.3994826169858]
  We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning. Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy. Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
  arXiv  Detail & Related papers  (2023-10-23T17:50:08Z)
• Polybot: Training One Policy Across Robots While Embracing Variability [70.74462430582163]
  We propose a set of key design decisions to train a single policy for deployment on multiple robotic platforms. Our framework first aligns the observation and action spaces of our policy across embodiments via utilizing wrist cameras. We evaluate our method on a dataset collected over 60 hours spanning 6 tasks and 3 robots with varying joint configurations and sizes.
  arXiv  Detail & Related papers  (2023-07-07T17:21:16Z)
• REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfer [57.045140028275036]
  We consider the problem of transferring a policy across two different robots with significantly different parameters such as kinematics and morphology. Existing approaches that train a new policy by matching the action or state transition distribution, including imitation learning methods, fail due to optimal action and/or state distribution being mismatched in different robots. We propose a novel method named $REvolveR$ of using continuous evolutionary models for robotic policy transfer implemented in a physics simulator.
  arXiv  Detail & Related papers  (2022-02-10T18:50:25Z)
• Fast Online Adaptation in Robotics through Meta-Learning Embeddings of Simulated Priors [3.4376560669160385]
  In the real world, a robot might encounter any situation starting from motor failures to finding itself in a rocky terrain. We show that FAMLE allows the robots to adapt to novel damages in significantly fewer time-steps than the baselines.
  arXiv  Detail & Related papers  (2020-03-10T12:37:52Z)
• Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning [65.88200578485316]
  We present a new meta-learning method that allows robots to quickly adapt to changes in dynamics. Our method significantly improves adaptation to changes in dynamics in high noise settings. We validate our approach on a quadruped robot that learns to walk while subject to changes in dynamics.
  arXiv  Detail & Related papers  (2020-03-02T22:56:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.