Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic
Platforms
- URL: http://arxiv.org/abs/2103.03697v1
- Date: Fri, 5 Mar 2021 14:16:20 GMT
- Title: Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic
Platforms
- Authors: Ali Ghadirzadeh, Xi Chen, Petra Poklukar, Chelsea Finn, M{\aa}rten
Bj\"orkman and Danica Kragic
- Abstract summary: Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform.
We formulate it as a few-shot meta-learning problem where the goal is to find a model that captures the common structure shared across different robotic platforms.
We experimentally evaluate our framework on a simulated reaching and a real-robot picking task using 400 simulated robots.
- Score: 60.59764170868101
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Reinforcement learning methods can achieve significant performance but
require a large amount of training data collected on the same robotic platform.
A policy trained with expensive data is rendered useless after making even a
minor change to the robot hardware. In this paper, we address the challenging
problem of adapting a policy, trained to perform a task, to a novel robotic
hardware platform given only few demonstrations of robot motion trajectories on
the target robot. We formulate it as a few-shot meta-learning problem where the
goal is to find a meta-model that captures the common structure shared across
different robotic platforms such that data-efficient adaptation can be
performed. We achieve such adaptation by introducing a learning framework
consisting of a probabilistic gradient-based meta-learning algorithm that
models the uncertainty arising from the few-shot setting with a low-dimensional
latent variable. We experimentally evaluate our framework on a simulated
reaching and a real-robot picking task using 400 simulated robots generated by
varying the physical parameters of an existing set of robotic platforms. Our
results show that the proposed method can successfully adapt a trained policy
to different robotic platforms with novel physical parameters and the
superiority of our meta-learning algorithm compared to state-of-the-art methods
for the introduced few-shot policy adaptation problem.
Related papers
- EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learning [36.0274770291531]
We propose EquiBot, a robust, data-efficient, and generalizable approach for robot manipulation task learning.
Our approach combines SIM(3)-equivariant neural network architectures with diffusion models.
In the real world, we show with in total 10 variations of 6 mobile manipulation tasks that our method can easily generalize to novel objects and scenes.
arXiv Detail & Related papers (2024-07-01T17:09:43Z) - RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation [68.70755196744533]
RoboGen is a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation.
Our work attempts to extract the extensive and versatile knowledge embedded in large-scale models and transfer them to the field of robotics.
arXiv Detail & Related papers (2023-11-02T17:59:21Z) - Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for
Autonomous Real-World Reinforcement Learning [58.3994826169858]
We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning.
Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy.
Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
arXiv Detail & Related papers (2023-10-23T17:50:08Z) - Polybot: Training One Policy Across Robots While Embracing Variability [70.74462430582163]
We propose a set of key design decisions to train a single policy for deployment on multiple robotic platforms.
Our framework first aligns the observation and action spaces of our policy across embodiments via utilizing wrist cameras.
We evaluate our method on a dataset collected over 60 hours spanning 6 tasks and 3 robots with varying joint configurations and sizes.
arXiv Detail & Related papers (2023-07-07T17:21:16Z) - REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy
Transfer [57.045140028275036]
We consider the problem of transferring a policy across two different robots with significantly different parameters such as kinematics and morphology.
Existing approaches that train a new policy by matching the action or state transition distribution, including imitation learning methods, fail due to optimal action and/or state distribution being mismatched in different robots.
We propose a novel method named $REvolveR$ of using continuous evolutionary models for robotic policy transfer implemented in a physics simulator.
arXiv Detail & Related papers (2022-02-10T18:50:25Z) - Learning compositional models of robot skills for task and motion
planning [39.36562555272779]
We learn to use sensorimotor primitives to solve complex long-horizon manipulation problems.
We use state-of-the-art methods for active learning and sampling.
We evaluate our approach both in simulation and in the real world through measuring the quality of the selected primitive actions.
arXiv Detail & Related papers (2020-06-08T20:45:34Z) - Fast Online Adaptation in Robotics through Meta-Learning Embeddings of
Simulated Priors [3.4376560669160385]
In the real world, a robot might encounter any situation starting from motor failures to finding itself in a rocky terrain.
We show that FAMLE allows the robots to adapt to novel damages in significantly fewer time-steps than the baselines.
arXiv Detail & Related papers (2020-03-10T12:37:52Z) - Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning [65.88200578485316]
We present a new meta-learning method that allows robots to quickly adapt to changes in dynamics.
Our method significantly improves adaptation to changes in dynamics in high noise settings.
We validate our approach on a quadruped robot that learns to walk while subject to changes in dynamics.
arXiv Detail & Related papers (2020-03-02T22:56:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.