Towards More Generalizable One-shot Visual Imitation Learning
- URL: http://arxiv.org/abs/2110.13423v1
- Date: Tue, 26 Oct 2021 05:49:46 GMT
- Title: Towards More Generalizable One-shot Visual Imitation Learning
- Authors: Zhao Mandi, Fangchen Liu, Kimin Lee, Pieter Abbeel
- Abstract summary: A general-purpose robot should be able to master a wide range of tasks and quickly learn a novel one by leveraging past experiences.
One-shot imitation learning (OSIL) approaches this goal by training an agent with (pairs of) expert demonstrations.
We push for a higher level of generalization ability by investigating a more ambitious multi-task setup.
- Score: 81.09074706236858
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: A general-purpose robot should be able to master a wide range of tasks and
quickly learn a novel one by leveraging past experiences. One-shot imitation
learning (OSIL) approaches this goal by training an agent with (pairs of)
expert demonstrations, such that at test time, it can directly execute a new
task from just one demonstration. However, so far this framework has been
limited to training on many variations of one task, and testing on other unseen
but similar variations of the same task. In this work, we push for a higher
level of generalization ability by investigating a more ambitious multi-task
setup. We introduce a diverse suite of vision-based robot manipulation tasks,
consisting of 7 tasks, a total of 61 variations, and a continuum of instances
within each variation. For consistency and comparison purposes, we first train
and evaluate single-task agents (as done in prior few-shot imitation work). We
then study the multi-task setting, where multi-task training is followed by (i)
one-shot imitation on variations within the training tasks, (ii) one-shot
imitation on new tasks, and (iii) fine-tuning on new tasks. Prior
state-of-the-art, while performing well within some single tasks, struggles in
these harder multi-task settings. To address these limitations, we propose
MOSAIC (Multi-task One-Shot Imitation with self-Attention and Contrastive
learning), which integrates a self-attention model architecture and a temporal
contrastive module to enable better task disambiguation and more robust
representation learning. Our experiments show that MOSAIC outperforms prior
state of the art in learning efficiency, final performance, and learns a
multi-task policy with promising generalization ability via fine-tuning on
novel tasks.
Related papers
- LIMT: Language-Informed Multi-Task Visual World Models [6.128332310539627]
Multi-task reinforcement learning can be very challenging due to the increased sample complexity and the potentially conflicting task objectives.
We propose a method for learning multi-task visual world models, leveraging pre-trained language models to extract semantically meaningful task representations.
Our results highlight the benefits of using language-driven task representations for world models and a clear advantage of model-based multi-task learning over the more common model-free paradigm.
arXiv Detail & Related papers (2024-07-18T12:40:58Z) - Continual Robot Learning using Self-Supervised Task Inference [19.635428830237842]
We propose a self-supervised task inference approach to continually learn new tasks.
We use a behavior-matching self-supervised learning objective to train a novel Task Inference Network (TINet)
A multi-task policy is built on top of the TINet and trained with reinforcement learning to optimize performance over tasks.
arXiv Detail & Related papers (2023-09-10T09:32:35Z) - Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts [75.75548749888029]
We present a vision-language model whose parameters are jointly trained on all tasks and fully shared among multiple heterogeneous tasks.
With a single model, Musketeer achieves results comparable to or better than strong baselines trained on single tasks, almost uniformly across multiple tasks.
arXiv Detail & Related papers (2023-05-11T17:57:49Z) - OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist
Models [72.8156832931841]
Generalist models are capable of performing diverse multi-modal tasks in a task-agnostic way within a single model.
We release a generalist model learning system, OFASys, built on top of a declarative task interface named multi-modal instruction.
arXiv Detail & Related papers (2022-12-08T17:07:09Z) - Is Multi-Task Learning an Upper Bound for Continual Learning? [26.729088618251282]
This paper proposes a novel continual self-supervised learning setting, where each task corresponds to learning an invariant representation for a specific class of data augmentations.
We show that continual learning often beats multi-task learning on various benchmark datasets, including MNIST, CIFAR-10, and CIFAR-100.
arXiv Detail & Related papers (2022-10-26T15:45:11Z) - Generalizing to New Tasks via One-Shot Compositional Subgoals [23.15624959305799]
The ability to generalize to previously unseen tasks with little to no supervision is a key challenge in modern machine learning research.
We introduce CASE which attempts to address these issues by training an Imitation Learning agent using adaptive "near future" subgoals.
Our experiments show that the proposed approach consistently outperforms the previous state-of-the-art compositional Imitation Learning approach by 30%.
arXiv Detail & Related papers (2022-05-16T14:30:11Z) - Modular Adaptive Policy Selection for Multi-Task Imitation Learning
through Task Division [60.232542918414985]
Multi-task learning often suffers from negative transfer, sharing information that should be task-specific.
This is done by using proto-policies as modules to divide the tasks into simple sub-behaviours that can be shared.
We also demonstrate its ability to autonomously divide the tasks into both shared and task-specific sub-behaviours.
arXiv Detail & Related papers (2022-03-28T15:53:17Z) - Variational Multi-Task Learning with Gumbel-Softmax Priors [105.22406384964144]
Multi-task learning aims to explore task relatedness to improve individual tasks.
We propose variational multi-task learning (VMTL), a general probabilistic inference framework for learning multiple related tasks.
arXiv Detail & Related papers (2021-11-09T18:49:45Z) - MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale [103.7609761511652]
We show how a large-scale collective robotic learning system can acquire a repertoire of behaviors simultaneously.
New tasks can be continuously instantiated from previously learned tasks.
We train and evaluate our system on a set of 12 real-world tasks with data collected from 7 robots.
arXiv Detail & Related papers (2021-04-16T16:38:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.