Related papers: Behavior Self-Organization Supports Task Inference for Continual Robot Learning

Behavior Self-Organization Supports Task Inference for Continual Robot Learning

URL: http://arxiv.org/abs/2107.04533v1
Date: Fri, 9 Jul 2021 16:37:27 GMT
Title: Behavior Self-Organization Supports Task Inference for Continual Robot Learning
Authors: Muhammad Burhan Hafez, Stefan Wermter
Abstract summary: We propose a novel approach to continual learning of robotic control tasks. Our approach performs unsupervised learning of behavior embeddings by incrementally self-organizing demonstrated behaviors. Unlike previous approaches, our approach makes no assumptions about task distribution and requires no task exploration to infer tasks.
Score: 18.071689266826212
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Recent advances in robot learning have enabled robots to become increasingly better at mastering a predefined set of tasks. On the other hand, as humans, we have the ability to learn a growing set of tasks over our lifetime. Continual robot learning is an emerging research direction with the goal of endowing robots with this ability. In order to learn new tasks over time, the robot first needs to infer the task at hand. Task inference, however, has received little attention in the multi-task learning literature. In this paper, we propose a novel approach to continual learning of robotic control tasks. Our approach performs unsupervised learning of behavior embeddings by incrementally self-organizing demonstrated behaviors. Task inference is made by finding the nearest behavior embedding to a demonstrated behavior, which is used together with the environment state as input to a multi-task policy trained with reinforcement learning to optimize performance over tasks. Unlike previous approaches, our approach makes no assumptions about task distribution and requires no task exploration to infer tasks. We evaluate our approach in experiments with concurrently and sequentially presented tasks and show that it outperforms other multi-task learning approaches in terms of generalization performance and convergence speed, particularly in the continual learning setting.

Related papers

Continual Robot Learning using Self-Supervised Task Inference [19.635428830237842]
We propose a self-supervised task inference approach to continually learn new tasks. We use a behavior-matching self-supervised learning objective to train a novel Task Inference Network (TINet) A multi-task policy is built on top of the TINet and trained with reinforcement learning to optimize performance over tasks.
arXiv Detail & Related papers (2023-09-10T09:32:35Z)
Autonomous Open-Ended Learning of Tasks with Non-Stationary Interdependencies [64.0476282000118]
Intrinsic motivations have proven to generate a task-agnostic signal to properly allocate the training time amongst goals. While the majority of works in the field of intrinsically motivated open-ended learning focus on scenarios where goals are independent from each other, only few of them studied the autonomous acquisition of interdependent tasks. In particular, we first deepen the analysis of a previous system, showing the importance of incorporating information about the relationships between tasks at a higher level of the architecture. Then we introduce H-GRAIL, a new system that extends the previous one by adding a new learning layer to store the autonomously acquired sequences
arXiv Detail & Related papers (2022-05-16T10:43:01Z)
Unsupervised Reinforcement Learning for Transferable Manipulation Skill Discovery [22.32327908453603]
Current reinforcement learning (RL) in robotics often experiences difficulty in generalizing to new downstream tasks. We propose a framework that pre-trains the agent in a task-agnostic manner without access to the task-specific reward. We show that our approach achieves the most diverse interacting behavior and significantly improves sample efficiency in downstream tasks.
arXiv Detail & Related papers (2022-04-29T06:57:46Z)
Towards More Generalizable One-shot Visual Imitation Learning [81.09074706236858]
A general-purpose robot should be able to master a wide range of tasks and quickly learn a novel one by leveraging past experiences. One-shot imitation learning (OSIL) approaches this goal by training an agent with (pairs of) expert demonstrations. We push for a higher level of generalization ability by investigating a more ambitious multi-task setup.
arXiv Detail & Related papers (2021-10-26T05:49:46Z)
Lifelong Robotic Reinforcement Learning by Retaining Experiences [61.79346922421323]
Many multi-task reinforcement learning efforts assume the robot can collect data from all tasks at all times. In this work, we study a practical sequential multi-task RL problem motivated by the practical constraints of physical robotic systems. We derive an approach that effectively leverages the data and policies learned for previous tasks to cumulatively grow the robot's skill-set.
arXiv Detail & Related papers (2021-09-19T18:00:51Z)
Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks [60.380501589764144]
A longstanding goal of artificial intelligence is to create artificial agents capable of learning to perform tasks that require sequential decision making. While it is the artificial agent that learns and acts, it is still up to humans to specify the particular task to be performed. This survey provides a high-level overview of five recent machine learning frameworks that primarily rely on human guidance.
arXiv Detail & Related papers (2021-07-13T03:11:04Z)
Discovering Generalizable Skills via Automated Generation of Diverse Tasks [82.16392072211337]
We propose a method to discover generalizable skills via automated generation of a diverse set of tasks. As opposed to prior work on unsupervised discovery of skills, our method pairs each skill with a unique task produced by a trainable task generator. A task discriminator defined on the robot behaviors in the generated tasks is jointly trained to estimate the evidence lower bound of the diversity objective. The learned skills can then be composed in a hierarchical reinforcement learning algorithm to solve unseen target tasks.
arXiv Detail & Related papers (2021-06-26T03:41:51Z)
CRIL: Continual Robot Imitation Learning via Generative and Prediction Model [8.896427780114703]
We study how to realize continual imitation learning ability that empowers robots to continually learn new tasks one by one. We propose a novel trajectory generation model that employs both a generative adversarial network and a dynamics prediction model. Our experiments on both simulation and real world manipulation tasks demonstrate the effectiveness of our method.
arXiv Detail & Related papers (2021-06-17T12:15:57Z)
Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer Learning to Discover Task Hierarchy [0.0]
In open-ended continuous environments, robots need to learn multiple parameterised control tasks in hierarchical reinforcement learning. We show that the most complex tasks can be learned more easily by transferring knowledge from simpler tasks, and faster by adapting the complexity of the actions to the task. We propose a task-oriented representation of complex actions, called procedures, to learn online task relationships and unbounded sequences of action primitives to control the different observables of the environment.
arXiv Detail & Related papers (2021-02-19T10:44:08Z)
Scalable Multi-Task Imitation Learning with Autonomous Improvement [159.9406205002599]
We build an imitation learning system that can continuously improve through autonomous data collection. We leverage the robot's own trials as demonstrations for tasks other than the one that the robot actually attempted. In contrast to prior imitation learning approaches, our method can autonomously collect data with sparse supervision for continuous improvement.
arXiv Detail & Related papers (2020-02-25T18:56:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.