Autonomous learning of multiple, context-dependent tasks
- URL: http://arxiv.org/abs/2011.13847v1
- Date: Fri, 27 Nov 2020 17:25:36 GMT
- Title: Autonomous learning of multiple, context-dependent tasks
- Authors: Vieri Giuliano Santucci and Davide Montella and Bruno Castro da Silva
and Gianluca Baldassarre
- Abstract summary: In complex environments, the same task might need a set of different skills to be solved.
We propose a novel open-ended learning robot architecture, C-GRAIL, that solves the two challenges in an integrated fashion.
The architecture is tested in a simulated robotic environment involving a robot that autonomously learns to reach relevant target objects.
- Score: 1.1470070927586016
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: When facing the problem of autonomously learning multiple tasks with
reinforcement learning systems, researchers typically focus on solutions where
just one parametrised policy per task is sufficient to solve them. However, in
complex environments presenting different contexts, the same task might need a
set of different skills to be solved. These situations pose two challenges: (a)
to recognise the different contexts that need different policies; (b) quickly
learn the policies to accomplish the same tasks in the new discovered contexts.
These two challenges are even harder if faced within an open-ended learning
framework where an agent has to autonomously discover the goals that it might
accomplish in a given environment, and also to learn the motor skills to
accomplish them. We propose a novel open-ended learning robot architecture,
C-GRAIL, that solves the two challenges in an integrated fashion. In
particular, the architecture is able to detect new relevant contests, and
ignore irrelevant ones, on the basis of the decrease of the expected
performance for a given goal. Moreover, the architecture can quickly learn the
policies for the new contexts by exploiting transfer learning importing
knowledge from already acquired policies. The architecture is tested in a
simulated robotic environment involving a robot that autonomously learns to
reach relevant target objects in the presence of multiple obstacles generating
several different obstacles. The proposed architecture outperforms other models
not using the proposed autonomous context-discovery and transfer-learning
mechanisms.
Related papers
- I Know How: Combining Prior Policies to Solve New Tasks [17.214443593424498]
Multi-Task Reinforcement Learning aims at developing agents that are able to continually evolve and adapt to new scenarios.
Learning from scratch for each new task is not a viable or sustainable option.
We propose a new framework, I Know How, which provides a common formalization.
arXiv Detail & Related papers (2024-06-14T08:44:51Z) - MacGyver: Are Large Language Models Creative Problem Solvers? [87.70522322728581]
We explore the creative problem-solving capabilities of modern LLMs in a novel constrained setting.
We create MACGYVER, an automatically generated dataset consisting of over 1,600 real-world problems.
We present our collection to both LLMs and humans to compare and contrast their problem-solving abilities.
arXiv Detail & Related papers (2023-11-16T08:52:27Z) - Autonomous Open-Ended Learning of Tasks with Non-Stationary
Interdependencies [64.0476282000118]
Intrinsic motivations have proven to generate a task-agnostic signal to properly allocate the training time amongst goals.
While the majority of works in the field of intrinsically motivated open-ended learning focus on scenarios where goals are independent from each other, only few of them studied the autonomous acquisition of interdependent tasks.
In particular, we first deepen the analysis of a previous system, showing the importance of incorporating information about the relationships between tasks at a higher level of the architecture.
Then we introduce H-GRAIL, a new system that extends the previous one by adding a new learning layer to store the autonomously acquired sequences
arXiv Detail & Related papers (2022-05-16T10:43:01Z) - Policy Architectures for Compositional Generalization in Control [71.61675703776628]
We introduce a framework for modeling entity-based compositional structure in tasks.
Our policies are flexible and can be trained end-to-end without requiring any action primitives.
arXiv Detail & Related papers (2022-03-10T06:44:24Z) - Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in
Dynamic Environments [0.5277756703318046]
Key challenge for AI is to build embodied systems that operate in dynamically changing environments.
Standard deep learning systems often struggle in dynamic scenarios.
In this article we investigate biologically inspired architectures as solutions.
arXiv Detail & Related papers (2021-12-31T19:52:42Z) - From Machine Learning to Robotics: Challenges and Opportunities for
Embodied Intelligence [113.06484656032978]
Article argues that embodied intelligence is a key driver for the advancement of machine learning technology.
We highlight challenges and opportunities specific to embodied intelligence.
We propose research directions which may significantly advance the state-of-the-art in robot learning.
arXiv Detail & Related papers (2021-10-28T16:04:01Z) - Self-supervised Reinforcement Learning with Independently Controllable
Subgoals [20.29444813790076]
Self-supervised agents set their own goals by exploiting the structure in the environment.
Some of them were applied to learn basic manipulation skills in compositional multi-object environments.
We propose a novel self-supervised agent that estimates relations between environment components and uses them to independently control different parts of the environment state.
arXiv Detail & Related papers (2021-09-09T10:21:02Z) - Towards Coordinated Robot Motions: End-to-End Learning of Motion
Policies on Transform Trees [63.31965375413414]
We propose to solve multi-task problems through learning structured policies from human demonstrations.
Our structured policy is inspired by RMPflow, a framework for combining subtask policies on different spaces.
We derive an end-to-end learning objective function that is suitable for the multi-task problem.
arXiv Detail & Related papers (2020-12-24T22:46:22Z) - Latent Skill Planning for Exploration and Transfer [49.25525932162891]
In this paper, we investigate how these two approaches can be integrated into a single reinforcement learning agent.
We leverage the idea of partial amortization for fast adaptation at test time.
We demonstrate the benefits of our design decisions across a suite of challenging locomotion tasks.
arXiv Detail & Related papers (2020-11-27T18:40:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.