Autonomous Open-Ended Learning of Tasks with Non-Stationary Interdependencies

• URL: http://arxiv.org/abs/2205.07562v1
• Date: Mon, 16 May 2022 10:43:01 GMT
• Title: Autonomous Open-Ended Learning of Tasks with Non-Stationary Interdependencies
• Authors: Alejandro Romero, Gianluca Baldassarre, Richard J. Duro, Vieri Giuliano Santucci
• Abstract summary: Intrinsic motivations have proven to generate a task-agnostic signal to properly allocate the training time amongst goals. While the majority of works in the field of intrinsically motivated open-ended learning focus on scenarios where goals are independent from each other, only few of them studied the autonomous acquisition of interdependent tasks. In particular, we first deepen the analysis of a previous system, showing the importance of incorporating information about the relationships between tasks at a higher level of the architecture. Then we introduce H-GRAIL, a new system that extends the previous one by adding a new learning layer to store the autonomously acquired sequences
• Score: 64.0476282000118
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous open-ended learning is a relevant approach in machine learning and robotics, allowing the design of artificial agents able to acquire goals and motor skills without the necessity of user assigned tasks. A crucial issue for this approach is to develop strategies to ensure that agents can maximise their competence on as many tasks as possible in the shortest possible time. Intrinsic motivations have proven to generate a task-agnostic signal to properly allocate the training time amongst goals. While the majority of works in the field of intrinsically motivated open-ended learning focus on scenarios where goals are independent from each other, only few of them studied the autonomous acquisition of interdependent tasks, and even fewer tackled scenarios where goals involve non-stationary interdependencies. Building on previous works, we tackle these crucial issues at the level of decision making (i.e., building strategies to properly select between goals), and we propose a hierarchical architecture that treating sub-tasks selection as a Markov Decision Process is able to properly learn interdependent skills on the basis of intrinsically generated motivations. In particular, we first deepen the analysis of a previous system, showing the importance of incorporating information about the relationships between tasks at a higher level of the architecture (that of goal selection). Then we introduce H-GRAIL, a new system that extends the previous one by adding a new learning layer to store the autonomously acquired sequences of tasks to be able to modify them in case the interdependencies are non-stationary. All systems are tested in a real robotic scenario, with a Baxter robot performing multiple interdependent reaching tasks.

Related papers

• Practice Makes Perfect: Planning to Learn Skill Parameter Policies [34.51008914846429]
  In this work, we focus on the active learning problem of choosing which skills to practice to maximize expected future task success. We propose that the robot should estimate the competence of each skill, extrapolate the competence, and situate the skill in the task distribution through competence-aware planning. This approach is implemented within a fully autonomous system where the robot repeatedly plans, practices, and learns without any environment resets.
  arXiv  Detail & Related papers  (2024-02-22T23:58:26Z)
• Continual Robot Learning using Self-Supervised Task Inference [19.635428830237842]
  We propose a self-supervised task inference approach to continually learn new tasks. We use a behavior-matching self-supervised learning objective to train a novel Task Inference Network (TINet) A multi-task policy is built on top of the TINet and trained with reinforcement learning to optimize performance over tasks.
  arXiv  Detail & Related papers  (2023-09-10T09:32:35Z)
• Multitask Learning with No Regret: from Improved Confidence Bounds to Active Learning [79.07658065326592]
  Quantifying uncertainty in the estimated tasks is of pivotal importance for many downstream applications, such as online or active learning. We provide novel multitask confidence intervals in the challenging setting when neither the similarity between tasks nor the tasks' features are available to the learner. We propose a novel online learning algorithm that achieves such improved regret without knowing this parameter in advance.
  arXiv  Detail & Related papers  (2023-08-03T13:08:09Z)
• Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping [94.89128390954572]
  We propose a novel self-supervised learning phase on the pre-collected dataset to understand the structure and the dynamics of the model. We evaluate our method on three continuous control tasks, and show that our model significantly outperforms existing approaches.
  arXiv  Detail & Related papers  (2023-01-05T15:07:10Z)
• Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks [85.56153200251713]
  We introduce EMBR, a model-based RL method for learning primitive skills that are suitable for completing long-horizon visuomotor tasks. On a Franka Emika robot arm, we find that EMBR enables the robot to complete three long-horizon visuomotor tasks at 85% success rate.
  arXiv  Detail & Related papers  (2021-09-21T16:48:07Z)
• Behavior Self-Organization Supports Task Inference for Continual Robot Learning [18.071689266826212]
  We propose a novel approach to continual learning of robotic control tasks. Our approach performs unsupervised learning of behavior embeddings by incrementally self-organizing demonstrated behaviors. Unlike previous approaches, our approach makes no assumptions about task distribution and requires no task exploration to infer tasks.
  arXiv  Detail & Related papers  (2021-07-09T16:37:27Z)
• Autonomous learning of multiple, context-dependent tasks [1.1470070927586016]
  In complex environments, the same task might need a set of different skills to be solved. We propose a novel open-ended learning robot architecture, C-GRAIL, that solves the two challenges in an integrated fashion. The architecture is tested in a simulated robotic environment involving a robot that autonomously learns to reach relevant target objects.
  arXiv  Detail & Related papers  (2020-11-27T17:25:36Z)
• Automatic Curriculum Learning through Value Disagreement [95.19299356298876]
  Continually solving new, unsolved tasks is the key to learning diverse behaviors. In the multi-task domain, where an agent needs to reach multiple goals, the choice of training goals can largely affect sample efficiency. We propose setting up an automatic curriculum for goals that the agent needs to solve. We evaluate our method across 13 multi-goal robotic tasks and 5 navigation tasks, and demonstrate performance gains over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-06-17T03:58:25Z)
• Scalable Multi-Task Imitation Learning with Autonomous Improvement [159.9406205002599]
  We build an imitation learning system that can continuously improve through autonomous data collection. We leverage the robot's own trials as demonstrations for tasks other than the one that the robot actually attempted. In contrast to prior imitation learning approaches, our method can autonomously collect data with sparse supervision for continuous improvement.
  arXiv  Detail & Related papers  (2020-02-25T18:56:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.