Evolving Hierarchical Memory-Prediction Machines in Multi-Task Reinforcement Learning

• URL: http://arxiv.org/abs/2106.12659v1
• Date: Wed, 23 Jun 2021 21:34:32 GMT
• Title: Evolving Hierarchical Memory-Prediction Machines in Multi-Task Reinforcement Learning
• Authors: Stephen Kelly, Tatiana Voegerl, Wolfgang Banzhaf, Cedric Gondro
• Abstract summary: Behavioural agents must generalize across a variety of environments and objectives over time. We use genetic programming to evolve highly-generalized agents capable of operating in six unique environments from the control literature. We show that emergent hierarchical structure in the evolving programs leads to multi-task agents that succeed by performing a temporal decomposition and encoding of the problem environments in memory.
• Score: 4.030910640265943
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: A fundamental aspect of behaviour is the ability to encode salient features of experience in memory and use these memories, in combination with current sensory information, to predict the best action for each situation such that long-term objectives are maximized. The world is highly dynamic, and behavioural agents must generalize across a variety of environments and objectives over time. This scenario can be modeled as a partially-observable multi-task reinforcement learning problem. We use genetic programming to evolve highly-generalized agents capable of operating in six unique environments from the control literature, including OpenAI's entire Classic Control suite. This requires the agent to support discrete and continuous actions simultaneously. No task-identification sensor inputs are provided, thus agents must identify tasks from the dynamics of state variables alone and define control policies for each task. We show that emergent hierarchical structure in the evolving programs leads to multi-task agents that succeed by performing a temporal decomposition and encoding of the problem environments in memory. The resulting agents are competitive with task-specific agents in all six environments. Furthermore, the hierarchical structure of programs allows for dynamic run-time complexity, which results in relatively efficient operation.

Related papers

• AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation [89.68433168477227]
  Large Language Model (LLM) based agents have garnered significant attention and are becoming increasingly popular. This paper investigates enhancing the planning abilities of LLMs through instruction tuning. To address this limitation, this paper explores the automated synthesis of diverse environments and a gradual range of planning tasks.
  arXiv  Detail & Related papers  (2024-08-01T17:59:46Z)
• TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generation [45.028795422801764]
  We propose a multi-agent framework based on dynamic Task Decomposition and Agent Generation (TDAG) This framework dynamically decomposes complex tasks into smaller subtasks and assigns each to a specifically generated subagent. ItineraryBench is designed to assess agents' abilities in memory, planning, and tool usage across tasks of varying complexity.
  arXiv  Detail & Related papers  (2024-02-15T18:27:37Z)
• RObotic MAnipulation Network (ROMAN) $\unicode{x2013}$ Hybrid Hierarchical Learning for Solving Complex Sequential Tasks [70.69063219750952]
  We present a Hybrid Hierarchical Learning framework, the Robotic Manipulation Network (ROMAN) ROMAN achieves task versatility and robust failure recovery by integrating behavioural cloning, imitation learning, and reinforcement learning. Experimental results show that by orchestrating and activating these specialised manipulation experts, ROMAN generates correct sequential activations for accomplishing long sequences of sophisticated manipulation tasks.
  arXiv  Detail & Related papers  (2023-06-30T20:35:22Z)
• Relax, it doesn't matter how you get there: A new self-supervised approach for multi-timescale behavior analysis [8.543808476554695]
  We develop a multi-task representation learning model for behavior that combines two novel components. Our model ranks 1st overall and on all global tasks, and 1st or 2nd on 7 out of 9 frame-level tasks.
  arXiv  Detail & Related papers  (2023-03-15T17:58:48Z)
• Controllable Dynamic Multi-Task Architectures [92.74372912009127]
  We propose a controllable multi-task network that dynamically adjusts its architecture and weights to match the desired task preference as well as the resource constraints. We propose a disentangled training of two hypernetworks, by exploiting task affinity and a novel branching regularized loss, to take input preferences and accordingly predict tree-structured models with adapted weights.
  arXiv  Detail & Related papers  (2022-03-28T17:56:40Z)
• Reinforcement Learning for Location-Aware Scheduling [1.0660480034605238]
  We show how various aspects of the warehouse environment affect performance and execution priority. We propose a compact representation of the state and action space for location-aware multi-agent systems. We also show how agents trained in certain environments maintain performance in completely unseen settings.
  arXiv  Detail & Related papers  (2022-03-07T15:51:00Z)
• Hierarchically Structured Scheduling and Execution of Tasks in a Multi-Agent Environment [1.0660480034605238]
  In a warehouse environment, tasks appear dynamically. Consequently, a task management system that matches them with the workforce too early is necessarily sub-optimal. We propose to use deep reinforcement learning to solve both the high-level scheduling problem and the low-level multi-agent problem of schedule execution.
  arXiv  Detail & Related papers  (2022-03-06T18:11:34Z)
• Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning [59.62721526353915]
  Multi-agent settings in the real world often involve tasks with varying types and quantities of agents and non-agent entities. Our method aims to leverage these commonalities by asking the question: What is the expected utility of each agent when only considering a randomly selected sub-group of its observed entities?''
  arXiv  Detail & Related papers  (2020-06-07T18:28:41Z)
• Dynamic Multi-Robot Task Allocation under Uncertainty and Temporal Constraints [52.58352707495122]
  We present a multi-robot allocation algorithm that decouples the key computational challenges of sequential decision-making under uncertainty and multi-agent coordination. We validate our results over a wide range of simulations on two distinct domains: multi-arm conveyor belt pick-and-place and multi-drone delivery dispatch in a city.
  arXiv  Detail & Related papers  (2020-05-27T01:10:41Z)
• Hierarchical Reinforcement Learning as a Model of Human Task Interleaving [60.95424607008241]
  We develop a hierarchical model of supervisory control driven by reinforcement learning. The model reproduces known empirical effects of task interleaving. The results support hierarchical RL as a plausible model of task interleaving.
  arXiv  Detail & Related papers  (2020-01-04T17:53:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.