Solving Compositional Reinforcement Learning Problems via Task Reduction

• URL: http://arxiv.org/abs/2103.07607v1
• Date: Sat, 13 Mar 2021 03:26:33 GMT
• Title: Solving Compositional Reinforcement Learning Problems via Task Reduction
• Authors: Yunfei Li, Yilin Wu, Huazhe Xu, Xiaolong Wang, Yi Wu
• Abstract summary: We propose a novel learning paradigm, Self-Imitation via Reduction (SIR) for solving compositional reinforcement learning problems. SIR is based on two core ideas: task reduction and self-imitation. Experiment results show that SIR can significantly accelerate and improve learning on a variety of challenging sparse-reward continuous-control problems.
• Score: 18.120631058025406
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a novel learning paradigm, Self-Imitation via Reduction (SIR), for solving compositional reinforcement learning problems. SIR is based on two core ideas: task reduction and self-imitation. Task reduction tackles a hard-to-solve task by actively reducing it to an easier task whose solution is known by the RL agent. Once the original hard task is successfully solved by task reduction, the agent naturally obtains a self-generated solution trajectory to imitate. By continuously collecting and imitating such demonstrations, the agent is able to progressively expand the solved subspace in the entire task space. Experiment results show that SIR can significantly accelerate and improve learning on a variety of challenging sparse-reward continuous-control problems with compositional structures.

Related papers

• Continual Deep Reinforcement Learning with Task-Agnostic Policy Distillation [0.0]
  The Task-Agnostic Policy Distillation (TAPD) framework is introduced. This paper addresses the problem of continual learning. By utilizing task-agnostic distilled knowledge, the agent can solve downstream tasks more efficiently.
  arXiv  Detail & Related papers  (2024-11-25T16:18:39Z)
• Reduce, Reuse, Recycle: Categories for Compositional Reinforcement Learning [19.821117942806474]
  We view task composition through the prism of category theory. The categorical properties of Markov decision processes untangle complex tasks into manageable sub-tasks. Experimental results support the categorical theory of reinforcement learning.
  arXiv  Detail & Related papers  (2024-08-23T21:23:22Z)
• RObotic MAnipulation Network (ROMAN) $\unicode{x2013}$ Hybrid Hierarchical Learning for Solving Complex Sequential Tasks [70.69063219750952]
  We present a Hybrid Hierarchical Learning framework, the Robotic Manipulation Network (ROMAN) ROMAN achieves task versatility and robust failure recovery by integrating behavioural cloning, imitation learning, and reinforcement learning. Experimental results show that by orchestrating and activating these specialised manipulation experts, ROMAN generates correct sequential activations for accomplishing long sequences of sophisticated manipulation tasks.
  arXiv  Detail & Related papers  (2023-06-30T20:35:22Z)
• Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning [56.26889258704261]
  We propose a novel "disentangled" decision-making method, Semantically Aligned task decomposition in MARL (SAMA) SAMA prompts pretrained language models with chain-of-thought that can suggest potential goals, provide suitable goal decomposition and subgoal allocation as well as self-reflection-based replanning. SAMA demonstrates considerable advantages in sample efficiency compared to state-of-the-art ASG methods.
  arXiv  Detail & Related papers  (2023-05-18T10:37:54Z)
• Learning Complex Teamwork Tasks Using a Given Sub-task Decomposition [11.998708550268978]
  We propose an approach which uses an expert-provided decomposition of a task into simpler multi-agent sub-tasks. In each sub-task, a subset of the entire team is trained to acquire sub-task-specific policies. The sub-teams are then merged and transferred to the target task, where their policies are collectively fine-tuned to solve the more complex target task.
  arXiv  Detail & Related papers  (2023-02-09T21:24:56Z)
• Relay Hindsight Experience Replay: Continual Reinforcement Learning for Robot Manipulation Tasks with Sparse Rewards [26.998587654269873]
  We propose a novel model-free continual RL algorithm, called Relay-HER (RHER) The proposed method first decomposes and rearranges the original long-horizon task into new sub-tasks with incremental complexity. The experimental results indicate the significant improvements in sample efficiency of RHER compared to vanilla-HER in five typical robot manipulation tasks.
  arXiv  Detail & Related papers  (2022-08-01T13:30:01Z)
• Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization [101.72755769194677]
  We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph. Our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks. Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks.
  arXiv  Detail & Related papers  (2022-05-25T10:44:25Z)
• Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention [67.1936055742498]
  We show that multi-task learning can effectively scale reset-free learning schemes to much more complex problems. This work shows the ability to learn dexterous manipulation behaviors in the real world with RL without any human intervention.
  arXiv  Detail & Related papers  (2021-04-22T17:38:27Z)
• Continual Learning of Control Primitives: Skill Discovery via Reset-Games [128.36174682118488]
  We show how a single method can allow an agent to acquire skills with minimal supervision. We do this by exploiting the insight that the need to "reset" an agent to a broad set of initial states for a learning task provides a natural setting to learn a diverse set of "reset-skills"
  arXiv  Detail & Related papers  (2020-11-10T18:07:44Z)
• Weakly-Supervised Reinforcement Learning for Controllable Behavior [126.04932929741538]
  Reinforcement learning (RL) is a powerful framework for learning to take actions to solve tasks. In many settings, an agent must winnow down the inconceivably large space of all possible tasks to the single task that it is currently being asked to solve. We introduce a framework for using weak supervision to automatically disentangle this semantically meaningful subspace of tasks from the enormous space of nonsensical "chaff" tasks.
  arXiv  Detail & Related papers  (2020-04-06T17:50:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.