Projected Task-Specific Layers for Multi-Task Reinforcement Learning

• URL: http://arxiv.org/abs/2309.08776v2
• Date: Wed, 6 Mar 2024 18:51:45 GMT
• Title: Projected Task-Specific Layers for Multi-Task Reinforcement Learning
• Authors: Josselin Somerville Roberts, Julia Di
• Abstract summary: Multi-task reinforcement learning could enable robots to scale across a wide variety of manipulation tasks in homes and workplaces. Addressing this challenge by successfully sharing information across tasks will depend on how well the structure underlying the tasks is captured.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-task reinforcement learning could enable robots to scale across a wide variety of manipulation tasks in homes and workplaces. However, generalizing from one task to another and mitigating negative task interference still remains a challenge. Addressing this challenge by successfully sharing information across tasks will depend on how well the structure underlying the tasks is captured. In this work, we introduce our new architecture, Projected Task-Specific Layers (PTSL), that leverages a common policy with dense task-specific corrections through task-specific layers to better express shared and variable task information. We then show that our model outperforms the state of the art on the MT10 and MT50 benchmarks of Meta-World consisting of 10 and 50 goal-conditioned tasks for a Sawyer arm.

Related papers

• Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts [75.75548749888029]
  We present a vision-language model whose parameters are jointly trained on all tasks and fully shared among multiple heterogeneous tasks. With a single model, Musketeer achieves results comparable to or better than strong baselines trained on single tasks, almost uniformly across multiple tasks.
  arXiv  Detail & Related papers  (2023-05-11T17:57:49Z)
• Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization [101.72755769194677]
  We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph. Our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks. Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks.
  arXiv  Detail & Related papers  (2022-05-25T10:44:25Z)
• Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners [67.5865966762559]
  We study whether sparsely activated Mixture-of-Experts (MoE) improve multi-task learning. We devise task-aware gating functions to route examples from different tasks to specialized experts. This results in a sparsely activated multi-task model with a large number of parameters, but with the same computational cost as that of a dense model.
  arXiv  Detail & Related papers  (2022-04-16T00:56:12Z)
• Modular Adaptive Policy Selection for Multi-Task Imitation Learning through Task Division [60.232542918414985]
  Multi-task learning often suffers from negative transfer, sharing information that should be task-specific. This is done by using proto-policies as modules to divide the tasks into simple sub-behaviours that can be shared. We also demonstrate its ability to autonomously divide the tasks into both shared and task-specific sub-behaviours.
  arXiv  Detail & Related papers  (2022-03-28T15:53:17Z)
• Learning Multi-Tasks with Inconsistent Labels by using Auxiliary Big Task [24.618094251341958]
  Multi-task learning is to improve the performance of the model by transferring and exploiting common knowledge among tasks. We propose a framework to learn these tasks by jointly leveraging both abundant information from a learnt auxiliary big task with sufficiently many classes to cover those of all these tasks. Our experimental results demonstrate its effectiveness in comparison with the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-01-07T02:46:47Z)
• Multi-Task Learning with Sequence-Conditioned Transporter Networks [67.57293592529517]
  We aim to solve multi-task learning through the lens of sequence-conditioning and weighted sampling. We propose a new suite of benchmark aimed at compositional tasks, MultiRavens, which allows defining custom task combinations. Second, we propose a vision-based end-to-end system architecture, Sequence-Conditioned Transporter Networks, which augments Goal-Conditioned Transporter Networks with sequence-conditioning and weighted sampling.
  arXiv  Detail & Related papers  (2021-09-15T21:19:11Z)
• Efficiently Identifying Task Groupings for Multi-Task Learning [55.80489920205404]
  Multi-task learning can leverage information learned by one task to benefit the training of other tasks. We suggest an approach to select which tasks should train together in multi-task learning models. Our method determines task groupings in a single training run by co-training all tasks together and quantifying the effect to which one task's gradient would affect another task's loss.
  arXiv  Detail & Related papers  (2021-09-10T02:01:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.