Learning Generalizable Skills from Offline Multi-Task Data for Multi-Agent Cooperation

• URL: http://arxiv.org/abs/2503.21200v1
• Date: Thu, 27 Mar 2025 06:35:59 GMT
• Title: Learning Generalizable Skills from Offline Multi-Task Data for Multi-Agent Cooperation
• Authors: Sicong Liu, Yang Shu, Chenjuan Guo, Bin Yang,
• Abstract summary: Hierarchical and Separate Skill Discovery (HiSSD) is a novel approach for generalizable offline multi-task MARL through skill learning.<n>HiSSD leverages a hierarchical framework that jointly learns common and task-specific skills.<n>The task-specific skills represent the priors of each task and achieve a task-guided fine-grained action execution.
• Score: 16.107745320103444
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning cooperative multi-agent policy from offline multi-task data that can generalize to unseen tasks with varying numbers of agents and targets is an attractive problem in many scenarios. Although aggregating general behavior patterns among multiple tasks as skills to improve policy transfer is a promising approach, two primary challenges hinder the further advancement of skill learning in offline multi-task MARL. Firstly, extracting general cooperative behaviors from various action sequences as common skills lacks bringing cooperative temporal knowledge into them. Secondly, existing works only involve common skills and can not adaptively choose independent knowledge as task-specific skills in each task for fine-grained action execution. To tackle these challenges, we propose Hierarchical and Separate Skill Discovery (HiSSD), a novel approach for generalizable offline multi-task MARL through skill learning. HiSSD leverages a hierarchical framework that jointly learns common and task-specific skills. The common skills learn cooperative temporal knowledge and enable in-sample exploitation for offline multi-task MARL. The task-specific skills represent the priors of each task and achieve a task-guided fine-grained action execution. To verify the advancement of our method, we conduct experiments on multi-agent MuJoCo and SMAC benchmarks. After training the policy using HiSSD on offline multi-task data, the empirical results show that HiSSD assigns effective cooperative behaviors and obtains superior performance in unseen tasks.

Related papers

• Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learning [11.790581500542439]
  Reinforcement learning (RL) with diverse offline datasets can have the advantage of leveraging the relation of multiple tasks. We present a skill-based multi-task RL technique on heterogeneous datasets that are generated by behavior policies of different quality. We show that our multi-task offline RL approach is robust to the mixed configurations of different-quality datasets.
  arXiv  Detail & Related papers  (2024-08-28T07:36:20Z)
• Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skills [17.666749042008178]
  We propose a framework called decoupled meta-reinforcement learning (DCMRL) DCMRL pulls in similar task contexts within the same task and pushing away different task contexts of different tasks. Experiments show that DCMRL is more effective than previous meta-RL methods with more generalizable prior experience.
  arXiv  Detail & Related papers  (2023-12-11T16:50:14Z)
• Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding [52.723297744257536]
  Pre-trained language models (LMs) have shown effectiveness in scientific literature understanding tasks. We propose a multi-task contrastive learning framework, SciMult, to facilitate common knowledge sharing across different literature understanding tasks.
  arXiv  Detail & Related papers  (2023-05-23T16:47:22Z)
• Multi-agent Continual Coordination via Progressive Task Contextualization [5.31057635825112]
  This paper proposes an approach Multi-Agent Continual Coordination via Progressive Task Contextualization, dubbed MACPro. We show in multiple multi-agent benchmarks that existing continual learning methods fail, while MACPro is able to achieve close-to-optimal performance.
  arXiv  Detail & Related papers  (2023-05-07T15:04:56Z)
• Active Task Randomization: Learning Robust Skills via Unsupervised Generation of Diverse and Feasible Tasks [37.73239471412444]
  We introduce Active Task Randomization (ATR), an approach that learns robust skills through the unsupervised generation of training tasks. ATR selects suitable tasks, which consist of an initial environment state and manipulation goal, for learning robust skills by balancing the diversity and feasibility of the tasks. We demonstrate that the learned skills can be composed by a task planner to solve unseen sequential manipulation problems based on visual inputs.
  arXiv  Detail & Related papers  (2022-11-11T11:24:55Z)
• LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learning [122.47938710284784]
  We propose a novel framework for learning dynamic subtask assignment (LDSA) in cooperative MARL. To reasonably assign agents to different subtasks, we propose an ability-based subtask selection strategy. We show that LDSA learns reasonable and effective subtask assignment for better collaboration.
  arXiv  Detail & Related papers  (2022-05-05T10:46:16Z)
• Modular Adaptive Policy Selection for Multi-Task Imitation Learning through Task Division [60.232542918414985]
  Multi-task learning often suffers from negative transfer, sharing information that should be task-specific. This is done by using proto-policies as modules to divide the tasks into simple sub-behaviours that can be shared. We also demonstrate its ability to autonomously divide the tasks into both shared and task-specific sub-behaviours.
  arXiv  Detail & Related papers  (2022-03-28T15:53:17Z)
• Variational Multi-Task Learning with Gumbel-Softmax Priors [105.22406384964144]
  Multi-task learning aims to explore task relatedness to improve individual tasks. We propose variational multi-task learning (VMTL), a general probabilistic inference framework for learning multiple related tasks.
  arXiv  Detail & Related papers  (2021-11-09T18:49:45Z)
• Towards More Generalizable One-shot Visual Imitation Learning [81.09074706236858]
  A general-purpose robot should be able to master a wide range of tasks and quickly learn a novel one by leveraging past experiences. One-shot imitation learning (OSIL) approaches this goal by training an agent with (pairs of) expert demonstrations. We push for a higher level of generalization ability by investigating a more ambitious multi-task setup.
  arXiv  Detail & Related papers  (2021-10-26T05:49:46Z)
• Efficiently Identifying Task Groupings for Multi-Task Learning [55.80489920205404]
  Multi-task learning can leverage information learned by one task to benefit the training of other tasks. We suggest an approach to select which tasks should train together in multi-task learning models. Our method determines task groupings in a single training run by co-training all tasks together and quantifying the effect to which one task's gradient would affect another task's loss.
  arXiv  Detail & Related papers  (2021-09-10T02:01:43Z)
• Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study [75.42182503265056]
  Multi-Task Learning has emerged as a methodology in which multiple tasks are jointly learned by a shared learning algorithm. We deal with heterogeneous MTL, simultaneously addressing detection, classification & regression problems. We build FaceBehaviorNet, the first framework for large-scale face analysis, by jointly learning all facial behavior tasks.
  arXiv  Detail & Related papers  (2021-05-08T22:26:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.