Heterogeneous Multi-agent Zero-Shot Coordination by Coevolution

• URL: http://arxiv.org/abs/2208.04957v2
• Date: Sat, 20 Jan 2024 06:01:06 GMT
• Title: Heterogeneous Multi-agent Zero-Shot Coordination by Coevolution
• Authors: Ke Xue, Yutong Wang, Cong Guan, Lei Yuan, Haobo Fu, Qiang Fu, Chao Qian, Yang Yu
• Abstract summary: We study the heterogeneous zero-shot coordination (ZSC) problem for the first time. We propose a general method based on coevolution, which coevolves two populations of agents and partners through three sub-processes: pairing, updating and selection.
• Score: 41.23036865145942
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generating agents that can achieve zero-shot coordination (ZSC) with unseen partners is a new challenge in cooperative multi-agent reinforcement learning (MARL). Recently, some studies have made progress in ZSC by exposing the agents to diverse partners during the training process. They usually involve self-play when training the partners, implicitly assuming that the tasks are homogeneous. However, many real-world tasks are heterogeneous, and hence previous methods may be inefficient. In this paper, we study the heterogeneous ZSC problem for the first time and propose a general method based on coevolution, which coevolves two populations of agents and partners through three sub-processes: pairing, updating and selection. Experimental results on various heterogeneous tasks highlight the necessity of considering the heterogeneous setting and demonstrate that our proposed method is a promising solution for heterogeneous ZSC tasks.

Related papers

• QTypeMix: Enhancing Multi-Agent Cooperative Strategies through Heterogeneous and Homogeneous Value Decomposition [11.170571181947274]
  We propose QTypeMix, which divides the value decomposition process into homogeneous and heterogeneous stages. The results of testing the proposed method on 14 maps from SMAC and SMACv2 show that QTypeMix achieves state-of-the-art performance in tasks of varying difficulty.
  arXiv  Detail & Related papers  (2024-08-12T12:27:58Z)
• Prioritized League Reinforcement Learning for Large-Scale Heterogeneous Multiagent Systems [11.017749510087059]
  We propose a prioritized Heterogeneous League Reinforcement Learning (PHLRL) method to address large-scale heterogeneous cooperation problems. We use Unreal Engine to design a large-scale heterogeneous cooperation benchmark named Large-Scale Multiagent Operation (LSMO)
  arXiv  Detail & Related papers  (2024-03-26T19:21:50Z)
• An Extensible Framework for Open Heterogeneous Collaborative Perception [58.70875361688463]
  Collaborative perception aims to mitigate the limitations of single-agent perception. In this paper, we introduce a new open heterogeneous problem: how to accommodate continually emerging new heterogeneous agent types into collaborative perception. We propose HEterogeneous ALliance (HEAL), a novel collaborative perception framework.
  arXiv  Detail & Related papers  (2024-01-25T05:55:03Z)
• Towards Few-shot Coordination: Revisiting Ad-hoc Teamplay Challenge In the Game of Hanabi [15.917861586043813]
  We show that state-of-the-art ZSC algorithms have poor performance when paired with agents trained with different learning methods. We create a framework based on a popular cooperative multi-agent game called Hanabi to evaluate the adaptability of MARL methods.
  arXiv  Detail & Related papers  (2023-08-20T14:44:50Z)
• Heterogeneous Embodied Multi-Agent Collaboration [21.364827833498254]
  Heterogeneous multi-agent tasks are common in real-world scenarios. We propose the heterogeneous multi-agent tidying-up task, in which multiple heterogeneous agents collaborate to detect misplaced objects and place them in reasonable locations. We propose the hierarchical decision model based on misplaced object detection, reasonable receptacle prediction, as well as the handshake-based group communication mechanism.
  arXiv  Detail & Related papers  (2023-07-26T04:33:05Z)
• Adaptive Coordination in Social Embodied Rearrangement [49.35582108902819]
  We study zero-shot coordination (ZSC) in this task, where an agent collaborates with a new partner, emulating a scenario where a robot collaborates with a new human partner. We propose Behavior Diversity Play (BDP), a novel ZSC approach that encourages diversity through a discriminability objective. Our results demonstrate that BDP learns adaptive agents that can tackle visual coordination, and zero-shot generalize to new partners in unseen environments, achieving 35% higher success and 32% higher efficiency compared to baselines.
  arXiv  Detail & Related papers  (2023-05-31T18:05:51Z)
• Learning Reward Machines in Cooperative Multi-Agent Tasks [75.79805204646428]
  This paper presents a novel approach to Multi-Agent Reinforcement Learning (MARL) It combines cooperative task decomposition with the learning of reward machines (RMs) encoding the structure of the sub-tasks. The proposed method helps deal with the non-Markovian nature of the rewards in partially observable environments.
  arXiv  Detail & Related papers  (2023-03-24T15:12:28Z)
• PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordination [52.991211077362586]
  We propose a policy ensemble method to increase the diversity of partners in the population. We then develop a context-aware method enabling the ego agent to analyze and identify the partner's potential policy primitives. In this way, the ego agent is able to learn more universal cooperative behaviors for collaborating with diverse partners.
  arXiv  Detail & Related papers  (2023-01-16T12:14:58Z)
• Learning Heterogeneous Agent Cooperation via Multiagent League Training [6.801749815385998]
  This work proposes a general-purpose reinforcement learning algorithm named Heterogeneous League Training (HLT) to address heterogeneous multiagent problems. HLT keeps track of a pool of policies that agents have explored during training, gathering a league of heterogeneous policies to facilitate future policy optimization. A hyper-network is introduced to increase the diversity of agent behaviors when collaborating with teammates having different levels of cooperation skills.
  arXiv  Detail & Related papers  (2022-11-13T13:57:15Z)
• UneVEn: Universal Value Exploration for Multi-Agent Reinforcement Learning [53.73686229912562]
  We propose a novel MARL approach called Universal Value Exploration (UneVEn) UneVEn learns a set of related tasks simultaneously with a linear decomposition of universal successor features. Empirical results on a set of exploration games, challenging cooperative predator-prey tasks requiring significant coordination among agents, and StarCraft II micromanagement benchmarks show that UneVEn can solve tasks where other state-of-the-art MARL methods fail.
  arXiv  Detail & Related papers  (2020-10-06T19:08:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.