Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition

• URL: http://arxiv.org/abs/2105.08692v1
• Date: Tue, 18 May 2021 17:27:37 GMT
• Title: Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition
• Authors: Bo Liu, Qiang Liu, Peter Stone, Animesh Garg, Yuke Zhu and Animashree Anandkumar
• Abstract summary: In real-world multiagent systems, agents with different capabilities may join or leave without altering the team's overarching goals. We propose COPA, a coach-player framework to tackle this problem. We 1) adopt the attention mechanism for both the coach and the players; 2) propose a variational objective to regularize learning; and 3) design an adaptive communication method to let the coach decide when to communicate with the players.
• Score: 88.26752130107259
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In real-world multiagent systems, agents with different capabilities may join or leave without altering the team's overarching goals. Coordinating teams with such dynamic composition is challenging: the optimal team strategy varies with the composition. We propose COPA, a coach-player framework to tackle this problem. We assume the coach has a global view of the environment and coordinates the players, who only have partial views, by distributing individual strategies. Specifically, we 1) adopt the attention mechanism for both the coach and the players; 2) propose a variational objective to regularize learning; and 3) design an adaptive communication method to let the coach decide when to communicate with the players. We validate our methods on a resource collection task, a rescue game, and the StarCraft micromanagement tasks. We demonstrate zero-shot generalization to new team compositions. Our method achieves comparable or better performance than the setting where all players have a full view of the environment. Moreover, we see that the performance remains high even when the coach communicates as little as 13% of the time using the adaptive communication strategy.

Related papers

• Leading the Pack: N-player Opponent Shaping [52.682734939786464]
  We extend Opponent Shaping (OS) methods to environments involving multiple co-players and multiple shaping agents. We find that when playing with a large number of co-players, OS methods' relative performance reduces, suggesting that in the limit OS methods may not perform well.
  arXiv  Detail & Related papers  (2023-12-19T20:01:42Z)
• All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization [57.615269148301515]
  In a competitive game scenario, a set of agents have to learn decisions that maximize their goals and minimize their adversaries' goals at the same time. We propose a novel model composed of three neural layers that learn a representation of a competitive game, learn how to map the strategy of specific opponents, and how to disrupt them. Our experiments demonstrate that our model achieves better performance when playing against offline, online, and competitive-specific models, in particular when playing against the same opponent multiple times.
  arXiv  Detail & Related papers  (2023-10-02T08:11:07Z)
• Value-based CTDE Methods in Symmetric Two-team Markov Game: from Cooperation to Team Competition [3.828689444527739]
  We evaluate cooperative value-based methods in a mixed cooperative-competitive environment. We selected three training methods based on the centralised training and decentralised execution paradigm. For our experiments, we modified the StarCraft Multi-Agent Challenge environment to create competitive environments where both teams could learn and compete simultaneously.
  arXiv  Detail & Related papers  (2022-11-21T22:25:55Z)
• Learning to Transfer Role Assignment Across Team Sizes [48.43860606706273]
  We propose a framework to learn role assignment and transfer across team sizes. We demonstrate that re-using the role-based credit assignment structure can foster the learning process of larger reinforcement learning teams.
  arXiv  Detail & Related papers  (2022-04-17T11:22:01Z)
• Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversity [49.68758494467258]
  We study how to construct diverse populations of agents by carefully structuring how individuals within a population interact. Our approach is based on interaction graphs, which control the flow of information between agents during training. We provide evidence for the importance of diversity in multi-agent training and analyse the effect of applying different interaction graphs on the training trajectories, diversity and performance of populations in a range of games.
  arXiv  Detail & Related papers  (2021-10-08T11:29:52Z)
• Multi-Agent Coordination in Adversarial Environments through Signal Mediated Strategies [37.00818384785628]
  Team members can coordinate their strategies before the beginning of the game, but are unable to communicate during the playing phase of the game. In this setting, model-free RL methods are oftentimes unable to capture coordination because agents' policies are executed in a decentralized fashion. We show convergence to coordinated equilibria in cases where previous state-of-the-art multi-agent RL algorithms did not.
  arXiv  Detail & Related papers  (2021-02-09T18:44:16Z)
• An analysis of Reinforcement Learning applied to Coach task in IEEE Very Small Size Soccer [2.5400028272658144]
  This paper proposes an end-to-end approach for the coaching task based on Reinforcement Learning (RL) We trained two RL policies against three different teams in a simulated environment. Our results were assessed against one of the top teams of the VSSS league.
  arXiv  Detail & Related papers  (2020-11-23T23:10:06Z)
• Natural Emergence of Heterogeneous Strategies in Artificially Intelligent Competitive Teams [0.0]
  We develop a competitive multi agent environment called FortAttack in which two teams compete against each other. We observe a natural emergence of heterogeneous behavior amongst homogeneous agents when such behavior can lead to the team's success. We propose ensemble training, in which we utilize the evolved opponent strategies to train a single policy for friendly agents.
  arXiv  Detail & Related papers  (2020-07-06T22:35:56Z)
• Learning from Learners: Adapting Reinforcement Learning Agents to be Competitive in a Card Game [71.24825724518847]
  We present a study on how popular reinforcement learning algorithms can be adapted to learn and to play a real-world implementation of a competitive multiplayer card game. We propose specific training and validation routines for the learning agents, in order to evaluate how the agents learn to be competitive and explain how they adapt to each others' playing style.
  arXiv  Detail & Related papers  (2020-04-08T14:11:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.