Learning to Transfer Role Assignment Across Team Sizes

• URL: http://arxiv.org/abs/2204.12937v1
• Date: Sun, 17 Apr 2022 11:22:01 GMT
• Title: Learning to Transfer Role Assignment Across Team Sizes
• Authors: Dung Nguyen, Phuoc Nguyen, Svetha Venkatesh, Truyen Tran
• Abstract summary: We propose a framework to learn role assignment and transfer across team sizes. We demonstrate that re-using the role-based credit assignment structure can foster the learning process of larger reinforcement learning teams.
• Score: 48.43860606706273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-agent reinforcement learning holds the key for solving complex tasks that demand the coordination of learning agents. However, strong coordination often leads to expensive exploration over the exponentially large state-action space. A powerful approach is to decompose team works into roles, which are ideally assigned to agents with the relevant skills. Training agents to adaptively choose and play emerging roles in a team thus allows the team to scale to complex tasks and quickly adapt to changing environments. These promises, however, have not been fully realised by current role-based multi-agent reinforcement learning methods as they assume either a pre-defined role structure or a fixed team size. We propose a framework to learn role assignment and transfer across team sizes. In particular, we train a role assignment network for small teams by demonstration and transfer the network to larger teams, which continue to learn through interaction with the environment. We demonstrate that re-using the role-based credit assignment structure can foster the learning process of larger reinforcement learning teams to achieve tasks requiring different roles. Our proposal outperforms competing techniques in enriched role-enforcing Prey-Predator games and in new scenarios in the StarCraft II Micro-Management benchmark.

Related papers

• Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation [52.930183136111864]
  We propose using scorable negotiation to evaluate Large Language Models (LLMs) To reach an agreement, agents must have strong arithmetic, inference, exploration, and planning capabilities. We provide procedures to create new games and increase games' difficulty to have an evolving benchmark.
  arXiv  Detail & Related papers  (2023-09-29T13:33:06Z)
• MindAgent: Emergent Gaming Interaction [103.73707345211892]
  Large Language Models (LLMs) have the capacity of performing complex scheduling in a multi-agent system. We propose MindAgent to evaluate planning and coordination emergent capabilities for gaming interaction.
  arXiv  Detail & Related papers  (2023-09-18T17:52:22Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• Towards a Better Understanding of Learning with Multiagent Teams [4.746424588605832]
  We show that some team structures help agents learn to specialize into specific roles, resulting in more favorable global results. Large teams create credit assignment challenges that reduce coordination, leading to large teams performing poorly compared to smaller ones.
  arXiv  Detail & Related papers  (2023-06-28T13:37:48Z)
• Learning to Learn Group Alignment: A Self-Tuning Credo Framework with Multiagent Teams [1.370633147306388]
  Mixed incentives among a population with multiagent teams has been shown to have advantages over a fully cooperative system. We propose a framework where individual learning agents self-regulate their configuration of incentives through various parts of their reward function.
  arXiv  Detail & Related papers  (2023-04-14T18:16:19Z)
• Multi-Agent Policy Transfer via Task Relationship Modeling [28.421365805638953]
  We try to discover and exploit common structures among tasks for more efficient transfer. We propose to learn effect-based task representations as a common space of tasks, using an alternatively fixed training scheme. As a result, the proposed method can help transfer learned cooperation knowledge to new tasks after training on a few source tasks.
  arXiv  Detail & Related papers  (2022-03-09T01:49:21Z)
• Two-stage training algorithm for AI robot soccer [2.0757564643017092]
  Two-stage heterogeneous centralized training is proposed to improve the learning performance of heterogeneous agents. The proposed method is applied to 5 versus 5 AI robot soccer for validation.
  arXiv  Detail & Related papers  (2021-04-13T04:24:13Z)
• RODE: Learning Roles to Decompose Multi-Agent Tasks [69.56458960841165]
  Role-based learning holds the promise of achieving scalable multi-agent learning by decomposing complex tasks using roles. We propose to first decompose joint action spaces into restricted role action spaces by clustering actions according to their effects on the environment and other agents. By virtue of these advances, our method outperforms the current state-of-the-art MARL algorithms on 10 of the 14 scenarios that comprise the challenging StarCraft II micromanagement benchmark.
  arXiv  Detail & Related papers  (2020-10-04T09:20:59Z)
• Learning from Learners: Adapting Reinforcement Learning Agents to be Competitive in a Card Game [71.24825724518847]
  We present a study on how popular reinforcement learning algorithms can be adapted to learn and to play a real-world implementation of a competitive multiplayer card game. We propose specific training and validation routines for the learning agents, in order to evaluate how the agents learn to be competitive and explain how they adapt to each others' playing style.
  arXiv  Detail & Related papers  (2020-04-08T14:11:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.