Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module

• URL: http://arxiv.org/abs/2112.07222v2
• Date: Wed, 15 Dec 2021 13:14:35 GMT
• Title: Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module
• Authors: Wei-Cheng Tseng, Wei Wei, Da-Chen Juan, Min Sun
• Abstract summary: We formulate a multi-agent environment with a different number of agents as a multi-tasking problem. We propose a meta reinforcement learning (meta-RL) framework to tackle this problem. The proposed framework employs a meta-learned Communication Pattern Recognition (CPR) module to identify communication behavior.
• Score: 29.75594940509839
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Designing an effective communication mechanism among agents in reinforcement learning has been a challenging task, especially for real-world applications. The number of agents can grow or an environment sometimes needs to interact with a changing number of agents in real-world scenarios. To this end, a multi-agent framework needs to handle various scenarios of agents, in terms of both scales and dynamics, for being practical to real-world applications. We formulate the multi-agent environment with a different number of agents as a multi-tasking problem and propose a meta reinforcement learning (meta-RL) framework to tackle this problem. The proposed framework employs a meta-learned Communication Pattern Recognition (CPR) module to identify communication behavior and extract information that facilitates the training process. Experimental results are poised to demonstrate that the proposed framework (a) generalizes to an unseen larger number of agents and (b) allows the number of agents to change between episodes. The ablation study is also provided to reason the proposed CPR design and show such design is effective.

Related papers

• Communication Learning in Multi-Agent Systems from Graph Modeling Perspective [62.13508281188895]
  We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. We introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time.
  arXiv  Detail & Related papers  (2024-11-01T05:56:51Z)
• Learning Multi-Agent Communication from Graph Modeling Perspective [62.13508281188895]
  We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. Our proposed approach, CommFormer, efficiently optimize the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner.
  arXiv  Detail & Related papers  (2024-05-14T12:40:25Z)
• SEA: A Spatially Explicit Architecture for Multi-Agent Reinforcement Learning [14.935456456463731]
  We propose a spatial information extraction structure for multi-agent reinforcement learning. Agents can effectively share the neighborhood and global information through a spatially encoder-decoder structure.
  arXiv  Detail & Related papers  (2023-04-25T03:00:09Z)
• Graph Exploration for Effective Multi-agent Q-Learning [46.723361065955544]
  This paper proposes an exploration technique for multi-agent reinforcement learning (MARL) with graph-based communication among agents. We assume the individual rewards received by the agents are independent of the actions by the other agents, while their policies are coupled. In the proposed framework, neighbouring agents collaborate to estimate the uncertainty about the state-action space in order to execute more efficient explorative behaviour.
  arXiv  Detail & Related papers  (2023-04-19T10:28:28Z)
• Universally Expressive Communication in Multi-Agent Reinforcement Learning [6.086083595135936]
  We consider the question of whether a given communication protocol can express an arbitrary policy. With standard GNN approaches provably limited in their expressive capacity, we consider augmenting agent observations with: (1) unique agent IDs and (2) random noise. We provide a theoretical analysis as to how these approaches yield universally expressive communication, and also prove them capable of targeting arbitrary sets of actions for identical agents.
  arXiv  Detail & Related papers  (2022-06-14T11:16:33Z)
• PooL: Pheromone-inspired Communication Framework forLarge Scale Multi-Agent Reinforcement Learning [0.0]
  textbfPooL is an indirect communication framework applied to large scale multi-agent reinforcement textbfl. PooL uses the release and utilization mechanism of pheromones to control large-scale agent coordination. PooL can capture effective information and achieve higher rewards than other state-of-arts methods with lower communication costs.
  arXiv  Detail & Related papers  (2022-02-20T03:09:53Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• What is Going on Inside Recurrent Meta Reinforcement Learning Agents? [63.58053355357644]
  Recurrent meta reinforcement learning (meta-RL) agents are agents that employ a recurrent neural network (RNN) for the purpose of "learning a learning algorithm" We shed light on the internal working mechanisms of these agents by reformulating the meta-RL problem using the Partially Observable Markov Decision Process (POMDP) framework.
  arXiv  Detail & Related papers  (2021-04-29T20:34:39Z)
• UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers [108.92194081987967]
  We make the first attempt to explore a universal multi-agent reinforcement learning pipeline, designing one single architecture to fit tasks. Unlike previous RNN-based models, we utilize a transformer-based model to generate a flexible policy. The proposed model, named as Universal Policy Decoupling Transformer (UPDeT), further relaxes the action restriction and makes the multi-agent task's decision process more explainable.
  arXiv  Detail & Related papers  (2021-01-20T07:24:24Z)
• A Visual Communication Map for Multi-Agent Deep Reinforcement Learning [7.003240657279981]
  Multi-agent learning poses significant challenges in the effort to allocate a concealed communication medium. Recent studies typically combine a specialized neural network with reinforcement learning to enable communication between agents. This paper proposes a more scalable approach that not only deals with a great number of agents but also enables collaboration between dissimilar functional agents.
  arXiv  Detail & Related papers  (2020-02-27T02:38:21Z)
• Multi-Agent Interactions Modeling with Correlated Policies [53.38338964628494]
  In this paper, we cast the multi-agent interactions modeling problem into a multi-agent imitation learning framework. We develop a Decentralized Adrial Imitation Learning algorithm with Correlated policies (CoDAIL) Various experiments demonstrate that CoDAIL can better regenerate complex interactions close to the demonstrators.
  arXiv  Detail & Related papers  (2020-01-04T17:31:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.