PooL: Pheromone-inspired Communication Framework forLarge Scale Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2202.09722v1
• Date: Sun, 20 Feb 2022 03:09:53 GMT
• Title: PooL: Pheromone-inspired Communication Framework forLarge Scale Multi-Agent Reinforcement Learning
• Authors: Zixuan Cao, Mengzhi Shi, Zhanbo Zhao, Xiujun Ma
• Abstract summary: textbfPooL is an indirect communication framework applied to large scale multi-agent reinforcement textbfl. PooL uses the release and utilization mechanism of pheromones to control large-scale agent coordination. PooL can capture effective information and achieve higher rewards than other state-of-arts methods with lower communication costs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Being difficult to scale poses great problems in multi-agent coordination. Multi-agent Reinforcement Learning (MARL) algorithms applied in small-scale multi-agent systems are hard to extend to large-scale ones because the latter is far more dynamic and the number of interactions increases exponentially with the growing number of agents. Some swarm intelligence algorithms simulate the release and utilization mechanism of pheromones to control large-scale agent coordination. Inspired by such algorithms, \textbf{PooL}, an \textbf{p}her\textbf{o}m\textbf{o}ne-based indirect communication framework applied to large scale multi-agent reinforcement \textbf{l}earning is proposed in order to solve the large-scale multi-agent coordination problem. Pheromones released by agents of PooL are defined as outputs of most reinforcement learning algorithms, which reflect agents' views of the current environment. The pheromone update mechanism can efficiently organize the information of all agents and simplify the complex interactions among agents into low-dimensional representations. Pheromones perceived by agents can be regarded as a summary of the views of nearby agents which can better reflect the real situation of the environment. Q-Learning is taken as our base model to implement PooL and PooL is evaluated in various large-scale cooperative environments. Experiments show agents can capture effective information through PooL and achieve higher rewards than other state-of-arts methods with lower communication costs.

Related papers

• Very Large-Scale Multi-Agent Simulation in AgentScope [112.98986800070581]
  We develop new features and components for AgentScope, a user-friendly multi-agent platform. We propose an actor-based distributed mechanism towards great scalability and high efficiency. We also provide a web-based interface for conveniently monitoring and managing a large number of agents.
  arXiv  Detail & Related papers  (2024-07-25T05:50:46Z)
• Scaling Large-Language-Model-based Multi-Agent Collaboration [75.5241464256688]
  Pioneering advancements in large language model-powered agents have underscored the design pattern of multi-agent collaboration. Inspired by the neural scaling law, this study investigates whether a similar principle applies to increasing agents in multi-agent collaboration.
  arXiv  Detail & Related papers  (2024-06-11T11:02:04Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors [93.38830440346783]
  We propose a multi-agent framework framework that can collaboratively adjust its composition as a greater-than-the-sum-of-its-parts system. Our experiments demonstrate that framework framework can effectively deploy multi-agent groups that outperform a single agent. In view of these behaviors, we discuss some possible strategies to leverage positive ones and mitigate negative ones for improving the collaborative potential of multi-agent groups.
  arXiv  Detail & Related papers  (2023-08-21T16:47:11Z)
• MADiff: Offline Multi-agent Learning with Diffusion Models [79.18130544233794]
  Diffusion model (DM) recently achieved huge success in various scenarios including offline reinforcement learning. We propose MADiff, a novel generative multi-agent learning framework to tackle this problem. Our experiments show the superior performance of MADiff compared to baseline algorithms in a wide range of multi-agent learning tasks.
  arXiv  Detail & Related papers  (2023-05-27T02:14:09Z)
• Depthwise Convolution for Multi-Agent Communication with Enhanced Mean-Field Approximation [9.854975702211165]
  We propose a new method based on local communication learning to tackle the multi-agent RL (MARL) challenge. First, we design a new communication protocol that exploits the ability of depthwise convolution to efficiently extract local relations. Second, we introduce the mean-field approximation into our method to reduce the scale of agent interactions.
  arXiv  Detail & Related papers  (2022-03-06T07:42:43Z)
• Recursive Reasoning Graph for Multi-Agent Reinforcement Learning [44.890087638530524]
  Multi-agent reinforcement learning (MARL) provides an efficient way for simultaneously learning policies for multiple agents interacting with each other. Existing algorithms can suffer from an inability to accurately anticipate the influence of self-actions on other agents. The proposed algorithm, referred to as the Recursive Reasoning Graph (R2G), shows state-of-the-art performance on multiple multi-agent particle and robotics games.
  arXiv  Detail & Related papers  (2022-03-06T00:57:50Z)
• Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module [29.75594940509839]
  We formulate a multi-agent environment with a different number of agents as a multi-tasking problem. We propose a meta reinforcement learning (meta-RL) framework to tackle this problem. The proposed framework employs a meta-learned Communication Pattern Recognition (CPR) module to identify communication behavior.
  arXiv  Detail & Related papers  (2021-12-14T08:23:04Z)
• HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messaging [14.960795846548029]
  Cooperative multi-agent reinforcement learning (MARL) has achieved significant results, most notably by leveraging the representation learning abilities of deep neural networks. This paper considers the case where there is a single, powerful, central agent that can observe the entire observation space, and there are multiple, low powered, local agents that can only receive local observations and cannot communicate with each other. The job of the central agent is to learn what message to send to different local agents, based on the global observations, but by determining what additional information an individual agent should receive so that it can make a better decision.
  arXiv  Detail & Related papers  (2021-01-18T19:00:12Z)
• Scaling Up Multiagent Reinforcement Learning for Robotic Systems: Learn an Adaptive Sparse Communication Graph [39.48317026356428]
  The complexity of multiagent reinforcement learning increases exponentially with respect to the agent number. One critical feature in MARL that is often neglected is that the interactions between agents are quite sparse. We propose an adaptive sparse attention mechanism by generalizing a sparsity-inducing activation function. We show that our algorithm can learn an interpretable sparse structure and outperforms previous works by a significant margin on applications involving a large-scale multiagent system.
  arXiv  Detail & Related papers  (2020-03-02T17:18:25Z)
• Multi-Agent Interactions Modeling with Correlated Policies [53.38338964628494]
  In this paper, we cast the multi-agent interactions modeling problem into a multi-agent imitation learning framework. We develop a Decentralized Adrial Imitation Learning algorithm with Correlated policies (CoDAIL) Various experiments demonstrate that CoDAIL can better regenerate complex interactions close to the demonstrators.
  arXiv  Detail & Related papers  (2020-01-04T17:31:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.