Consensus Learning for Cooperative Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2206.02583v1
• Date: Mon, 6 Jun 2022 12:43:07 GMT
• Title: Consensus Learning for Cooperative Multi-Agent Reinforcement Learning
• Authors: Zhiwei Xu, Bin Zhang, Dapeng Li, Zeren Zhang, Guangchong Zhou, Guoliang Fan
• Abstract summary: We propose consensus learning for cooperative multi-agent reinforcement learning. We feed the inferred consensus as an explicit input to the network of agents. Our proposed method can be extended to various multi-agent reinforcement learning algorithms.
• Score: 12.74348597962689
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Almost all multi-agent reinforcement learning algorithms without communication follow the principle of centralized training with decentralized execution. During centralized training, agents can be guided by the same signals, such as the global state. During decentralized execution, however, agents lack the shared signal. Inspired by viewpoint invariance and contrastive learning, we propose consensus learning for cooperative multi-agent reinforcement learning in this paper. Although based on local observations, different agents can infer the same consensus in discrete space. During decentralized execution, we feed the inferred consensus as an explicit input to the network of agents, thereby developing their spirit of cooperation. Our proposed method can be extended to various multi-agent reinforcement learning algorithms. Moreover, we carry out them on some fully cooperative tasks and get convincing results.

Related papers

• Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learning [57.652899266553035]
  Decentralized and lifelong-adaptive multi-agent collaborative learning aims to enhance collaboration among multiple agents without a central server. We propose DeLAMA, a decentralized multi-agent lifelong collaborative learning algorithm with dynamic collaboration graphs.
  arXiv  Detail & Related papers  (2024-03-11T09:21:11Z)
• Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Survey [48.77342627610471]
  Cooperative multi-agent reinforcement learning is a powerful tool to solve many real-world cooperative tasks. It is challenging to derive algorithms that can converge to the optimal joint policy in a fully decentralized setting.
  arXiv  Detail & Related papers  (2024-01-10T05:07:42Z)
• Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL? [27.037348104661497]
  Training with Decentralized Execution is a popular framework for cooperative Multi-Agent Reinforcement Learning. We introduce a novel Advising and Decentralized Pruning (CADP) framework for multi-agent reinforcement learning.
  arXiv  Detail & Related papers  (2023-05-27T03:15:24Z)
• Dealing With Non-stationarity in Decentralized Cooperative Multi-Agent Deep Reinforcement Learning via Multi-Timescale Learning [15.935860288840466]
  Decentralized cooperative deep reinforcement learning (MARL) can be a versatile learning framework. One of the critical challenges in decentralized deep MARL is the non-stationarity of the learning environment when multiple agents are learning concurrently. We propose a decentralized cooperative MARL algorithm based on multi-timescale learning.
  arXiv  Detail & Related papers  (2023-02-06T14:10:53Z)
• ELIGN: Expectation Alignment as a Multi-Agent Intrinsic Reward [29.737986509769808]
  We propose a self-supervised intrinsic reward ELIGN - expectation alignment. Similar to how animals collaborate in a decentralized manner with those in their vicinity, agents trained with expectation alignment learn behaviors that match their neighbors' expectations. We show that agent coordination improves through expectation alignment because agents learn to divide tasks amongst themselves, break coordination symmetries, and confuse adversaries.
  arXiv  Detail & Related papers  (2022-10-09T22:24:44Z)
• Locality Matters: A Scalable Value Decomposition Approach for Cooperative Multi-Agent Reinforcement Learning [52.7873574425376]
  Cooperative multi-agent reinforcement learning (MARL) faces significant scalability issues due to state and action spaces that are exponentially large in the number of agents. We propose a novel, value-based multi-agent algorithm called LOMAQ, which incorporates local rewards in the Training Decentralized Execution paradigm.
  arXiv  Detail & Related papers  (2021-09-22T10:08:15Z)
• Survey of Recent Multi-Agent Reinforcement Learning Algorithms Utilizing Centralized Training [0.7588690078299698]
  We discuss variations of centralized training and describe a recent survey of algorithmic approaches. The goal is to explore how different implementations of information sharing mechanism in centralized learning may give rise to distinct group coordinated behaviors.
  arXiv  Detail & Related papers  (2021-07-29T20:29:12Z)
• Secure Distributed Training at Scale [65.7538150168154]
  Training in presence of peers requires specialized distributed training algorithms with Byzantine tolerance. We propose a novel protocol for secure (Byzantine-tolerant) decentralized training that emphasizes communication efficiency.
  arXiv  Detail & Related papers  (2021-06-21T17:00:42Z)
• HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messaging [14.960795846548029]
  Cooperative multi-agent reinforcement learning (MARL) has achieved significant results, most notably by leveraging the representation learning abilities of deep neural networks. This paper considers the case where there is a single, powerful, central agent that can observe the entire observation space, and there are multiple, low powered, local agents that can only receive local observations and cannot communicate with each other. The job of the central agent is to learn what message to send to different local agents, based on the global observations, but by determining what additional information an individual agent should receive so that it can make a better decision.
  arXiv  Detail & Related papers  (2021-01-18T19:00:12Z)
• F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning [110.35516334788687]
  Decentralized multi-agent reinforcement learning algorithms are sometimes unpractical in complicated applications. We propose a flexible fully decentralized actor-critic MARL framework, which can handle large-scale general cooperative multi-agent setting. Our framework can achieve scalability and stability for large-scale environment and reduce information transmission.
  arXiv  Detail & Related papers  (2020-04-17T14:56:29Z)
• Too many cooks: Bayesian inference for coordinating multi-agent collaboration [55.330547895131986]
  Collaboration requires agents to coordinate their behavior on the fly. Underlying the human ability to collaborate is theory-of-mind, the ability to infer the hidden mental states that drive others to act. We develop Bayesian Delegation, a decentralized multi-agent learning mechanism with these abilities.
  arXiv  Detail & Related papers  (2020-03-26T07:43:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.