Fully-Decentralized MADDPG with Networked Agents

• URL: http://arxiv.org/abs/2503.06747v1
• Date: Sun, 09 Mar 2025 20:05:32 GMT
• Title: Fully-Decentralized MADDPG with Networked Agents
• Authors: Diego Bolliger, Lorenz Zauter, Robert Ziegler,
• Abstract summary: We adapt the MADDPG algorithm by applying a networked communication approach between agents.<n>We introduce surrogate policies in order to decentralize the training while allowing for local communication during training.<n>The decentralized algorithms achieve comparable results to the original MADDPG in empirical tests, while reducing computational cost.
• Score: 0.5266869303483376
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we devise three actor-critic algorithms with decentralized training for multi-agent reinforcement learning in cooperative, adversarial, and mixed settings with continuous action spaces. To this goal, we adapt the MADDPG algorithm by applying a networked communication approach between agents. We introduce surrogate policies in order to decentralize the training while allowing for local communication during training. The decentralized algorithms achieve comparable results to the original MADDPG in empirical tests, while reducing computational cost. This is more pronounced with larger numbers of agents.

Related papers

• Distributed Value Decomposition Networks with Networked Agents [3.8779763612314633]
  We propose distributed value decomposition networks (DVDN) that generate a joint Q-function that factorizes into agent-wise Q-functions.<n>DVDN overcomes the need for centralized training by locally estimating the shared objective.<n> Empirically, both algorithms approximate the performance of value decomposition networks, in spite of the information loss during communication.
  arXiv  Detail & Related papers  (2025-02-11T15:23:05Z)
• Reducing Variance Caused by Communication in Decentralized Multi-agent Deep Reinforcement Learning [2.1461517065527445]
  We study the variance that is caused by communication in policy gradients.<n>We propose modular techniques to reduce the variance in policy gradients during training.<n>The results show that decentralized MADRL communication methods extended with our proposed techniques.
  arXiv  Detail & Related papers  (2025-02-10T08:53:13Z)
• Cluster-Based Multi-Agent Task Scheduling for Space-Air-Ground Integrated Networks [60.085771314013044]
  Low-altitude economy holds significant potential for development in areas such as communication and sensing. We propose a Clustering-based Multi-agent Deep Deterministic Policy Gradient (CMADDPG) algorithm to address the multi-UAV cooperative task scheduling challenges in SAGIN.
  arXiv  Detail & Related papers  (2024-12-14T06:17:33Z)
• Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learning [57.652899266553035]
  Decentralized and lifelong-adaptive multi-agent collaborative learning aims to enhance collaboration among multiple agents without a central server. We propose DeLAMA, a decentralized multi-agent lifelong collaborative learning algorithm with dynamic collaboration graphs.
  arXiv  Detail & Related papers  (2024-03-11T09:21:11Z)
• Consensus Learning for Cooperative Multi-Agent Reinforcement Learning [12.74348597962689]
  We propose consensus learning for cooperative multi-agent reinforcement learning. We feed the inferred consensus as an explicit input to the network of agents. Our proposed method can be extended to various multi-agent reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-06-06T12:43:07Z)
• Scalable Multi-Agent Model-Based Reinforcement Learning [1.95804735329484]
  We propose a new method called MAMBA which utilizes Model-Based Reinforcement Learning (MBRL) to further leverage centralized training in cooperative environments. We argue that communication between agents is enough to sustain a world model for each agent during execution phase while imaginary rollouts can be used for training, removing the necessity to interact with the environment.
  arXiv  Detail & Related papers  (2022-05-25T08:35:00Z)
• Emergence of Theory of Mind Collaboration in Multiagent Systems [65.97255691640561]
  We propose an adaptive training algorithm to develop effective collaboration between agents with ToM. We evaluate our algorithms with two games, where our algorithm surpasses all previous decentralized execution algorithms without modeling ToM.
  arXiv  Detail & Related papers  (2021-09-30T23:28:00Z)
• Mean-Field Multi-Agent Reinforcement Learning: A Decentralized Network Approach [6.802025156985356]
  This paper proposes a framework called localized training and decentralized execution to study MARL with network of states. The key idea is to utilize the homogeneity of agents and regroup them according to their states, thus the formulation of a networked Markov decision process.
  arXiv  Detail & Related papers  (2021-08-05T16:52:36Z)
• Adaptive Serverless Learning [114.36410688552579]
  We propose a novel adaptive decentralized training approach, which can compute the learning rate from data dynamically. Our theoretical results reveal that the proposed algorithm can achieve linear speedup with respect to the number of workers. To reduce the communication-efficient overhead, we further propose a communication-efficient adaptive decentralized training approach.
  arXiv  Detail & Related papers  (2020-08-24T13:23:02Z)
• F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning [110.35516334788687]
  Decentralized multi-agent reinforcement learning algorithms are sometimes unpractical in complicated applications. We propose a flexible fully decentralized actor-critic MARL framework, which can handle large-scale general cooperative multi-agent setting. Our framework can achieve scalability and stability for large-scale environment and reduce information transmission.
  arXiv  Detail & Related papers  (2020-04-17T14:56:29Z)
• Decentralized MCTS via Learned Teammate Models [89.24858306636816]
  We present a trainable online decentralized planning algorithm based on decentralized Monte Carlo Tree Search. We show that deep learning and convolutional neural networks can be employed to produce accurate policy approximators.
  arXiv  Detail & Related papers  (2020-03-19T13:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.