Reducing Redundant Computation in Multi-Agent Coordination through Locally Centralized Execution

• URL: http://arxiv.org/abs/2404.13096v1
• Date: Fri, 19 Apr 2024 06:13:37 GMT
• Title: Reducing Redundant Computation in Multi-Agent Coordination through Locally Centralized Execution
• Abstract summary: This study introduces a novel method referred to as locally centralized team transformer (LCTT) LCTT establishes a locally centralized execution framework where selected agents serve as leaders, issuing instructions, while the rest agents, designated as workers, act as these instructions without activating their policy networks. Our experimental results demonstrate that the proposed method effectively reduces redundant computation, does not decrease reward levels, and leads to faster learning convergence.
• Score: 1.260132853894322
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In multi-agent reinforcement learning, decentralized execution is a common approach, yet it suffers from the redundant computation problem. This occurs when multiple agents redundantly perform the same or similar computation due to overlapping observations. To address this issue, this study introduces a novel method referred to as locally centralized team transformer (LCTT). LCTT establishes a locally centralized execution framework where selected agents serve as leaders, issuing instructions, while the rest agents, designated as workers, act as these instructions without activating their policy networks. For LCTT, we proposed the team-transformer (T-Trans) architecture that allows leaders to provide specific instructions to each worker, and the leadership shift mechanism that allows agents autonomously decide their roles as leaders or workers. Our experimental results demonstrate that the proposed method effectively reduces redundant computation, does not decrease reward levels, and leads to faster learning convergence.

Related papers

• Communication-Efficient Digital-Twin Coordination for Heterogeneous LLM Embodied Agents over Computing Power Networks [48.90302419915548]
  Embodied agent teams powered by heterogeneous large language models (LLMs) are being widely deployed in smart factories, warehouses, and service robotics.<n>We propose LDT-Coord, a networked coordination framework built upon a lightweight digital twin (DT)<n>Specifically, each agent independently selects its intended action and reports both the action decision and a structured temporal constraint over shared resources to the DT server.
  arXiv  Detail & Related papers  (2026-07-10T12:10:31Z)
• Recursive Agent Optimization [57.77922299531435]
  Recursive agents can spawn and delegate sub-tasks to new instantiations of themselves.<n>Recursive agents implement an inference-time scaling algorithm that naturally allows agents to scale to longer contexts.
  arXiv  Detail & Related papers  (2026-05-07T17:49:09Z)
• Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs [52.26652574704317]
  Large language models (LLMs) are increasingly deployed in teams, yet existing coordination approaches often occupy two extremes.<n>We introduce Language Agent Teams for Task Evolution (LATTE), a framework for coordinating LLM teams inspired by distributed systems.
  arXiv  Detail & Related papers  (2026-05-07T14:19:17Z)
• Multi-agent In-context Coordination via Decentralized Memory Retrieval [39.106914463842685]
  Large transformer models, trained on diverse datasets, have demonstrated impressive few-shot performance on previously unseen tasks.<n>In cooperative Multi-Agent Reinforcement Learning (MARL), where agents must coordinate toward a shared goal, decentralized policy deployment can lead to mismatches in task alignment and reward assignment.<n>We introduce Multi-agent In-context Coordination via Decentralized Memory Retrieval (MAICC), a novel approach designed to enhance coordination by fast adaptation.
  arXiv  Detail & Related papers  (2025-11-13T07:08:31Z)
• Multi-Agent Collaboration via Evolving Orchestration [61.93162413517026]
  Large language models (LLMs) have achieved remarkable results across diverse downstream tasks, but their monolithic nature restricts scalability and efficiency in complex problem-solving.<n>We propose a puppeteer-style paradigm for LLM-based multi-agent collaboration, where a central orchestrator dynamically directs agents in response to evolving task states.<n> Experiments on closed- and open-domain scenarios show that this method achieves superior performance with reduced computational costs.
  arXiv  Detail & Related papers  (2025-05-26T07:02:17Z)
• Fully-Decentralized MADDPG with Networked Agents [0.5266869303483376]
  We adapt the MADDPG algorithm by applying a networked communication approach between agents. We introduce surrogate policies in order to decentralize the training while allowing for local communication during training. The decentralized algorithms achieve comparable results to the original MADDPG in empirical tests, while reducing computational cost.
  arXiv  Detail & Related papers  (2025-03-09T20:05:32Z)
• Hierarchical Reinforcement Learning for Optimal Agent Grouping in Cooperative Systems [0.4759142872591625]
  This paper presents a hierarchical reinforcement learning (RL) approach to address the agent grouping or pairing problem in cooperative multi-agent systems. By employing a hierarchical RL framework, we distinguish between high-level decisions of grouping and low-level agents' actions. We incorporate permutation-in neural networks to handle the homogeneity and cooperation among agents, enabling effective coordination.
  arXiv  Detail & Related papers  (2025-01-11T14:22:10Z)
• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [94.2860766709971]
  We address the challenge of sampling and remote estimation for autoregressive Markovian processes in a wireless network with statistically-identical agents. Our goal is to minimize time-average estimation error and/or age of information with decentralized scalable sampling and transmission policies.
  arXiv  Detail & Related papers  (2024-04-04T06:24:11Z)
• Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL? [27.037348104661497]
  Training with Decentralized Execution is a popular framework for cooperative Multi-Agent Reinforcement Learning. We introduce a novel Advising and Decentralized Pruning (CADP) framework for multi-agent reinforcement learning.
  arXiv  Detail & Related papers  (2023-05-27T03:15:24Z)
• The challenge of redundancy on multi-agent value factorisation [12.63182277116319]
  In the field of cooperative multi-agent reinforcement learning (MARL), the standard paradigm is the use of centralised training and decentralised execution. We propose leveraging layerwise relevance propagation (LRP) to instead separate the learning of the joint value function and generation of local reward signals. We find that although the performance of both baselines VDN and Qmix degrades with the number of redundant agents, RDN is unaffected.
  arXiv  Detail & Related papers  (2023-03-28T20:41:12Z)
• Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet [48.96004919910818]
  We propose a multi-agent deep reinforcement learning (MADRL) based power control scheme for the HetNet. To promote cooperation among agents, we develop a penalty-based Q learning (PQL) algorithm for MADRL systems. In this way, an agent's policy can be learned by other agents more easily, resulting in a more efficient collaboration process.
  arXiv  Detail & Related papers  (2022-12-15T17:01:56Z)
• Explaining Reinforcement Learning Policies through Counterfactual Trajectories [147.7246109100945]
  A human developer must validate that an RL agent will perform well at test-time. Our method conveys how the agent performs under distribution shifts by showing the agent's behavior across a wider trajectory distribution. In a user study, we demonstrate that our method enables users to score better than baseline methods on one of two agent validation tasks.
  arXiv  Detail & Related papers  (2022-01-29T00:52:37Z)
• Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks [82.02891936174221]
  Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach. In this paper, a novel semantic-aware CDRL method is proposed to enable a group of untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network.
  arXiv  Detail & Related papers  (2021-11-23T18:24:47Z)
• DSDF: An approach to handle stochastic agents in collaborative multi-agent reinforcement learning [0.0]
  We show how thisity of agents, which could be a result of malfunction or aging of robots, can add to the uncertainty in coordination. Our solution, DSDF which tunes the discounted factor for the agents according to uncertainty and use the values to update the utility networks of individual agents.
  arXiv  Detail & Related papers  (2021-09-14T12:02:28Z)
• Mean-Field Multi-Agent Reinforcement Learning: A Decentralized Network Approach [6.802025156985356]
  This paper proposes a framework called localized training and decentralized execution to study MARL with network of states. The key idea is to utilize the homogeneity of agents and regroup them according to their states, thus the formulation of a networked Markov decision process.
  arXiv  Detail & Related papers  (2021-08-05T16:52:36Z)
• What is Going on Inside Recurrent Meta Reinforcement Learning Agents? [63.58053355357644]
  Recurrent meta reinforcement learning (meta-RL) agents are agents that employ a recurrent neural network (RNN) for the purpose of "learning a learning algorithm" We shed light on the internal working mechanisms of these agents by reformulating the meta-RL problem using the Partially Observable Markov Decision Process (POMDP) framework.
  arXiv  Detail & Related papers  (2021-04-29T20:34:39Z)
• Reward Machines for Cooperative Multi-Agent Reinforcement Learning [30.84689303706561]
  In cooperative multi-agent reinforcement learning, a collection of agents learns to interact in a shared environment to achieve a common goal. We propose the use of reward machines (RM) -- Mealy machines used as structured representations of reward functions -- to encode the team's task. The proposed novel interpretation of RMs in the multi-agent setting explicitly encodes required teammate interdependencies, allowing the team-level task to be decomposed into sub-tasks for individual agents.
  arXiv  Detail & Related papers  (2020-07-03T23:08:14Z)
• F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning [110.35516334788687]
  Decentralized multi-agent reinforcement learning algorithms are sometimes unpractical in complicated applications. We propose a flexible fully decentralized actor-critic MARL framework, which can handle large-scale general cooperative multi-agent setting. Our framework can achieve scalability and stability for large-scale environment and reduce information transmission.
  arXiv  Detail & Related papers  (2020-04-17T14:56:29Z)
• Counterfactual Multi-Agent Policy Gradients [47.45255170608965]
  We propose a new multi-agent actor-critic method called counterfactual multi-agent (COMA) policy gradients. COMA uses a centralised critic to estimate the Q-function and decentralised actors to optimise the agents' policies. We evaluate COMA in the testbed of StarCraft unit micromanagement, using a decentralised variant with significant partial observability.
  arXiv  Detail & Related papers  (2017-05-24T18:52:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.