Enhancing Multi-Agent Systems via Reinforcement Learning with LLM-based Planner and Graph-based Policy

• URL: http://arxiv.org/abs/2503.10049v1
• Date: Thu, 13 Mar 2025 05:02:49 GMT
• Title: Enhancing Multi-Agent Systems via Reinforcement Learning with LLM-based Planner and Graph-based Policy
• Authors: Ziqi Jia, Junjie Li, Xiaoyang Qu, Jianzong Wang,
• Abstract summary: Graph Collaboration MARL (LGC-MARL) is a framework that efficiently combines Large Language Models (LLMs) and Multi-Agent Reinforcement Learning (MARL)<n>LGC-MARL decomposes complex tasks into executable subtasks and achieves efficient collaboration among multiple agents through graph-based coordination.<n> Experimental results on the AI2-THOR simulation platform demonstrate the superior performance and scalability of LGC-MARL.
• Score: 31.041340552853004
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-agent systems (MAS) have shown great potential in executing complex tasks, but coordination and safety remain significant challenges. Multi-Agent Reinforcement Learning (MARL) offers a promising framework for agent collaboration, but it faces difficulties in handling complex tasks and designing reward functions. The introduction of Large Language Models (LLMs) has brought stronger reasoning and cognitive abilities to MAS, but existing LLM-based systems struggle to respond quickly and accurately in dynamic environments. To address these challenges, we propose LLM-based Graph Collaboration MARL (LGC-MARL), a framework that efficiently combines LLMs and MARL. This framework decomposes complex tasks into executable subtasks and achieves efficient collaboration among multiple agents through graph-based coordination. Specifically, LGC-MARL consists of two main components: an LLM planner and a graph-based collaboration meta policy. The LLM planner transforms complex task instructions into a series of executable subtasks, evaluates the rationality of these subtasks using a critic model, and generates an action dependency graph. The graph-based collaboration meta policy facilitates communication and collaboration among agents based on the action dependency graph, and adapts to new task environments through meta-learning. Experimental results on the AI2-THOR simulation platform demonstrate the superior performance and scalability of LGC-MARL in completing various complex tasks.

Related papers

• Scaling Autonomous Agents via Automatic Reward Modeling And Planning [52.39395405893965]
  Large language models (LLMs) have demonstrated remarkable capabilities across a range of tasks. However, they still struggle with problems requiring multi-step decision-making and environmental feedback. We propose a framework that can automatically learn a reward model from the environment without human annotations.
  arXiv  Detail & Related papers  (2025-02-17T18:49:25Z)
• When One LLM Drools, Multi-LLM Collaboration Rules [98.71562711695991]
  We argue for multi-LLM collaboration to better represent the extensive diversity of data, skills, and people.<n>We organize existing multi-LLM collaboration methods into a hierarchy, based on the level of access and information exchange.<n>We envision multi-LLM collaboration as an essential path toward compositional intelligence and collaborative AI development.
  arXiv  Detail & Related papers  (2025-02-06T21:13:44Z)
• Interactive and Expressive Code-Augmented Planning with Large Language Models [62.799579304821826]
  Large Language Models (LLMs) demonstrate strong abilities in common-sense reasoning and interactive decision-making. Recent techniques have sought to structure LLM outputs using control flow and other code-adjacent techniques to improve planning performance. We propose REPL-Plan, an LLM planning approach that is fully code-expressive and dynamic.
  arXiv  Detail & Related papers  (2024-11-21T04:23:17Z)
• MaCTG: Multi-Agent Collaborative Thought Graph for Automatic Programming [10.461509044478278]
  MaCTG (MultiAgent Collaborative Thought Graph) is a novel multi-agent framework that employs a dynamic graph structure. It autonomously assigns agent roles based on programming requirements, dynamically refines task distribution, and systematically verifies and integrates project-level code. MaCTG significantly reduced operational costs by 89.09% compared to existing multi-agent frameworks.
  arXiv  Detail & Related papers  (2024-10-25T01:52:15Z)
• Optimizing Collaboration of LLM based Agents for Finite Element Analysis [1.5039745292757671]
  This paper investigates the interactions between multiple agents within Large Language Models (LLMs) in the context of programming and coding tasks. We utilize the AutoGen framework to facilitate communication among agents, evaluating different configurations based on the success rates from 40 random runs for each setup.
  arXiv  Detail & Related papers  (2024-08-23T23:11:08Z)
• Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration [70.09561665520043]
  We propose a novel framework for multi-agent collaboration that introduces Reinforced Advantage feedback (ReAd) for efficient self-refinement of plans. We provide theoretical analysis by extending advantage-weighted regression in reinforcement learning to multi-agent systems. Experiments on Over-AI and a difficult variant of RoCoBench show that ReAd surpasses baselines in success rate, and also significantly decreases the interaction steps of agents.
  arXiv  Detail & Related papers  (2024-05-23T08:33:19Z)
• Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Model [15.558269067931374]
  We propose STEVE-2, a hierarchical knowledge distillation framework for open-ended embodied tasks. After distillation, embodied agents can complete complex, open-ended tasks without additional expert guidance.
  arXiv  Detail & Related papers  (2024-04-06T12:51:00Z)
• Small LLMs Are Weak Tool Learners: A Multi-LLM Agent [73.54562551341454]
  Large Language Model (LLM) agents significantly extend the capabilities of standalone LLMs. We propose a novel approach that decomposes the aforementioned capabilities into a planner, caller, and summarizer. This modular framework facilitates individual updates and the potential use of smaller LLMs for building each capability.
  arXiv  Detail & Related papers  (2024-01-14T16:17:07Z)
• ADaPT: As-Needed Decomposition and Planning with Language Models [131.063805299796]
  We introduce As-Needed Decomposition and Planning for complex Tasks (ADaPT) ADaPT explicitly plans and decomposes complex sub-tasks as-needed, when the Large Language Models is unable to execute them. Our results demonstrate that ADaPT substantially outperforms established strong baselines.
  arXiv  Detail & Related papers  (2023-11-08T17:59:15Z)
• Improving Planning with Large Language Models: A Modular Agentic Architecture [7.63815864256878]
  Large language models (LLMs) often struggle with tasks that require multi-step reasoning or goal-directed planning. We propose an agentic architecture, the Modular Agentic Planner (MAP), in which planning is accomplished via the recurrent interaction of specialized modules. We find that MAP yields significant improvements over both standard LLM methods.
  arXiv  Detail & Related papers  (2023-09-30T00:10:14Z)
• Self-collaboration Code Generation via ChatGPT [35.88318116340547]
  Large Language Models (LLMs) have demonstrated remarkable code-generation ability, but struggle with complex tasks. We present a self-collaboration framework for code generation employing LLMs, exemplified by ChatGPT. To effectively organize and manage this virtual team, we incorporate software-development methodology into the framework.
  arXiv  Detail & Related papers  (2023-04-15T16:33:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.