Experience as a Compass: Multi-agent RAG with Evolving Orchestration and Agent Prompts

• URL: http://arxiv.org/abs/2604.00901v2
• Date: Thu, 02 Apr 2026 21:22:41 GMT
• Title: Experience as a Compass: Multi-agent RAG with Evolving Orchestration and Agent Prompts
• Abstract summary: HERA is a hierarchical framework that jointly evolves multi-agent orchestration and role-specific agent prompts.<n> HERA achieves an average improvement of 38.69% over recent baselines.<n>Topological analyses reveal emergent self-organization, where sparse exploration yields compact, high-utility multi-agent networks.
• Score: 22.323342919724492
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Multi-agent Retrieval-Augmented Generation (RAG), wherein each agent takes on a specific role, supports hard queries that require multiple steps and sources, or complex reasoning. Existing approaches, however, rely on static agent behaviors and fixed orchestration strategies, leading to brittle performance on diverse, multi-hop tasks. We identify two key limitations: the lack of continuously adaptive orchestration mechanisms and the absence of behavior-level learning for individual agents. To this end, we propose HERA, a hierarchical framework that jointly evolves multi-agent orchestration and role-specific agent prompts. At the global level, HERA optimizes query-specific agent topologies through reward-guided sampling and experience accumulation. At the local level, Role-Aware Prompt Evolution refines agent behaviors via credit assignment and dual-axes adaptation along operational and behavioral principles, enabling targeted, role-conditioned improvements. On six knowledge-intensive benchmarks, HERA achieves an average improvement of 38.69\% over recent baselines while maintaining robust generalization and token efficiency. Topological analyses reveal emergent self-organization, where sparse exploration yields compact, high-utility multi-agent networks, demonstrating both efficient coordination and robust reasoning.

Related papers

• Events as Triggers for Behavioral Diversity in Multi-Agent Reinforcement Learning [10.867011075689879]
  We argue that, to define these behavioral transitions, the missing ingredient is $textbfevents$.<n>We introduce a framework that decouples agent identity from behavior, capturing a continuous manifold from which agents instantiate their behaviors in response to events.
  arXiv  Detail & Related papers  (2026-05-12T16:51:23Z)
• MARTI-MARS$^2$: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generation [64.2621682259008]
  Multi-Agent Reinforced Training and Inference Framework with Self-Search Scaling (MARTI-MARS2)<n>We propose a Multi-Agent Reinforced Training and Inference Framework with Self-Search Scaling (MARTI-MARS2) to integrate policy learning with multi-agent tree search.<n>We show that MARTI-MARS2 achieves 77.7%, outperforming strong baselines like GPT-5.1 on challenging code generation benchmarks.
  arXiv  Detail & Related papers  (2026-02-08T07:28:44Z)
• AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answering [51.07491603393163]
  tAgent is a framework that formulates multi-agent QA as a knowledge-graph-guided routing problem supervised by empirical performance signals.<n>By leveraging soft supervision and weighted aggregation of agent outputs, Agent learns principled collaboration schemes that capture the complementary strengths of diverse agents.
  arXiv  Detail & Related papers  (2025-10-06T23:20:49Z)
• InfiAgent: Self-Evolving Pyramid Agent Framework for Infinite Scenarios [28.65914611521654]
  InfiAgent is a Pyramid-like DAG-based Multi-Agent Framework that can be applied to textbfinfinite scenarios.<n>InfiAgent achieves 9.9% higher performance compared to ADAS (similar auto-generated agent framework)
  arXiv  Detail & Related papers  (2025-09-26T15:44:09Z)
• Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems [43.960406518628865]
  Multi-agent systems solve complex tasks by dividing responsibilities among specialized agents.<n>Most existing methods typically fine-tune these agents independently, leading to capability gaps among them with poor coordination.<n>We propose MOAT, a Multi-Agent Joint Alignment Tuning framework that improves agents collaboration through iterative alignment.
  arXiv  Detail & Related papers  (2025-09-11T17:15:45Z)
• AgentOrchestra: Orchestrating Hierarchical Multi-Agent Intelligence with the Tool-Environment-Agent(TEA) Protocol [22.406849007798858]
  We propose the Tool-Environment-Agent Protocol to integrate environments, agents, and tools into an unified system.<n>We introduce AgentOrchestra, a hierarchical multi-agent framework with a central planning agent that decomposes complex objectives and coordinates specialized agents.
  arXiv  Detail & Related papers  (2025-06-14T13:45:37Z)
• Multi-Agent Collaboration via Evolving Orchestration [55.574417128944226]
  Large language models (LLMs) have achieved remarkable results across diverse downstream tasks, but their monolithic nature restricts scalability and efficiency in complex problem-solving.<n>We propose a puppeteer-style paradigm for LLM-based multi-agent collaboration, where a centralized orchestrator ("puppeteer") dynamically directs agents ("puppets") in response to evolving task states.<n> Experiments on closed- and open-domain scenarios show that this method achieves superior performance with reduced computational costs.
  arXiv  Detail & Related papers  (2025-05-26T07:02:17Z)
• MorphAgent: Empowering Agents through Self-Evolving Profiles and Decentralized Collaboration [11.01813164951313]
  This paper introduces MorphAgent, a novel Autonomous, Self-Organizing, and Self-Adaptive Multi-Agent System.<n>Our approach employs self-evolving agent profiles, optimized through three key metrics, guiding agents in refining their individual expertise.<n>Our experimental results show that MorphAgent outperforms existing frameworks in terms of task performance and adaptability to changing requirements.
  arXiv  Detail & Related papers  (2024-10-19T09:10:49Z)
• Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinement [50.481380478458945]
  Iterative step-level Process Refinement (IPR) framework provides detailed step-by-step guidance to enhance agent training. Our experiments on three complex agent tasks demonstrate that our framework outperforms a variety of strong baselines.
  arXiv  Detail & Related papers  (2024-06-17T03:29:13Z)
• DCIR: Dynamic Consistency Intrinsic Reward for Multi-Agent Reinforcement Learning [84.22561239481901]
  We propose a new approach that enables agents to learn whether their behaviors should be consistent with that of other agents. We evaluate DCIR in multiple environments including Multi-agent Particle, Google Research Football and StarCraft II Micromanagement.
  arXiv  Detail & Related papers  (2023-12-10T06:03:57Z)
• Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RL [107.58821842920393]
  We quantify the agent's behavior difference and build its relationship with the policy performance via bf Role Diversity We find that the error bound in MARL can be decomposed into three parts that have a strong relation to the role diversity. The decomposed factors can significantly impact policy optimization on three popular directions.
  arXiv  Detail & Related papers  (2022-06-01T04:58:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.