Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization

• URL: http://arxiv.org/abs/2604.20714v1
• Date: Wed, 22 Apr 2026 16:00:46 GMT
• Title: Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization
• Abstract summary: "Agent Engineering" is the process of designing and optimizing multi-agent systems.<n>TPGO is a framework that enables a multi-agent system to learn to evolve.<n>The core of our framework is Group Relative Agent Optimization (GRAO), a novel meta-learning strategy.
• Score: 18.683909290649876
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Designing and optimizing multi-agent systems (MAS) is a complex, labor-intensive process of "Agent Engineering." Existing automatic optimization methods, primarily focused on flat prompt tuning, lack the structural awareness to debug the intricate web of interactions in MAS. More critically, these optimizers are static; they do not learn from experience to improve their own optimization strategies. To address these gaps, we introduce Textual Parameter Graph Optimization (TPGO), a framework that enables a multi-agent system to learn to evolve. TPGO first models the MAS as a Textual Parameter Graph (TPG), where agents, tools, and workflows are modular, optimizable nodes. To guide evolution, we derive "textual gradients," structured natural language feedback from execution traces, to pinpoint failures and suggest granular modifications. The core of our framework is Group Relative Agent Optimization (GRAO), a novel meta-learning strategy that learns from historical optimization experiences. By analyzing past successes and failures, GRAO becomes progressively better at proposing effective updates, allowing the system to learn how to optimize itself. Extensive experiments on complex benchmarks like GAIA and MCP-Universe show that TPGO significantly enhances the performance of state-of-the-art agent frameworks, achieving higher success rates through automated, self-improving optimization.

Related papers

• AgenticRecTune: Multi-Agent with Self-Evolving Skillhub for Recommendation System Optimization [13.524716959938237]
  We propose AgenticRecTune to manage the end-to-end configuration optimization workflow.<n>The framework comprises five specialized agents, Actor, Critic, Insight, Skill, and Online.<n>We also introduce a self-evolving Skillhub, which utilizes a collaboration between the Insight Agent and Skill Agent to summarize the history results.
  arXiv  Detail & Related papers  (2026-04-21T23:15:58Z)
• Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models [69.55139736609367]
  Agent-GWO is a dynamic prompt optimization framework for complex reasoning.<n>We show that Agent-GWO consistently improves accuracy and stability over existing prompt optimization methods.
  arXiv  Detail & Related papers  (2026-04-14T07:35:37Z)
• VeriAgent: A Tool-Integrated Multi-Agent System with Evolving Memory for PPA-Aware RTL Code Generation [52.43796056267479]
  We propose a PPA-aware, tool-integrated multi-agent framework for high-quality verilog code generation.<n>Our framework explicitly incorporates EDA tools into a closed-loop workflow composed of a textitProgrammer Agent, a textitCorrectness Agent, and a textitPPA Agent<n>To support continuous improvement without model retraining, we introduce an textitEvolved Memory Mechanism that externalizes optimization experience into structured memory nodes.
  arXiv  Detail & Related papers  (2026-03-18T11:25:40Z)
• Sample-Efficient Online Learning in LM Agents via Hindsight Trajectory Rewriting [92.57796055887995]
  We introduce ECHO, a prompting framework that adapts hindsight experience replay from reinforcement learning for language model agents.<n> ECHO generates optimized trajectories for alternative goals that could have been achieved during failed attempts.<n>We evaluate ECHO on stateful versions of XMiniGrid, a text-based navigation and planning benchmark, and PeopleJoinQA, a collaborative information-gathering enterprise simulation.
  arXiv  Detail & Related papers  (2025-10-11T18:11:09Z)
• Reflection-Enhanced Meta-Optimization Integrating TextGrad-style Prompt Optimization with Memory-Driven Self-Evolution [0.0]
  We propose a framework that integrates a memory-augmented Reflection RetrievalRAG module and a Self-Adaptive meta-controller.<n>REMO achieves more stable and robust tuning, albeit at the cost of increased computational overhead.
  arXiv  Detail & Related papers  (2025-08-26T07:25:45Z)
• Language-Guided Tuning: Enhancing Numeric Optimization with Textual Feedback [1.2106694524257837]
  We introduce Language-Guided Tuning (LGT), a novel framework that employs multi-agent Large Language Models to intelligently optimize configurations through natural language reasoning.<n>We apply textual - qualitative feedback signals that complement numerical optimization by providing semantic understanding of training dynamics and configuration interdependencies.<n>LGT demonstrates substantial improvements over traditional optimization methods, achieving performance gains while maintaining high interpretability.
  arXiv  Detail & Related papers  (2025-08-21T17:55:07Z)
• ORFS-agent: Tool-Using Agents for Chip Design Optimization [0.8088986164437757]
  Large Language Models (LLMs) offer new opportunities for learning and reasoning within such high-dimensional optimization tasks.<n>We introduce ORFS-agent, an LLM-based iterative optimization agent that automates parameter tuning in an open-source hardware design flow.<n>Our empirical evaluations on two different technology nodes and a range of circuit benchmarks indicate that ORFS-agent can improve both routed wirelength and effective clock period by over 13%.
  arXiv  Detail & Related papers  (2025-06-10T01:38:57Z)
• A Survey on the Optimization of Large Language Model-based Agents [16.733092886211097]
  Large Language Models (LLMs) have been widely adopted in various fields, becoming essential for autonomous decision-making and interactive tasks.<n>However, current work typically relies on prompt design or fine-tuning strategies applied to vanilla LLMs.<n>We provide a comprehensive review of LLM-based agent optimization approaches, categorizing them into parameter-driven and parameter-free methods.
  arXiv  Detail & Related papers  (2025-03-16T10:09:10Z)
• LLM as a Complementary Optimizer to Gradient Descent: A Case Study in Prompt Tuning [69.95292905263393]
  We show that gradient-based and high-level LLMs can effectively collaborate a combined optimization framework.<n>In this paper, we show that these complementary to each other and can effectively collaborate a combined optimization framework.
  arXiv  Detail & Related papers  (2024-05-30T06:24:14Z)
• Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers [108.72225067368592]
  We propose a novel perspective to investigate the design of large language models (LLMs)-based prompts.<n>We identify two pivotal factors in model parameter learning: update direction and update method.<n>We develop a capable Gradient-inspired Prompt-based GPO.
  arXiv  Detail & Related papers  (2024-02-27T15:05:32Z)
• Optimization-Inspired Learning with Architecture Augmentations and Control Mechanisms for Low-Level Vision [74.9260745577362]
  This paper proposes a unified optimization-inspired learning framework to aggregate Generative, Discriminative, and Corrective (GDC) principles. We construct three propagative modules to effectively solve the optimization models with flexible combinations. Experiments across varied low-level vision tasks validate the efficacy and adaptability of GDC.
  arXiv  Detail & Related papers  (2020-12-10T03:24:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.