Recursive Agent Optimization

• URL: http://arxiv.org/abs/2605.06639v1
• Date: Thu, 07 May 2026 17:49:09 GMT
• Title: Recursive Agent Optimization
• Abstract summary: Recursive agents can spawn and delegate sub-tasks to new instantiations of themselves.<n>Recursive agents implement an inference-time scaling algorithm that naturally allows agents to scale to longer contexts.
• Score: 57.77922299531435
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Recursive Agent Optimization (RAO), a reinforcement learning approach for training recursive agents: agents that can spawn and delegate sub-tasks to new instantiations of themselves recursively. Recursive agents implement an inference-time scaling algorithm that naturally allows agents to scale to longer contexts and generalize to more difficult problems via divide-and-conquer. RAO provides a method to train models to best take advantage of such recursive inference, teaching agents when and how to delegate and communicate. We find that recursive agents trained in this way enjoy better training efficiency, can scale to tasks that go beyond the model's context window, generalize to tasks much harder than the ones the agent was trained on, and can enjoy reduced wall-clock time compared to single-agent systems.

Related papers

• Learning Agent-Compatible Context Management for Long-Horizon Tasks [79.09330817637205]
  We introduce Adaptive Context Management (AdaCoM), which trains an external LLM to manage the context of a frozen agent.<n>AdaCoM substantially improves performance by preserving task constraints and progress while pruning stale content.<n>The learned strategies reveal a Fidelity-Reliability Trade-off: agents with higher vanilla ReAct performance benefit from higher-fidelity context preservation, whereas lower-performing agents require more aggressive compression to stay within a reliable reasoning regime.
  arXiv  Detail & Related papers  (2026-05-29T03:21:08Z)
• Learning to Retrieve from Agent Trajectories [72.8923565916533]
  We argue that retrieval models for agentic search should be trained directly from agent interaction data.<n>We introduce learning to retrieve from agent trajectories as a new training paradigm, where supervision is derived from multi-step agent interactions.<n>Our results highlight agent trajectories as a practical and scalable supervision source, pointing to a promising direction for retrieval in the era of agentic search.
  arXiv  Detail & Related papers  (2026-03-30T17:59:02Z)
• TIR-Agent: Training an Explorative and Efficient Agent for Image Restoration [25.116475029653884]
  TIR-Agent is a trainable image restoration agent that performs a direct tool-calling policy through a two-stage training pipeline.<n>Two key designs underpin effective RL training: (i) a random perturbation strategy applied to the SFT data, which broadens the policy's exploration over task schedules and tool compositions, and (ii) a multi-dimensional adaptive reward mechanism that dynamically re-weights heterogeneous image quality metrics to reward hacking.
  arXiv  Detail & Related papers  (2026-03-29T15:50:36Z)
• Training-Free Agentic AI: Probabilistic Control and Coordination in Multi-Agent LLM Systems [6.036652381757588]
  We introduce REDEREF, a lightweight and training-free controller for multi-agent LLM collaboration.<n>We show that belief-guided routing reduces token usage by 28%, agent calls by 17%, and time-to-success by 19%.<n>Results demonstrate that simple, interpretable probabilistic control can meaningfully improve the efficiency and robustness of multi-agent LLM systems without training or fine-tuning.
  arXiv  Detail & Related papers  (2026-02-24T21:39:14Z)
• Empowering RepoQA-Agent based on Reinforcement Learning Driven by Monte-carlo Tree Search [70.63903518295785]
  We introduce RepoSearch-R1, a novel agentic reinforcement learning framework driven by Monte-carlo Tree Search.<n>Based on RepoSearch-R1, we construct a RepoQA-Agent specifically designed for repository question-answering tasks.
  arXiv  Detail & Related papers  (2025-10-30T09:10:36Z)
• From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning [62.54484062185869]
  We introduce StepAgent, which utilizes step-wise reward to optimize the agent's reinforcement learning process.<n>We propose implicit-reward and inverse reinforcement learning techniques to facilitate agent reflection and policy adjustment.
  arXiv  Detail & Related papers  (2024-11-06T10:35:11Z)
• Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement [112.04307762405669]
  G"odel Agent is a self-evolving framework inspired by the G"odel machine.<n>G"odel Agent can achieve continuous self-improvement, surpassing manually crafted agents in performance, efficiency, and generalizability.
  arXiv  Detail & Related papers  (2024-10-06T10:49:40Z)
• Reducing Redundant Computation in Multi-Agent Coordination through Locally Centralized Execution [1.260132853894322]
  This study introduces a novel method referred to as locally centralized team transformer (LCTT) LCTT establishes a locally centralized execution framework where selected agents serve as leaders, issuing instructions, while the rest agents, designated as workers, act as these instructions without activating their policy networks. Our experimental results demonstrate that the proposed method effectively reduces redundant computation, does not decrease reward levels, and leads to faster learning convergence.
  arXiv  Detail & Related papers  (2024-04-19T06:13:37Z)
• Recursive Reasoning Graph for Multi-Agent Reinforcement Learning [44.890087638530524]
  Multi-agent reinforcement learning (MARL) provides an efficient way for simultaneously learning policies for multiple agents interacting with each other. Existing algorithms can suffer from an inability to accurately anticipate the influence of self-actions on other agents. The proposed algorithm, referred to as the Recursive Reasoning Graph (R2G), shows state-of-the-art performance on multiple multi-agent particle and robotics games.
  arXiv  Detail & Related papers  (2022-03-06T00:57:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.