Tree-of-Experience: Hierarchical Experience Management for Self-Evolving Agents

• URL: http://arxiv.org/abs/2608.09044v1
• Date: Mon, 10 Aug 2026 02:49:38 GMT
• Title: Tree-of-Experience: Hierarchical Experience Management for Self-Evolving Agents
• Abstract summary: We propose textbfTree-textbfof-textbfExperience (ToE), a structured experience-management framework.<n>ToE organizes the experience into a shared tree of analytical perspectives and reasoning paths.<n>Experiments show ToE substantially improves both problem-solving performance and efficiency.
• Score: 7.400600301289333
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual self-evolution requires LLM agents to transform environmental interactions into reliable and reusable experience. Existing methods typically refine individual trajectories or abstract shared knowledge from related trajectories, but their experience representations are often disconnected from the underlying reasoning process. This limits feedback attribution, cross-task transfer, and update and retrieval efficiency, particularly in complex reasoning tasks with outcome-level feedback. To overcome this limitation, we propose \textbf{T}ree-\textbf{o}f-\textbf{E}xperience (ToE), a structured experience-management framework that aligns experience organization with the hierarchical reasoning process of LLM agents. Specifically, ToE organizes the experience into a shared tree of analytical perspectives and reasoning paths, whose reliability is calibrated through environmental outcomes to support systematic updating, transfer, and efficient retrieval. The experimental results on \textsc{Game of 24} and \textsc{FinEvolveBench} show that ToE substantially improves both problem-solving performance and efficiency. On \textsc{Game of 24}, ToE achieves a 31.4\% relative improvement in accuracy over the experience-free ToT baseline. On \textsc{FinEvolveBench}, ToE improves tsIC by an average of 41.24\% over the experience-free pipeline across 12 evaluation settings, whereas conventional experience-management methods often underperform experience-free baselines.

Related papers

• S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement? [66.17176838165994]
  Large language models (LLMs) increasingly interact with external environments and accumulate substantial behavioral experience.<n>It remains unclear whether an agent can actively test its behavior, judge the resulting experience, and use that experience to improve future decisions.<n>We introduce textbfStextsuperscript3Gym, an interactive benchmark for evaluating LLM self-improvement.
  arXiv  Detail & Related papers  (2026-08-31T17:05:41Z)
• Self-Improving Large Language Models via Progressive Experience Evolution [52.03479747358687]
  We propose textbfSPEE (textbfSelf-textbfProgressive textbfExperience textbfEvolution), a unified post-training framework that sequentially performs explicit experience evolution followed by implicit policy optimization.<n> Experiments on five mathematical reasoning benchmarks demonstrate that SPEE consistently outperforms both test-time and training-time self-evolution baselines across three model scales.
  arXiv  Detail & Related papers  (2026-08-03T12:27:32Z)
• PATH-Bench: Path-Dependent Evaluation of Lifelong Agents [27.837334107109786]
  PATH-Bench is a benchmark for path-dependent evaluation of lifelong agents.<n>We propose Selective Experience Use (SEU), an agent harness that regulates how path-accumulated experience influences each new task.
  arXiv  Detail & Related papers  (2026-08-02T11:00:04Z)
• Tree-of-Experience: A Structured Experience-Management Solution for Self-Evolving Agents under Low-Repetition and Implicit-Reward Environments [7.400600301289333]
  We study a more challenging setting: low-repetition tasks with implicit rewards, where past experience is difficult to reuse and feedback is delayed, noisy, and outcome-level.<n>We propose Tree-of-Experience (ToE), a structured experience-management method that organizes, retrieves, validates, and updates agent experience.
  arXiv  Detail & Related papers  (2026-06-05T06:39:16Z)
• EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning [39.60899845728349]
  Multimodal Large Language Model (MLLM)-driven image restoration agent demonstrates effectiveness in degradation coupling scenarios.<n>Training-based methods embed intrinsic experience into parameters, achieving high inference efficiency but lacking compatibility with new tools or degradation.<n>We propose EvoIR-Agent, which first systematically formulates the experience components of a training-free image restoration agent.
  arXiv  Detail & Related papers  (2026-05-21T09:14:25Z)
• Rethinking Experience Utilization in Self-Evolving Language Model Agents [51.10420305280499]
  Self-evolving agents improve by accumulating and reusing experience from past interactions.<n>This paper studies experience utilization as a critical design dimension of self-evolving agents.
  arXiv  Detail & Related papers  (2026-05-08T02:48:49Z)
• Decocted Experience Improves Test-Time Inference in LLM Agents [30.56835805919089]
  We show that effective context construction critically depends on emphdecocted experience<n>We identify emphdecocted experience as a key mechanism for effective context construction.<n>We validate our findings across reasoning and agentic tasks, including math reasoning, web browsing, and software engineering.
  arXiv  Detail & Related papers  (2026-04-06T02:49:32Z)
• Large Language Model Agents Are Not Always Faithful Self-Evolvers [84.08646612111092]
  Self-evolving large language model (LLM) agents continually improve by accumulating and reusing past experience.<n>We present the first systematic investigation of experience faithfulness, the causal dependence of an agent's decisions on the experience it is given.
  arXiv  Detail & Related papers  (2026-01-30T01:05:15Z)
• Toward Structured Knowledge Reasoning: Contrastive Retrieval-Augmented Generation on Experience [8.270615745804557]
  Large language models (LLMs) achieve strong performance on plain text tasks but underperform on structured data like tables and databases.<n>We introduce Contrastive Retrieval-Augmented Generation on Experience (CoRE), a framework that builds experience memory representations.<n> Experiments show CoRE significantly improves performance, achieving average gains of 3.44% and 4.24%, with up to 17.2% on challenging tasks.
  arXiv  Detail & Related papers  (2025-06-01T05:22:00Z)
• CoPS: Empowering LLM Agents with Provable Cross-Task Experience Sharing [70.25689961697523]
  We propose a generalizable algorithm that enhances sequential reasoning by cross-task experience sharing and selection. Our work bridges the gap between existing sequential reasoning paradigms and validates the effectiveness of leveraging cross-task experiences.
  arXiv  Detail & Related papers  (2024-10-22T03:59:53Z)
• Iterative Experience Refinement of Software-Developing Agents [81.09737243969758]
  Large language models (LLMs) can leverage past experiences to reduce errors and enhance efficiency. This paper introduces the Iterative Experience Refinement framework, enabling LLM agents to refine experiences iteratively during task execution.
  arXiv  Detail & Related papers  (2024-05-07T11:33:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.