Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learning

• URL: http://arxiv.org/abs/2606.24428v1
• Date: Tue, 23 Jun 2026 11:05:05 GMT
• Title: Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learning
• Abstract summary: Experience-driven self-evolution is critical for large language model (LLM) agents to improve through open-world interaction.<n>Existing experience learning methods mostly rely on single-agent loops, where the same agent executes tasks, summarizes outcomes, and determines memory content.<n>We propose EDV, an Execute-Distill-Verify framework for reliable experience learning.
• Score: 20.96920050192708
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Experience-driven self-evolution is critical for large language model (LLM) agents to improve through open-world interaction. However, existing experience learning methods mostly rely on single-agent loops, where the same agent executes tasks, summarizes outcomes, and determines memory content. This setup makes agents vulnerable to the Self-Confirmation Trap: wrong-but-self-consistent trajectories are misidentified as successful experience, leading to cumulative errors during retrieval and reuse. To address this issue, we propose EDV, an Execute-Distill-Verify framework for reliable experience learning. In the Execute stage, multiple heterogeneous agents explore the same task space in parallel to generate diverse candidate trajectories. In the Distill stage, a dedicated third-party agent comparatively analyzes these trajectories to produce candidate experiences, reducing executor-centric summarization bias. In the Verify stage, the execution group validates candidates via a consensus mechanism, and only approved experiences are written into shared or private memory. By decoupling the three stages, EDV transforms experience learning from isolated self-reflection into collaborative construction, filtering erroneous and noisy content before memory insertion. We evaluate EDV on three challenging long-horizon benchmarks: tau2-bench, Mind2Web and MMTB. Results show EDV consistently outperforms strong baselines, validating that reliable experience construction is essential for robust agent self-evolution. Our code is available at https://github.com/shidingz/EDV.

Related papers

• HarnessEvolve: Learning from Reference Trajectories for Reliable Agent Self-Evolution [11.431479238875712]
  We introduce HarnessEvolve, a self-evolving framework that learns from reference trajectories to achieve reliable agent self-evolution.<n>We conduct extensive experiments on several benchmarks spanning open-domain and enterprise scenarios, using different models and agent frameworks.<n>Results demonstrate that HarnessEvolve consistently outperforms state-of-the-art baselines across all benchmarks and settings.
  arXiv  Detail & Related papers  (2026-09-01T07:31:18Z)
• S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement? [66.17176838165994]
  Large language models (LLMs) increasingly interact with external environments and accumulate substantial behavioral experience.<n>It remains unclear whether an agent can actively test its behavior, judge the resulting experience, and use that experience to improve future decisions.<n>We introduce textbfStextsuperscript3Gym, an interactive benchmark for evaluating LLM self-improvement.
  arXiv  Detail & Related papers  (2026-08-31T17:05:41Z)
• Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents [67.97365535829098]
  Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.<n>We propose Evo-Harness, which distills noisy, single-shot executions into reusable skill harnesses for cross-domain and topic-level adaptation.<n>Our analysis demonstrates the effectiveness of Evo-Harness and provides a principled understanding of how LLM agents can effectively learn on the fly.
  arXiv  Detail & Related papers  (2026-08-15T06:43:56Z)
• PATH-Bench: Path-Dependent Evaluation of Lifelong Agents [27.837334107109786]
  PATH-Bench is a benchmark for path-dependent evaluation of lifelong agents.<n>We propose Selective Experience Use (SEU), an agent harness that regulates how path-accumulated experience influences each new task.
  arXiv  Detail & Related papers  (2026-08-02T11:00:04Z)
• AEVAL: From Anecdotal to Deterministic Testing for Agentic Skill Workflows [1.6785141970297952]
  AEVAL (Agentic Evaluation) is a CI-integrated framework that replaces this practice with a deterministic, reproducible test pipeline for agentic skills.<n>Every skill change triggers a test event: the skill runs against a developer declared evaluation contract inside an automated executor.<n>A key ingredient is a structural separation between executor and grader, preventing a subtle but pervasive failure mode.
  arXiv  Detail & Related papers  (2026-07-16T21:33:05Z)
• Rethinking Continual Experience Internalization for Self-Evolving LLM Agents [36.80404778289742]
  We find that principle-level experience is more durable than instance-level experience.<n>Step-wise injection significantly outperforms global injection by aligning experience with intermediate decision states.<n>Off-policy context-distillation on high-quality teacher trajectories provides a substantially more stable training signal than on-policy context-distillation.
  arXiv  Detail & Related papers  (2026-06-03T10:30:09Z)
• DELTAMEM: Incremental Experience Memory for LLM Agents via Residual Trees [15.040112664020503]
  We introduce residual experience, positing that newly acquired experience is often an incremental variation of existing knowledge.<n>We propose DeltaMem, a framework that organizes experience memory into two independent residual trees.<n>Experiments across diverse interactive environments show that DeltaMem consistently outperforms existing baselines.
  arXiv  Detail & Related papers  (2026-06-02T03:13:50Z)
• EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning [39.60899845728349]
  Multimodal Large Language Model (MLLM)-driven image restoration agent demonstrates effectiveness in degradation coupling scenarios.<n>Training-based methods embed intrinsic experience into parameters, achieving high inference efficiency but lacking compatibility with new tools or degradation.<n>We propose EvoIR-Agent, which first systematically formulates the experience components of a training-free image restoration agent.
  arXiv  Detail & Related papers  (2026-05-21T09:14:25Z)
• Rethinking Experience Utilization in Self-Evolving Language Model Agents [51.10420305280499]
  Self-evolving agents improve by accumulating and reusing experience from past interactions.<n>This paper studies experience utilization as a critical design dimension of self-evolving agents.
  arXiv  Detail & Related papers  (2026-05-08T02:48:49Z)
• Steve-Evolving: Open-World Embodied Self-Evolution via Fine-Grained Diagnosis and Dual-Track Knowledge Distillation [8.148658798211313]
  We present a non-parametric self-evolving framework that tightly couples fine-grained execution diagnosis with dual-track knowledge distillation in a closed loop.<n>The method follows three phases: Experience Anchoring, Experience Distillation, and KnowledgeDriven ClosedLoopLoop.<n> Experiments on the long-horizon suite of Minecraft MCU demonstrate consistent trajectories over static-retrievals.
  arXiv  Detail & Related papers  (2026-03-13T16:23:34Z)
• Self-Consolidation for Self-Evolving Agents [51.94826934403236]
  Large language model (LLM) agents operate as static systems, lacking the ability to evolve through lifelong interaction.<n>We propose a novel self-evolving framework for LLM agents that introduces a complementary evolution mechanism.
  arXiv  Detail & Related papers  (2026-02-02T11:16:07Z)
• Large Language Model Agents Are Not Always Faithful Self-Evolvers [84.08646612111092]
  Self-evolving large language model (LLM) agents continually improve by accumulating and reusing past experience.<n>We present the first systematic investigation of experience faithfulness, the causal dependence of an agent's decisions on the experience it is given.
  arXiv  Detail & Related papers  (2026-01-30T01:05:15Z)
• Propose, Solve, Verify: Self-Play Through Formal Verification [75.44204610186587]
  We study self-play in the verified code generation setting, where formal verification provides reliable correctness signals.<n>We introduce Propose, Solve, Verify (PSV) a simple self-play framework where formal verification signals are used to create a proposer capable of generating challenging synthetic problems and a solver trained via expert iteration.<n>We show that performance scales with the number of generated questions and training iterations, and through ablations identify formal verification and difficulty-aware proposal as essential ingredients for successful self-play.
  arXiv  Detail & Related papers  (2025-12-20T00:56:35Z)
• A Tale of Two Experts: Cooperative Learning for Source-Free Unsupervised Domain Adaptation [59.88864205383671]
  Source-Free Unsupervised Domain Adaptation (SFUDA) addresses the realistic challenge of adapting a source-trained model to a target domain without access to the source data.<n>Existing SFUDA methods either exploit only the source model's predictions or fine-tune large multimodal models.<n>We propose the Experts Cooperative Learning (EXCL) to exploit complementary insights and the latent structure of target data.
  arXiv  Detail & Related papers  (2025-09-26T11:39:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.