Experiential Reflective Learning for Self-Improving LLM Agents

• URL: http://arxiv.org/abs/2603.24639v2
• Date: Tue, 31 Mar 2026 19:23:46 GMT
• Title: Experiential Reflective Learning for Self-Improving LLM Agents
• Abstract summary: We introduce Experiential Reflective Learning (ERL), a simple self-improvement framework that enables rapid environment adaptation.<n>ERL reflects on task trajectories and outcomes to generate actionable lessons that transfer across tasks.<n>ERL improves success rate by 7.8% over a ReAct baseline, with large gains in task completion reliability.
• Score: 1.1074589887824053
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in large language models (LLMs) have enabled the development of autonomous agents capable of complex reasoning and multi-step problem solving. However, these agents struggle to adapt to specialized environments and do not leverage past interactions, approaching each new task from scratch regardless of their accumulated experience. We introduce Experiential Reflective Learning (ERL), a simple self-improvement framework that enables rapid environment adaptation through experiential learning. ERL reflects on task trajectories and outcomes to generate heuristics, capturing actionable lessons that transfer across tasks. At test time, relevant heuristics are retrieved based on the current task and injected into the agent's context to guide execution. On the Gaia2 benchmark, ERL improves success rate by 7.8% over a ReAct baseline, with large gains in task completion reliability, and outperforms prior experiential learning methods. Through systematic ablations, we find that selective retrieval is essential and that heuristics provide more transferable abstractions than few-shot trajectory prompting. These results demonstrate that reflecting on single-attempt experiences to extract transferable heuristics enables effective agent self-improvement.

Related papers

• Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents [67.97365535829098]
  Learning from experience is critical for developing capable, self-improving large language model (LLM) agents.<n>We propose Evo-Harness, which distills noisy, single-shot executions into reusable skill harnesses for cross-domain and topic-level adaptation.<n>Our analysis demonstrates the effectiveness of Evo-Harness and provides a principled understanding of how LLM agents can effectively learn on the fly.
  arXiv  Detail & Related papers  (2026-08-15T06:43:56Z)
• EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning [39.60899845728349]
  Multimodal Large Language Model (MLLM)-driven image restoration agent demonstrates effectiveness in degradation coupling scenarios.<n>Training-based methods embed intrinsic experience into parameters, achieving high inference efficiency but lacking compatibility with new tools or degradation.<n>We propose EvoIR-Agent, which first systematically formulates the experience components of a training-free image restoration agent.
  arXiv  Detail & Related papers  (2026-05-21T09:14:25Z)
• SEARL: Joint Optimization of Policy and Tool Graph Memory for Self-Evolving Agents [35.45207852488779]
  We introduce a Tool-Memory based self-evolving agentic framework SEARL.<n>Unlike approaches that directly utilize interaction experiences, our method constructs a structured experience memory that integrates planning with execution.<n>We evaluate our framework on knowledge reasoning and mathematics tasks, demonstrating its effectiveness in achieving more practical and efficient learning.
  arXiv  Detail & Related papers  (2026-04-09T04:38:47Z)
• Retrieval-Augmented LLM Agents: Learning to Learn from Experience [16.248836438253814]
  We study how to train retrieval-augmented LLM agents to leverage retrieved trajectories in-context.<n>We establish a robust supervised fine-tuning (SFT) recipe using LoRA that outperforms several state-of-the-art agent training pipelines.<n>Our results demonstrate that this combined approach significantly improves generalization to unseen tasks.
  arXiv  Detail & Related papers  (2026-03-18T20:45:04Z)
• Complementary Reinforcement Learning [31.660877399506493]
  Reinforcement Learning (RL) has emerged as a powerful paradigm for training LLM-based agents.<n>Existing approaches suffer from a critical weakness: the experience distilled from history is either stored statically or fail to coevolve with the improving actor.<n>Inspired by complementary learning systems in neuroscience, we present Complementary RL to achieve seamless co-evolution of an experience extractor and a policy actor.
  arXiv  Detail & Related papers  (2026-03-18T11:38:01Z)
• RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedback [54.39884046754265]
  RetroAgent is an online RL framework that enables agents to master complex interactive environments.<n>Our experiments show that RetroAgent achieves state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2026-03-09T16:23:33Z)
• AgentEvolver: Towards Efficient Self-Evolving Agent System [51.54882384204726]
  We present AgentEvolver, a self-evolving agent system that drives autonomous agent learning.<n>AgentEvolver introduces three synergistic mechanisms: self-questioning, self-navigating, and self-attributing.<n>Preliminary experiments indicate that AgentEvolver achieves more efficient exploration, better sample utilization, and faster adaptation compared to traditional RL-based baselines.
  arXiv  Detail & Related papers  (2025-11-13T15:14:47Z)
• SAMULE: Self-Learning Agents Enhanced by Multi-level Reflection [14.40651157974557]
  SAMULE is a new framework for self-learning agents powered by a retrospective language model that is trained based on Multi-Level Reflection Synthesis.<n>It first synthesizes high-quality reflections across three complementary levels: Single-Trajectory Learning (micro-level) for detailed error correction; Intra-Task Learning (meso-level) to build error across multiple trials of the same task, and Inter-Task Learning (macro-level) to extract transferable insights based on same typed errors from diverse task failures.
  arXiv  Detail & Related papers  (2025-09-24T21:02:15Z)
• Exploring Expert Failures Improves LLM Agent Tuning [74.0772570556016]
  We propose Exploring Expert Failures (EEF), which identifies beneficial actions from failed expert trajectories.<n>EEF successfully solves some previously unsolvable subtasks and improves agent tuning performance.
  arXiv  Detail & Related papers  (2025-04-17T17:53:54Z)
• From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning [62.54484062185869]
  We introduce StepAgent, which utilizes step-wise reward to optimize the agent's reinforcement learning process.<n>We propose implicit-reward and inverse reinforcement learning techniques to facilitate agent reflection and policy adjustment.
  arXiv  Detail & Related papers  (2024-11-06T10:35:11Z)
• Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents [49.85633804913796]
  We present an exploration-based trajectory optimization approach, referred to as ETO. This learning method is designed to enhance the performance of open LLM agents. Our experiments on three complex tasks demonstrate that ETO consistently surpasses baseline performance by a large margin.
  arXiv  Detail & Related papers  (2024-03-04T21:50:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.