Trajectory-Informed Memory Generation for Self-Improving Agent Systems

• URL: http://arxiv.org/abs/2603.10600v1
• Date: Wed, 11 Mar 2026 09:54:09 GMT
• Title: Trajectory-Informed Memory Generation for Self-Improving Agent Systems
• Abstract summary: LLM-powered agents face a persistent challenge: learning from their execution experiences to improve future performance.<n>We present a novel framework for automatically extracting actionable learnings from agent execution trajectories.<n>Our framework understands execution patterns, extracts structured learnings with provenance, and retrieves guidance tailored to specific task contexts.
• Score: 4.933717407152962
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-powered agents face a persistent challenge: learning from their execution experiences to improve future performance. While agents can successfully complete many tasks, they often repeat inefficient patterns, fail to recover from similar errors, and miss opportunities to apply successful strategies from past executions. We present a novel framework for automatically extracting actionable learnings from agent execution trajectories and utilizing them to improve future performance through contextual memory retrieval. Our approach comprises four components: (1) a Trajectory Intelligence Extractor that performs semantic analysis of agent reasoning patterns, (2) a Decision Attribution Analyzer that identifies which decisions and reasoning steps led to failures, recoveries, or inefficiencies, (3) a Contextual Learning Generator that produces three types of guidance -- strategy tips from successful patterns, recovery tips from failure handling, and optimization tips from inefficient but successful executions, and (4) an Adaptive Memory Retrieval System that injects relevant learnings into agent prompts based on multi-dimensional similarity. Unlike existing memory systems that store generic conversational facts, our framework understands execution patterns, extracts structured learnings with provenance, and retrieves guidance tailored to specific task contexts. Evaluation on the AppWorld benchmark demonstrates consistent improvements, with up to 14.3 percentage point gains in scenario goal completion on held-out tasks and particularly strong benefits on complex tasks (28.5~pp scenario goal improvement, a 149\% relative increase).

Related papers

• What is Missing from AI Post-Training AI: An Empirical Analysis [74.56398070411755]
  We argue that this picture conflates two distinct capabilities: execution-level capability and strategy-level capability.<n>We find that across different tasks, the agent's training strategy is locked in at the very beginning.<n>We then examine three natural explanations--missing experience, missing guidance, and insufficient reasoning--with escalating interventions.
  arXiv  Detail & Related papers  (2026-08-19T16:17:39Z)
• The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows [72.9051182568928]
  We present ReASearch, a unified framework for reasoning-driven optimization.<n>With a shared agent loop and domain-specific tools, ReASearch instantiates the exact same scaffold to optimize prompts, programs, and ML.<n>We observe that complex search behaviors emerge naturally from the agent's reasoning process.
  arXiv  Detail & Related papers  (2026-08-07T02:15:39Z)
• SEARL: Joint Optimization of Policy and Tool Graph Memory for Self-Evolving Agents [35.45207852488779]
  We introduce a Tool-Memory based self-evolving agentic framework SEARL.<n>Unlike approaches that directly utilize interaction experiences, our method constructs a structured experience memory that integrates planning with execution.<n>We evaluate our framework on knowledge reasoning and mathematics tasks, demonstrating its effectiveness in achieving more practical and efficient learning.
  arXiv  Detail & Related papers  (2026-04-09T04:38:47Z)
• Procedural Knowledge at Scale Improves Reasoning [25.36077714467684]
  We introduce Reasoning Memory, a framework for reasoning models that explicitly retrieves and reuses procedural knowledge at scale.<n> Reasoning Memory consistently outperforms RAG with document, trajectory, and template knowledge, as well as a compute-matched test-time scaling baseline.
  arXiv  Detail & Related papers  (2026-04-01T20:01:47Z)
• A Hierarchical Error-Corrective Graph Framework for Autonomous Agents with LLM-Based Action Generation [3.503249117641051]
  We propose a Hierarchical Error-Corrective Graph FrameworkforAutonomousAgentswithLLM-BasedActionGeneration(HECG),whichincorporates three core innovations.<n>MDTS: by integrating task quality metrics (Q), confidence/cost metrics (C), reward metrics (R), and LLM-based semantic reasoning scores (LLM-Score), MDTS achieves multi-dimensional alignment between quantitative performance and semantic context.<n> EMC: unlike simple confusion matrices or overall performance metrics, EMC provides structured attribution of task failures by categorizing errors into ten types, such as Strategy Errors (Strategy Whe) and Script
  arXiv  Detail & Related papers  (2026-03-09T13:46:00Z)
• Exploring Reasoning Reward Model for Agents [30.458783880389216]
  Agentic Reinforcement Learning (Agentic RL) has achieved notable success in enabling agents to perform complex reasoning and tool use.<n>Most methods still relies on sparse outcome-based reward for training.<n>We introduce Agent Reasoning Reward Model (Agent-RRM), a multi-faceted reward model that produces structured feedback for agentic trajectories.
  arXiv  Detail & Related papers  (2026-01-29T18:59:52Z)
• ReflexGrad: Three-Way Synergistic Architecture for Zero-Shot Generalization in LLM Agents [0.0]
  We introduce ReflexGrad, a novel architecture that tightly couples three complementary mechanisms.<n>Our system achieves true zero-shot generalization through pure semantic reasoning.<n>Our work demonstrates that synergistic integration of complementary learning mechanisms enables robust zero-shot generalization.
  arXiv  Detail & Related papers  (2025-11-18T15:25:05Z)
• Experience-Guided Adaptation of Inference-Time Reasoning Strategies [49.954515048847874]
  Experience-Guided Reasoner (EGuR) generates tailored strategies at inference time based on accumulated experience.<n>EGuR achieves up to 14% accuracy improvements over the strongest baselines while reducing computational costs by up to 111x.
  arXiv  Detail & Related papers  (2025-11-14T17:45:28Z)
• AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress [71.02263260394261]
  Large language models (LLMs) still encounter challenges in multi-turn decision-making tasks.<n>We build process reward models (PRMs) to evaluate each decision and guide the agent's decision-making process.<n>AgentPRM captures both the interdependence between sequential decisions and their contribution to the final goal.
  arXiv  Detail & Related papers  (2025-11-11T14:57:54Z)
• Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories [10.751728274263536]
  This paper presents an empirical study of agent trajectories, namely the execution traces capturing the steps agents take when attempting to resolve software issues.<n>We analyse trajectories from three state-of-the-art code agents (OpenHands, SWE-agent, and Prometheus) on the SWE-Bench benchmark, examining both successful and failed attempts.
  arXiv  Detail & Related papers  (2025-10-31T18:58:13Z)
• Scaling Test-time Compute for LLM Agents [51.790752085445384]
  Scaling test time compute has shown remarkable success in improving the reasoning abilities of large language models (LLMs)<n>In this work, we conduct the first systematic exploration of applying test-time scaling methods to language agents.
  arXiv  Detail & Related papers  (2025-06-15T17:59:47Z)
• Learning Task Representations from In-Context Learning [67.66042137487287]
  Large language models (LLMs) have demonstrated remarkable proficiency in in-context learning (ICL)<n>We introduce an automated formulation for encoding task information in ICL prompts as a function of attention heads.<n>The proposed method successfully extracts task-specific information from in-context demonstrations and excels in both text and regression tasks.
  arXiv  Detail & Related papers  (2025-02-08T00:16:44Z)
• Understanding the Design Decisions of Retrieval-Augmented Generation Systems [7.10184268156888]
  Retrieval-Augmented Generation (RAG) has emerged as a critical technique for enhancing large language model (LLM) capabilities.<n>We present the first comprehensive study of three universal RAG deployment decisions.
  arXiv  Detail & Related papers  (2024-11-29T04:25:31Z)
• Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinement [50.481380478458945]
  Iterative step-level Process Refinement (IPR) framework provides detailed step-by-step guidance to enhance agent training. Our experiments on three complex agent tasks demonstrate that our framework outperforms a variety of strong baselines.
  arXiv  Detail & Related papers  (2024-06-17T03:29:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.