ExpGraph: Model-Agnostic Experience Learning with Graph-Structured Memory for LLM Agents

• URL: http://arxiv.org/abs/2605.30712v1
• Date: Fri, 29 May 2026 01:04:41 GMT
• Title: ExpGraph: Model-Agnostic Experience Learning with Graph-Structured Memory for LLM Agents
• Abstract summary: Large language model (LLM) agents have shown strong capabilities in reasoning, tool use, and multi-step interaction.<n>Fine-tuning on collected experience can improve reuse, but it is inflexible when stronger or more suitable executors emerge.<n>We propose ExpGraph, a model-agnostic experience learning framework that enables frozen and replaceable LLM executors to improve through external experience reuse.
• Score: 32.42010710901808
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) agents have shown strong capabilities in reasoning, tool use, and multi-step interaction, but they often solve tasks from scratch and fail to reuse successful strategies or failure lessons from prior experience. Fine-tuning on collected experience can improve reuse, but it is inflexible when stronger or more suitable executors emerge. We propose ExpGraph, a model-agnostic experience learning framework that enables frozen and replaceable LLM executors to improve through external experience reuse without parameter updates. ExpGraph summarizes historical trajectories into reusable skills and failure lessons, organizes them as nodes in a self-evolving experience graph, and retrieves useful experiences through graph diffusion and utility-aware ranking. A lightweight retrieval copilot is trained with reinforcement learning using feedback that compares executor performance with and without retrieved experiences, while the graph is updated online from downstream task outcomes. We evaluate ExpGraph on ExpSuite, covering question answering, mathematical reasoning, code generation, and multi-step agentic environments including ALFWorld and AppWorld. ExpGraph improves over the strongest baseline by 12.2% and 4.7% on static tasks with smaller and larger executors, and by 21.4% and 12.7% in agentic environments, while reducing average interaction steps by 12.7% and 21.6%. Ablations show that graph-structured experience, utility-aware ranking, and adaptive retrieval jointly enable effective experience reuse across diverse tasks and executor models.

Related papers

• SE-GoS: Self-Evolving Graph-of-Skills for Skill Library at Scale [10.571709993050748]
  We present Self-Evolving Graph-of-Skills (SE-GoS), a training-free framework that evolves an existing GoS graph from execution traces.<n>We show that SE-GoS consistently improves task reward while reducing input tokens relative to full skill loading.
  arXiv  Detail & Related papers  (2026-09-08T04:20:11Z)
• EXG: Self-Evolving Agents with Experience Graphs [25.363377373092955]
  We introduce EXG, an experience graph framework for self-evolving agents.<n> EXG explicitly organizes accumulated successes and failures into a structured, relational representation.<n>Our results suggest that structuring experience as a graph provides a principled foundation for scalable and transferable self-evolving agent behavior.
  arXiv  Detail & Related papers  (2026-05-18T00:50:23Z)
• Fine-tuning with RAG for Improving LLM Learning of New Skills [8.825427873545063]
  Large language model (LLM) agents frequently fail in predictable ways.<n>We propose a simple pipeline that converts inference-time retrieval into learned competence through distillation.
  arXiv  Detail & Related papers  (2025-10-01T19:03:48Z)
• Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering [75.12322966980003]
  Large Language Models (LLMs) have shown strong inductive reasoning ability across various domains.<n>Most existing RAG pipelines rely on unstructured text, limiting interpretability and structured reasoning.<n>Recent studies have explored integrating knowledge graphs with LLMs for knowledge graph question answering.<n>We propose RAPL, a novel framework for efficient and effective graph retrieval in KGQA.
  arXiv  Detail & Related papers  (2025-06-11T12:03:52Z)
• TOBUGraph: Knowledge Graph-Based Retrieval for Enhanced LLM Performance Beyond RAG [3.3593672127922836]
  TOBUGraph is a graph-based retrieval framework that first constructs the knowledge graph from unstructured data.<n>It extracts structured knowledge and diverse relationships among data, going beyond RAG's text-to-text similarity.<n>We demonstrate TOBUGraph's effectiveness in TOBU, a real-world application in production for personal memory organization and retrieval.
  arXiv  Detail & Related papers  (2024-12-06T22:05:39Z)
• VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought [41.72701516732208]
  Large-scale generative language and vision-language models (LLMs and VLMs) excel in few-shot learning but require high-quality demonstrations.<n>We propose In-Context Abstraction Learning (ICAL), enabling VLM agents to transform suboptimal trajectories into high-quality training data.
  arXiv  Detail & Related papers  (2024-06-20T17:45:02Z)
• AvaTaR: Optimizing LLM Agents for Tool Usage via Contrastive Reasoning [93.96463520716759]
  Large language model (LLM) agents have demonstrated impressive capabilities in utilizing external tools and knowledge to boost accuracy and hallucinations. Here, we introduce AvaTaR, a novel and automated framework that optimize an LLM agent to effectively leverage provided tools, improving performance on a given task.
  arXiv  Detail & Related papers  (2024-06-17T04:20:02Z)
• SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning [131.04781590452308]
  We present SimTeG, a frustratingly Simple approach for Textual Graph learning. We first perform supervised parameter-efficient fine-tuning (PEFT) on a pre-trained LM on the downstream task. We then generate node embeddings using the last hidden states of finetuned LM.
  arXiv  Detail & Related papers  (2023-08-03T07:00:04Z)
• GIF: A General Graph Unlearning Strategy via Influence Function [63.52038638220563]
  Graph Influence Function (GIF) is a model-agnostic unlearning method that can efficiently and accurately estimate parameter changes in response to a $epsilon$-mass perturbation in deleted data. We conduct extensive experiments on four representative GNN models and three benchmark datasets to justify GIF's superiority in terms of unlearning efficacy, model utility, and unlearning efficiency.
  arXiv  Detail & Related papers  (2023-04-06T03:02:54Z)
• Improving Few-Shot Generalization by Exploring and Exploiting Auxiliary Data [100.33096338195723]
  We focus on Few-shot Learning with Auxiliary Data (FLAD) FLAD assumes access to auxiliary data during few-shot learning in hopes of improving generalization. We propose two algorithms -- EXP3-FLAD and UCB1-FLAD -- and compare them with prior FLAD methods that either explore or exploit.
  arXiv  Detail & Related papers  (2023-02-01T18:59:36Z)
• Retrieval-Augmented Reinforcement Learning [63.32076191982944]
  We train a network to map a dataset of past experiences to optimal behavior. The retrieval process is trained to retrieve information from the dataset that may be useful in the current context. We show that retrieval-augmented R2D2 learns significantly faster than the baseline R2D2 agent and achieves higher scores.
  arXiv  Detail & Related papers  (2022-02-17T02:44:05Z)
• Graph Contrastive Learning Automated [94.41860307845812]
  Graph contrastive learning (GraphCL) has emerged with promising representation learning performance. The effectiveness of GraphCL hinges on ad-hoc data augmentations, which have to be manually picked per dataset. This paper proposes a unified bi-level optimization framework to automatically, adaptively and dynamically select data augmentations when performing GraphCL on specific graph data.
  arXiv  Detail & Related papers  (2021-06-10T16:35:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.