PATH-Bench: Path-Dependent Evaluation of Lifelong Agents

• URL: http://arxiv.org/abs/2608.01149v1
• Date: Sun, 02 Aug 2026 11:00:04 GMT
• Title: PATH-Bench: Path-Dependent Evaluation of Lifelong Agents
• Abstract summary: PATH-Bench is a benchmark for path-dependent evaluation of lifelong agents.<n>We propose Selective Experience Use (SEU), an agent harness that regulates how path-accumulated experience influences each new task.
• Score: 27.837334107109786
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lifelong LLM agents increasingly adapt through external learning states that store past interactions as retrievable memories or reusable skills, yet existing benchmarks rarely account for how the path of accumulated experience shapes what agents transfer and retain. In this work, we establish PATH-Bench, a benchmark for path-dependent evaluation of lifelong agents. PATH-Bench estimates directed task relationships via multi-model in-context learning, constructs probe-centered sequences with controlled helpful and interfering histories, and repeatedly evaluates probe tasks to measure average performance, forward transfer, backward transfer, and forgetting. We evaluate eight representative agents on single-turn code generation and multi-turn tool-use tasks under positive- and negative-dominant histories. Benchmark results show that experience utility depends jointly on how experience is represented and on the task's interaction structure, that strong transfer does not ensure retention, and that later experience can reshape gains acquired earlier in the learning path. Based on these findings, we propose Selective Experience Use (SEU), an agent harness that regulates how path-accumulated experience influences each new task, admitting helpful items while filtering out potential interference. SEU consistently reduces forgetting while improving forward transfer in the majority of settings. The PATH-Bench provides both a controlled evaluation framework and actionable guidance for designing more selective and robust lifelong agents.

Related papers

• LifeMem: Enabling Lifelong Experience Reuse for LLM Agents [79.11224209641739]
  LifeMem is a lifelong learning framework that enables agents to transfer knowledge across multiple environments.<n>LifeMem accumulated interaction trajectories based on underlying trajectories to extract reusable skills.<n>Results show that LifeMem enables effective experience reuse in lifelong learning, achieving both reduced forgetting on learned tasks and superior cross-task transfer.
  arXiv  Detail & Related papers  (2026-09-11T10:00:43Z)
• S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement? [66.17176838165994]
  Large language models (LLMs) increasingly interact with external environments and accumulate substantial behavioral experience.<n>It remains unclear whether an agent can actively test its behavior, judge the resulting experience, and use that experience to improve future decisions.<n>We introduce textbfStextsuperscript3Gym, an interactive benchmark for evaluating LLM self-improvement.
  arXiv  Detail & Related papers  (2026-08-31T17:05:41Z)
• EDGE: Experience-Distillation for Guided Exploration in Agentic Reinforcement Learning [25.200068256200293]
  reusable exploration patterns embedded in interaction trajectories are largely discarded after a single policy update.<n>We propose EDGE (Experience-Distillation for Guided Exploration), a framework that treats retrieved experiences as temporary training-time scaffolds.<n> EDGE improves over strong RL baselines by up to 12.5 points and remains effective without inference-time scaffolds or a proprietary reflector.
  arXiv  Detail & Related papers  (2026-08-22T13:16:25Z)
• PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents [50.9061759859778]
  PAST-Bench is a benchmark designed to isolate whether retained experience actually improves agents over time.<n>We report both later-task gains and whether those gains follow the intended save, retrieve, and update pathway.<n>We develop Hermes+, which extends Hermes with five targeted interventions across stages of the agent loop.
  arXiv  Detail & Related papers  (2026-08-04T17:58:05Z)
• From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents [56.187590188698955]
  Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful.<n>Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles.<n>We introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories.<n>We propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment.
  arXiv  Detail & Related papers  (2026-08-03T12:52:04Z)
• AgentCL: Toward Rigorous Evaluation of Continual Learning in Language Agents [30.801952443449633]
  Continual learning expects an agent to accumulate reusable experience across a stream of tasks, improve over time, and avoid interference from irrelevant experiences.<n>Most efforts focus on retrieval and reasoning over long-context conversations or documents, while recent lifelong-adaptation benchmarks often rely on naive task streams.<n>This paper presents an evaluation framework AgentCL for continual learning agents, centered on controlled task streams and metrics for transfer gains.
  arXiv  Detail & Related papers  (2026-06-01T16:32:59Z)
• Rethinking Experience Utilization in Self-Evolving Language Model Agents [51.10420305280499]
  Self-evolving agents improve by accumulating and reusing experience from past interactions.<n>This paper studies experience utilization as a critical design dimension of self-evolving agents.
  arXiv  Detail & Related papers  (2026-05-08T02:48:49Z)
• Experiential Reflective Learning for Self-Improving LLM Agents [1.1074589887824053]
  We introduce Experiential Reflective Learning (ERL), a simple self-improvement framework that enables rapid environment adaptation.<n>ERL reflects on task trajectories and outcomes to generate actionable lessons that transfer across tasks.<n>ERL improves success rate by 7.8% over a ReAct baseline, with large gains in task completion reliability.
  arXiv  Detail & Related papers  (2026-03-25T11:43:22Z)
• Internalizing Agency from Reflective Experience [20.650609947690196]
  LEAFE is a framework that internalizes recovery agency from reflective experience.<n>It consistently improves Pass@1 over the base model and achieves higher Pass@k than outcome-driven baselines.
  arXiv  Detail & Related papers  (2026-03-17T17:50:47Z)
• How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior [65.70584076918679]
  Memory is a critical component in large language model (LLM)-based agents.<n>This paper studies how memory management choices impact the LLM agents' behavior, especially their long-term performance.
  arXiv  Detail & Related papers  (2025-05-21T22:35:01Z)
• CoPS: Empowering LLM Agents with Provable Cross-Task Experience Sharing [70.25689961697523]
  We propose a generalizable algorithm that enhances sequential reasoning by cross-task experience sharing and selection. Our work bridges the gap between existing sequential reasoning paradigms and validates the effectiveness of leveraging cross-task experiences.
  arXiv  Detail & Related papers  (2024-10-22T03:59:53Z)
• Learning to Sample with Local and Global Contexts in Experience Replay Buffer [135.94190624087355]
  We propose a new learning-based sampling method that can compute the relative importance of transition. We show that our framework can significantly improve the performance of various off-policy reinforcement learning methods.
  arXiv  Detail & Related papers  (2020-07-14T21:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.