AgentCL: Toward Rigorous Evaluation of Continual Learning in Language Agents

• URL: http://arxiv.org/abs/2606.02461v2
• Date: Tue, 02 Jun 2026 03:07:54 GMT
• Title: AgentCL: Toward Rigorous Evaluation of Continual Learning in Language Agents
• Abstract summary: Continual learning expects an agent to accumulate reusable experience across a stream of tasks, improve over time, and avoid interference from irrelevant experiences.<n>Most efforts focus on retrieval and reasoning over long-context conversations or documents, while recent lifelong-adaptation benchmarks often rely on naive task streams.<n>This paper presents an evaluation framework AgentCL for continual learning agents, centered on controlled task streams and metrics for transfer gains.
• Score: 30.801952443449633
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Language agents spend substantial inference time solving individual tasks, yet the experience acquired in one episode is often underutilized in future episodes. Continual learning expects an agent to accumulate reusable experience across a stream of tasks, improve over time, and avoid interference from irrelevant experiences. Unfortunately, existing benchmarks struggle to evaluate continual learning in language agents rigorously. Most efforts focus on retrieval and reasoning over long-context conversations or documents, while recent lifelong-adaptation benchmarks often rely on naive task streams with limited analysis of cross-task relationships, making it difficult to understand what an agent learns and reuses over time. This paper presents an evaluation framework AgentCL for continual learning in agents, centered on controlled task streams and metrics for transfer gains. AgentCL constructs compositional streams where earlier sub-solutions, evidence, or workflows are intentionally reusable in later tasks, and contrasts them with naive streams where such reusability is not guaranteed. We use the benchmark to evaluate non-parametric memory designs for continual learning. To diagnose how memory design choices affect continual learning, we develop MemProbe, a probing method that stores interactions, insights, and skills, while filtering unreliable experiences during consolidation. Empirical analysis across coding, deep research, and language understanding/reasoning tasks shows that naive streams offer limited ability to distinguish memory designs, whereas controlled streams more clearly distinguish their plasticity. Meanwhile, naive and held-out settings often yield limited gains and can expose memory-induced degradation. These results highlight the need for stronger memory designs that balance plasticity and stable reuse.

Related papers

• LifeMem: Enabling Lifelong Experience Reuse for LLM Agents [79.11224209641739]
  LifeMem is a lifelong learning framework that enables agents to transfer knowledge across multiple environments.<n>LifeMem accumulated interaction trajectories based on underlying trajectories to extract reusable skills.<n>Results show that LifeMem enables effective experience reuse in lifelong learning, achieving both reduced forgetting on learned tasks and superior cross-task transfer.
  arXiv  Detail & Related papers  (2026-09-11T10:00:43Z)
• PATH-Bench: Path-Dependent Evaluation of Lifelong Agents [27.837334107109786]
  PATH-Bench is a benchmark for path-dependent evaluation of lifelong agents.<n>We propose Selective Experience Use (SEU), an agent harness that regulates how path-accumulated experience influences each new task.
  arXiv  Detail & Related papers  (2026-08-02T11:00:04Z)
• Hidden Forgetting in Continual Multimodal Learning: When Accuracy Survives but Grounding Fails [6.708653593551749]
  We study whether a continually adapted MLLM can preserve not only what it answers, but how it uses visual, textual, OCR, chart, and document evidence.<n>We identify emphhidden evidence-use forgetting, where answer accuracy is retained while the model silently shifts toward different or less grounded evidence channels.<n>We propose textscRCL, a replay-free reliance-constrained continual learning framework.
  arXiv  Detail & Related papers  (2026-07-02T10:50:37Z)
• SkillEvolBench: Benchmarking the Evolution from Episodic Experience to Procedural Skills [31.23929961213889]
  Large language model (LLM) agents accumulate rich episodic trajectories while solving real-world tasks.<n>We introduce SkillEvolBench, a diagnostic benchmark for evaluating this step from experience reuse to skill formation.<n>We find that current agents often adapt locally but rarely form robust reusable skills.
  arXiv  Detail & Related papers  (2026-05-22T18:23:31Z)
• FRESCO: Benchmarking and Optimizing Re-rankers for Evolving Semantic Conflict in Retrieval-Augmented Generation [73.22935457705057]
  We introduce FRESCO, a benchmark for evaluating re-rankers in temporally dynamic contexts.<n>By pairing recency-seeking queries with historical Wikipedia revisions, FRESCO tests whether re-rankers can prioritize factually recent evidence while maintaining semantic relevance.<n>Our evaluation reveals a consistent failure mode across existing re-rankers: a strong bias toward older, semantically rich documents, even when they are factually obsolete.
  arXiv  Detail & Related papers  (2026-04-14T17:04:25Z)
• AgentLongBench: A Controllable Long Benchmark For Long-Contexts Agents via Environment Rollouts [78.33143446024485]
  We introduce textbfAgentLongBench, which evaluates agents through simulated environment rollouts based on Lateral Thinking Puzzles.<n>This framework generates rigorous interaction trajectories across knowledge-intensive and knowledge-free scenarios.
  arXiv  Detail & Related papers  (2026-01-28T16:05:44Z)
• Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory [89.65731902036669]
  Evo-Memory is a streaming benchmark and framework for evaluating self-evolving memory in large language model (LLM) agents.<n>We evaluate over ten representative memory modules and evaluate them across 10 diverse multi-turn goal-oriented and single-turn reasoning and QA datasets.
  arXiv  Detail & Related papers  (2025-11-25T21:08:07Z)
• ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory [57.517214479414726]
  ReasoningBank is a memory framework that distills generalizable reasoning strategies from an agent's self-judged successful and failed experiences.<n>At test time, an agent retrieves relevant memories from ReasoningBank to inform its interaction and then integrates new learnings back, enabling it to become more capable over time.<n>We introduce memory-aware test-time scaling (MaTTS), which accelerates and diversifies this learning process by scaling up the agent's interaction experience.
  arXiv  Detail & Related papers  (2025-09-29T17:51:03Z)
• How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior [65.70584076918679]
  Memory is a critical component in large language model (LLM)-based agents.<n>This paper studies how memory management choices impact the LLM agents' behavior, especially their long-term performance.
  arXiv  Detail & Related papers  (2025-05-21T22:35:01Z)
• Remember and Recall: Associative-Memory-based Trajectory Prediction [25.349986959111757]
  We propose the Fragmented-Memory-based Trajectory Prediction (FMTP) model, inspired by the remarkable learning capabilities of humans. The FMTP model employs discrete representations to enhance computational efficiency by reducing information redundancy. We develop an advanced reasoning engine based on language models to deeply learn the associative rules among these discrete representations.
  arXiv  Detail & Related papers  (2024-10-03T04:32:21Z)
• AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning [16.457330925212606]
  We present adaptive-experience replay (AdaER) to address the challenge of continual lifelong learning. AdaER consists of two stages: memory replay and memory update. Results: AdaER outperforms existing continual lifelong learning baselines.
  arXiv  Detail & Related papers  (2023-08-07T01:25:45Z)
• Relational Experience Replay: Continual Learning by Adaptively Tuning Task-wise Relationship [54.73817402934303]
  We propose Experience Continual Replay (ERR), a bi-level learning framework to adaptively tune task-wise to achieve a better stability plasticity' tradeoff. ERR can consistently improve the performance of all baselines and surpass current state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-31T12:05:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.