SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessment

• URL: http://arxiv.org/abs/2604.08988v2
• Date: Tue, 14 Apr 2026 01:08:07 GMT
• Title: SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessment
• Abstract summary: This paper defines the Self-Evolving Evolutionary Agent and introduces SEA-Eval -- the first benchmark designed specifically for evaluating SEAs.<n> SEA-Eval establishes $SR$ and $T$ as primary metrics and enables through sequential task stream design the independent of evolutionary gain, evolutionary stability, and implicit alignment convergence.
• Score: 52.87618765740484
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current LLM-based agents demonstrate strong performance in episodic task execution but remain constrained by static toolsets and episodic amnesia, failing to accumulate experience across task boundaries. This paper presents the first formal definition of the Self-Evolving Agent (SEA), formalizes the Evolutionary Flywheel as its minimal sufficient architecture, and introduces SEA-Eval -- the first benchmark designed specifically for evaluating SEAs. Grounded in Flywheel theory, SEA-Eval establishes $SR$ and $T$ as primary metrics and enables through sequential task stream design the independent quantification of evolutionary gain, evolutionary stability, and implicit alignment convergence. Empirical evaluation reveals that under identical success rates, token consumption differs by up to 31.2$\times$ across frameworks, with divergent evolutionary trajectories under sequential analysis -- demonstrating that success rate alone creates a capability illusion and that the sequential convergence of $T$ is the key criterion for distinguishing genuine evolution from pseudo-evolution.

Related papers

• EvoTS-Agent: A Self-Evolving LLM Agent for Financial Time Series Change Point Detection [13.71748862936038]
  EvoTS-Agent is a validation-guided self-evolving LLM agent for autonomous financial time-series change-point detection.<n>We show that EvoTS-Agent consistently outperforms existing LLM-based agents while maintaining a 100% execution success rate.
  arXiv  Detail & Related papers  (2026-08-18T15:55:32Z)
• Self-Improving Large Language Models via Progressive Experience Evolution [52.03479747358687]
  We propose textbfSPEE (textbfSelf-textbfProgressive textbfExperience textbfEvolution), a unified post-training framework that sequentially performs explicit experience evolution followed by implicit policy optimization.<n> Experiments on five mathematical reasoning benchmarks demonstrate that SPEE consistently outperforms both test-time and training-time self-evolution baselines across three model scales.
  arXiv  Detail & Related papers  (2026-08-03T12:27:32Z)
• AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks? [65.05151983193441]
  Large language model (LLM) agents can self-evolve by continually improving from their own accumulated experience.<n>We introduce AgentStream, a unified framework that evaluates self-evolving agents spanning diverse evolution components.<n>Our results show that self-evolution reliability varies across streaming scenarios, the benefit of self-evolution is gated by model capability and non-monotonic in model strength.
  arXiv  Detail & Related papers  (2026-07-31T17:50:08Z)
• MobEvolve: An Agentic Self-Evolving Heuristic System for Interpretable Human Mobility Generation [51.824145170051516]
  MobEvolve is a self-evolving framework for human mobility generation.<n>It employs an agent to iteratively evolve its internal logic.<n>It significantly outperforms state-of-the-art deep generative and LLM-based methods.
  arXiv  Detail & Related papers  (2026-06-01T03:46:25Z)
• BenchTrace: A Benchmark for Testing Reflection Ability and Controlled Evolution in LLM Agents [24.997265534888626]
  We present textbfBenchTrace, a benchmark for evaluating self-evolution ability in LLM agents.<n>BenchTrace is built on a snapshot-reflection dataset of 1,821 annotated episodes spanning six diverse tasks.<n>We propose textbffailure avoidance rate (FAR), a new evaluation metric measuring the fraction of test cases in which the agent successfully avoids the target failure instance.
  arXiv  Detail & Related papers  (2026-05-28T01:25:37Z)
• Consistency as a Testable Property: Statistical Methods to Evaluate AI Agent Reliability [6.195988633460139]
  This paper establishes a rigorous measurement science for AI agent reliability.<n>By leveraging $U$-statistics for output-level reliability and kernel-based metrics for trajectory-level stability, we offer a principled approach to evaluating agents.
  arXiv  Detail & Related papers  (2026-05-11T13:06:24Z)
• Towards Self-Evolving Benchmarks: Synthesizing Agent Trajectories via Test-Time Exploration under Validate-by-Reproduce Paradigm [60.36837655498119]
  We propose a Trajectory-based validated-by-Reproducing Agent-benchmark Complexity Evolution framework.<n>This framework takes an original task from an existing benchmark and encourages agents to evolve it into a new task with higher difficulty.<n>Experiments on the GAIA benchmark demonstrate that the TRACE framework consistently enhances task complexity while improving the reliability of correctness.
  arXiv  Detail & Related papers  (2025-10-01T01:52:52Z)
• MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentation [55.37355146924576]
  MedSeqFT is a sequential fine-tuning framework for medical image analysis.<n>It adapts pre-trained models to new tasks while refining their representational capacity.<n>It consistently outperforms state-of-the-art fine-tuning strategies.
  arXiv  Detail & Related papers  (2025-09-07T15:22:53Z)
• State Stream Transformer (SST) : Emergent Metacognitive Behaviours Through Latent State Persistence [0.0]
  State Stream Transformer (SST) is a novel LLM architecture that reveals emergent reasoning behaviours latent in pretrained weights.<n>SST introduces a sliding window latent state (FFN) cache with weighted decay that maintains and evolves persistent latent processes throughout autoregressive generations.<n>SST achieves substantial performance improvements over the base model on two reasoning benchmarks, reaching 89.01% accuracy on GSM-8K (0-shot) and 91.04% on ARC Challenge (0-shot CoT)
  arXiv  Detail & Related papers  (2025-01-30T14:03:36Z)
• AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents [74.16170899755281]
  We introduce AgentBoard, a pioneering comprehensive benchmark and accompanied open-source evaluation framework tailored to analytical evaluation of LLM agents.<n>AgentBoard offers a fine-grained progress rate metric that captures incremental advancements as well as a comprehensive evaluation toolkit.<n>This not only sheds light on the capabilities and limitations of LLM agents but also propels the interpretability of their performance to the forefront.
  arXiv  Detail & Related papers  (2024-01-24T01:51:00Z)
• Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RL [86.0987896274354]
  We first identify a fundamental pattern, self-excitation, as the primary cause of Q-value estimation divergence in offline RL. We then propose a novel Self-Excite Eigenvalue Measure (SEEM) metric to measure the evolving property of Q-network at training. For the first time, our theory can reliably decide whether the training will diverge at an early stage.
  arXiv  Detail & Related papers  (2023-10-06T17:57:44Z)
• Data Augmentation through Expert-guided Symmetry Detection to Improve Performance in Offline Reinforcement Learning [0.0]
  offline estimation of the dynamical model of a Markov Decision Process (MDP) is a non-trivial task. Recent works showed that an expert-guided pipeline relying on Density Estimation methods effectively detects this structure in deterministic environments. We show that the former results lead to a performance improvement when solving the learned MDP and then applying the optimized policy in the real environment.
  arXiv  Detail & Related papers  (2021-12-18T14:32:32Z)
• Episodic Memory for Learning Subjective-Timescale Models [1.933681537640272]
  In model-based learning, an agent's model is commonly defined over transitions between consecutive states of an environment. In contrast, intelligent behaviour in biological organisms is characterised by the ability to plan over varying temporal scales depending on the context. We devise a novel approach to learning a transition dynamics model, based on the sequences of episodic memories that define the agent's subjective timescale.
  arXiv  Detail & Related papers  (2020-10-03T21:55:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.