EvoAgentBench: Benchmarking Agent Self-Evolution via Ability Transfer

• URL: http://arxiv.org/abs/2607.05202v1
• Date: Mon, 06 Jul 2026 15:17:09 GMT
• Title: EvoAgentBench: Benchmarking Agent Self-Evolution via Ability Transfer
• Abstract summary: EvoAgentBench is a benchmark for agent self-evolution via Ability-guided transfer.<n>Every test task is backed by verified training-side Ability support.
• Score: 25.657960361149833
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Agent self-evolution in long-horizon LLM systems is largely procedural: useful experience is not merely stored information, but reusable procedures for searching, debugging, and verification. Yet current evaluations do not isolate this form of transfer. Agent benchmarks test single-episode task solving; memory benchmarks target information retention rather than procedural reuse. We introduce EvoAgentBench, a benchmark for agent self-evolution via Ability-guided transfer across four agentic domains: web research, algorithmic reasoning, software engineering, and knowledge work. EvoAgentBench extracts trace-grounded Abilities from agent executions, canonicalizes them into operational units, and builds domain-specific Ability Graphs linking tasks that share procedural overlap. By design, every test task is backed by verified training-side Ability support. Across a 528/267 train/test split, two scaffolds, and three backbones, curated Ability content transfers reliably across model families, but no current automatic method sustains positive gain in all settings. EvoAgentBench shifts self-evolution evaluation from aggregate accuracy comparison to fine-grained diagnosis of experience encoding, routing, and uptake. The benchmark is publicly available at https://huggingface.co/datasets/EverMind-AI/EvoAgentBench.

Related papers

• PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents [50.9061759859778]
  PAST-Bench is a benchmark designed to isolate whether retained experience actually improves agents over time.<n>We report both later-task gains and whether those gains follow the intended save, retrieve, and update pathway.<n>We develop Hermes+, which extends Hermes with five targeted interventions across stages of the agent loop.
  arXiv  Detail & Related papers  (2026-08-04T17:58:05Z)
• AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizers [39.39345650320786]
  We introduce AgentHPOBench, a sequential benchmark comprising 30 executable machine learning tasks across seven research categories.<n>We evaluate 12 widely used agents and conventional HPO baselines under a unified protocol.<n>Results show that current agents exhibit measurable experimental optimization ability across domains, but still face clear limitations in sustained iterative refinement, complex log diagnosis, and consistent progress toward reported reference performance.
  arXiv  Detail & Related papers  (2026-07-31T16:58:00Z)
• Knowledge-Centric Self-Improvement [28.85741798328687]
  Self-improving AI systems treat the agent as the object that improves, by optimizing prompts, harnesses, or even the agent's own code.<n>This agent-centric view can make improvements expensive to maintain and difficult to transfer.<n>We study a complementary paradigm: knowledge-centric self-improvement, in which agents remain generic and disposable.
  arXiv  Detail & Related papers  (2026-07-21T21:38:39Z)
• Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills [12.442806027914097]
  We introduce Socratic-SWE, a closed-loop self-evolution framework that reuses the agent's historical solving traces as a source of training signal.<n>Rather than treating traces only as evidence for reward computation, Socratic-SWE distills them into structured agent skills that summarize recurring failures and effective repair patterns.
  arXiv  Detail & Related papers  (2026-06-05T16:00:17Z)
• AgentSim: A Platform for Verifiable Agent-Trace Simulation [3.2058241360543254]
  AgentSim is an open-source platform for simulating RAG agents.<n>It generates verifiable, stepwise traces of agent reasoning over any document collection.<n>It combines a multi-model validation pipeline with an active human-in-the-loop process.
  arXiv  Detail & Related papers  (2026-04-29T13:19:38Z)
• AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation [71.49152943451328]
  We introduce a benchmark AJ-Bench to evaluate Agent-as-a-Judge across three domains-search, data systems, and graphical user interfaces-comprising 155 tasks and 516 annotated trajectories.<n>Experiments demonstrate consistent performance gains over LLM-as-a-Judge baselines, while also revealing substantial open challenges in agent-based verification.
  arXiv  Detail & Related papers  (2026-04-20T13:23:38Z)
• AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents [50.481033105867205]
  We introduce AgentProcessBench, the first benchmark dedicated to evaluating step-level effectiveness in realistic, tool-augmented trajectories.<n>The benchmark comprises 1,000 diverse trajectories and 8,509 human-labeled step annotations with 89.1% inter-annotator agreement.<n>It features a ternary labeling scheme to capture exploration and an error propagation rule to reduce labeling ambiguity.
  arXiv  Detail & Related papers  (2026-03-15T16:13:58Z)
• AgentSelect: Benchmark for Narrative Query-to-Agent Recommendation [39.61543921719145]
  AgentSelect is a benchmark that reframes agent selection as narrative query-to-agent recommendation.<n>It converts heterogeneous evaluation artifacts into unified, positive-only interaction data.<n>AgentSelect provides the first unified data and evaluation infrastructure for agent recommendation.
  arXiv  Detail & Related papers  (2026-03-04T06:17:51Z)
• SmartSnap: Proactive Evidence Seeking for Self-Verifying Agents [45.71333459905404]
  SmartSnap is a paradigm shift from passive, post-hoc verification to proactive, in-situ self-verification by the agent itself.<n>We introduce the Self-Verifying Agent, a new type of agent designed with dual missions: to complete a task and to prove its accomplishment with curated evidences.<n>Experiments on mobile tasks across model families and scales demonstrate that our SmartSnap paradigm allows training LLM-driven agents in a scalable manner.
  arXiv  Detail & Related papers  (2025-12-26T14:51:39Z)
• AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress [71.02263260394261]
  Large language models (LLMs) still encounter challenges in multi-turn decision-making tasks.<n>We build process reward models (PRMs) to evaluate each decision and guide the agent's decision-making process.<n>AgentPRM captures both the interdependence between sequential decisions and their contribution to the final goal.
  arXiv  Detail & Related papers  (2025-11-11T14:57:54Z)
• Alita-G: Self-Evolving Generative Agent for Agent Generation [54.49365835457433]
  We present ALITA-G, a framework that transforms a general-purpose agent into a domain expert.<n>In this framework, a generalist agent executes a curated suite of target-domain tasks.<n>It attains strong gains while reducing computation costs.
  arXiv  Detail & Related papers  (2025-10-27T17:59:14Z)
• R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Science [70.1638335489284]
  High-level machine learning engineering tasks remain labor-intensive and iterative.<n>We introduce R&D-Agent, a comprehensive, decoupled, and framework that formalizes the machine learning process.<n>R&D-Agent defines the MLE into two phases and six components, turning agent design for MLE into a principled, testable process.
  arXiv  Detail & Related papers  (2025-05-20T06:07:00Z)
• Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuning [68.00304954972232]
  Multimodal agents, which integrate a controller e.g., a vision language model, with external tools, have demonstrated remarkable capabilities in tackling complex multimodal tasks.<n>Existing approaches for training these agents depend on extensive human-annotated task-answer pairs and tool trajectories.<n>We propose an iterative tool usage exploration method for multimodal agents without any pre-collected data, namely SPORT.<n>SPORT has four iterative components: task synthesis, step sampling, step verification, and preference tuning.
  arXiv  Detail & Related papers  (2025-04-30T12:01:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.