The World Won't Stay Still: Programmable Evolution for Agent Benchmarks

• URL: http://arxiv.org/abs/2603.05910v1
• Date: Fri, 06 Mar 2026 04:56:18 GMT
• Title: The World Won't Stay Still: Programmable Evolution for Agent Benchmarks
• Abstract summary: LLM-powered agents fulfill user requests by interacting with environments, querying data, and invoking tools in a multi-turn process.<n>Most existing benchmarks assume static environments with fixed schemas and toolsets, neglecting the evolutionary nature of the real world and agents' robustness to environmental changes.<n>We propose ProEvolve, a graph-based framework that makes environment evolution programmable.
• Score: 44.36372545284675
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM-powered agents fulfill user requests by interacting with environments, querying data, and invoking tools in a multi-turn process. Yet, most existing benchmarks assume static environments with fixed schemas and toolsets, neglecting the evolutionary nature of the real world and agents' robustness to environmental changes. In this paper, we study a crucial problem: how to evolve the agent environment in a scalable and controllable way, thereby better evaluating agents' adaptability to real-world dynamics. We propose ProEvolve, a graph-based framework that makes environment evolution programmable. At its core, a typed relational graph provides a unified, explicit representation of the environment: data, tools, and schema. Under this formalism, adding, removing, or modifying capabilities are expressed as graph transformations that coherently propagate updates across tools, schemas, and data access. Building on this, ProEvolve can (1) program the evolutionary dynamics as graph transformations to generate environments automatically, and (2) instantiate task sandboxes via subgraph sampling and programming. We validate ProEvolve by evolving a single environment into 200 environments and 3,000 task sandboxes, and benchmark representative agents accordingly.

Related papers

• OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution [71.06848799741152]
  AI agents operate in persistent environments where early state changes can influence decisions far into the future.<n>Current safety benchmarks often fail to capture cumulative risks because they focus on short, static tasks.<n>We introduce OpenART, an open-ended arena for scalable agent red teaming through environment evolution.
  arXiv  Detail & Related papers  (2026-08-01T13:51:55Z)
• Scalable Environments Drive Generalizable Agents [42.76199937252846]
  Generalizable agents should adapt to diverse tasks and unseen environments beyond their training distribution.<n>Current scaling practices focus on collecting more experience or broader task sets under fixed interaction rules.<n>We propose a unified taxonomy that separates trajectory scaling, task scaling, and environment scaling by their primary deliverables.
  arXiv  Detail & Related papers  (2026-05-18T10:24:05Z)
• Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence [122.8626256816085]
  textbfAgent-World is a self-evolving training arena for advancing general agent intelligence through scalable environments.<n>Agent-World-8B and 14B consistently outperforms strong proprietary models and environment scaling baselines.
  arXiv  Detail & Related papers  (2026-04-20T14:01:10Z)
• Autonomous Continual Learning of Computer-Use Agents for Environment Adaptation [57.65688895630163]
  We introduce ACuRL, an Autonomous Curriculum Reinforcement Learning framework that continually adapts agents to specific environments with zero human data.<n>Our method effectively enables both intra-environment and cross-environment continual learning, yielding 4-22% performance gains without forgetting existing environments.
  arXiv  Detail & Related papers  (2026-02-10T23:06:02Z)
• Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning [62.499592503950026]
  Large language model (LLM) have empowered autonomous agents to perform complex tasks that require multi-turn interactions with tools and environments.<n>We propose Agent World Model (AWM), a fully synthetic environment generation pipeline.<n>We scale to 1,000 environments covering everyday scenarios, in which agents can interact with rich toolsets.
  arXiv  Detail & Related papers  (2026-02-10T18:55:41Z)
• ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training [34.682505898865884]
  We introduce ScaleEnv, a framework that constructs fully interactive environments and verifiable tasks entirely from scratch.<n>By enabling agents to learn through exploration within ScaleEnv, we demonstrate significant performance improvements on unseen, multi-turn tool-use benchmarks.
  arXiv  Detail & Related papers  (2026-02-06T16:05:55Z)
• HiVA: Self-organized Hierarchical Variable Agent via Goal-driven Semantic-Topological Evolution [13.440964262446558]
  Hierarchical Variable Agent (HiVA) is a novel framework modeling agentic as self-organized graphs with the Semantic-Topological Evolution (STEV) algorithm.<n> Experiments on dialogue, coding, Longcontext Q&A, mathematical, and agentic benchmarks demonstrate improvements of 5-10% in task accuracy and enhanced resource efficiency.
  arXiv  Detail & Related papers  (2025-08-29T18:51:18Z)
• DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback [62.235925602004535]
  DataEnvGym is a testbed of teacher environments for data generation agents.<n>It frames data generation as a sequential decision-making task, involving an agent and a data generation engine.<n>Students are iteratively trained and evaluated on generated data, and their feedback is reported to the agent after each iteration.
  arXiv  Detail & Related papers  (2024-10-08T17:20:37Z)
• Improving out-of-distribution generalization in graphs via hierarchical semantic environments [5.481047026874547]
  We propose a novel approach to generate hierarchical environments for each graph. We introduce a new learning objective that guides our model to learn the diversity of environments within the same hierarchy. Our framework achieves up to 1.29% and 2.83% improvement over the best baselines on IC50 and EC50 prediction tasks, respectively.
  arXiv  Detail & Related papers  (2024-03-04T07:03:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.