EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments

• URL: http://arxiv.org/abs/2607.02440v1
• Date: Thu, 02 Jul 2026 17:10:13 GMT
• Title: EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments
• Abstract summary: We introduce Autonomous Policy Evolution, a controlled evaluation setting in which a harness-model agent repeatedly edits an executable policy system under a fixed interaction budget.<n>We instantiate this setting in EvoPolicyGym, a benchmark built from compact interactive RL environments.<n>On the EvoPolicyGym suite, GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments.
• Score: 64.33452656100393
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous agents are increasingly expected to improve executable policies through feedback, yet existing evaluations often collapse this process into a final score or confound it with open-ended software-engineering progress. We introduce Autonomous Policy Evolution, a controlled evaluation setting in which a harness-model agent repeatedly edits an executable policy system under a fixed interaction budget. We instantiate this setting in EvoPolicyGym, a benchmark built from compact interactive RL environments that evaluates how agents iteratively improve explored policies. On the EvoPolicyGym suite, GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments. Beyond leaderboard results, EvoPolicyGym also provides trajectory-level diagnostics that distinguish how agents allocate budget, convert feedback into parametric tuning. These analyses show that strong autonomous policy evolution depends not only on isolated task wins, but on discovering task-appropriate mechanisms and refining policies under bounded feedback.

Related papers

• OpenLoopEvolve: A Verifiable Self-Evolution Framework for Loop Policies in Long-Horizon Complex Tasks [17.773038952370865]
  This paper presents OpenLoopEvolve (OLE), a self-evolution framework centered on the Loop Policy.<n>OLE represents an agent's observation, planning, memory, action, verification, recovery, stopping, and budget control as portable policy assets with versions and lineages.<n>Online mode triggers candidate generation based on feedback from continuous operation, whereas the offline mode searches for candidate policies from archived traces and failure evidence.
  arXiv  Detail & Related papers  (2026-08-10T09:57:26Z)
• MEMENTO: Memory-Guided Memetic Code-as-Policy Evolution [3.5615347765658396]
  Long-horizon embodied tasks require policies that execute many dependent actions before task success can be observed.<n>We introduce MEMENTO, a memory-guided single-elite memetic framework for code-as-policy evolution.<n>We evaluate MEMENTO on two long- horizon embodied domains: Robosuite Franka Tower-of-Hanoi manipulation and AI2- THOR household interaction.
  arXiv  Detail & Related papers  (2026-07-24T18:16:35Z)
• Co-Evolving LLM Evaluators and Policies via DynamicRubric [67.2962847914162]
  Post-training with evaluator feedback on policy-induced samples serves as a major mechanism for improving large language models.<n>We propose DynamicRubric, a response-set-conditioned evaluator--policy co-evolution framework.<n>In experiments with 8B backbones, DynamicRubric improves evaluator performance and provides stronger policy supervision than baselines.
  arXiv  Detail & Related papers  (2026-07-22T12:35:40Z)
• Self-evolving LLM agents with in-distribution Optimization [60.05867547965365]
  Large Language Models (LLMs) have recently emerged as powerful controllers for interactive agents in complex environments.<n>We propose Q-Evolve, a self-evolving framework for LLM agents that unifies automatic process-reward labeling and policy learning.<n>We evaluate our method on AlfWorld, WebShop, and ScienceWorld, showing Q-Evolve outperforms strong baselines in sample efficiency, robustness, and overall task performance.
  arXiv  Detail & Related papers  (2026-06-05T15:09:52Z)
• EvoDrive: Pareto Evolution for Safety-Critical Autonomous Driving via Self-Improving LLM Agents [59.44069233970625]
  Existing methods usually manage this trade-off with handcrafteds and overlooking underexplored patterns.<n>Here we present EvoDrive, the first automated, LLM-based agentic evolution framework for scenario generation.<n>EvoDrive employs a simulator-grounded actor-critic architecture where a memory-driven actor iteratively proposes improvements to the generators and critics filter out implausible candidates.
  arXiv  Detail & Related papers  (2026-06-02T14:01:23Z)
• COMAP: Co-Evolving World Models and Agent Policies for LLM Agents [14.918267305899619]
  COMAP is a novel framework that co-evolves textual world models and agent policies through closed-loop interaction.<n>At each decision step, the world model predicts future state feedback for candidate actions, and the agent performs future-aware reflection by estimating the reliability of this feedback.<n>The resulting on-policy trajectories are then used to update the world model via self-distillation, allowing it to better match the agent's evolving interaction distribution.
  arXiv  Detail & Related papers  (2026-06-01T15:21:17Z)
• Harnessing Agentic Evolution [32.567492765277855]
  Agentic evolution is a powerful paradigm for improving programs, iteration, and scientific solutions by iteratively generating candidates, evaluating them, and using feedback to guide future search.<n>Existing methods are typically instantiated either as fixed hand-designed procedures that are modular but rigid, or as general-purpose agents that flexibly integrate feedback but can drift in long-horizon evolution.<n>We introduce AEvo, a harnessed meta-editing framework in which a meta-agent observes this state and acts not by directly proposing the next candidate, but by editing the procedure or agent context that controls future evolution.
  arXiv  Detail & Related papers  (2026-05-13T17:45:16Z)
• Policy Gradient Methods for Non-Markovian Reinforcement Learning [26.23306267677347]
  We study policy gradient methods for reinforcement learning in non-Markovian decision processes (NMDPs)<n>In contrast to approaches that treat the agent state dynamics as fixed or learn it via predictive objectives, we propose a reward-centric formulation.<n>We establish a novel policy gradient theorem for ASM policies, extending the classical policy gradient results from the Markovian setting to episodic and infinite-horizon discounted NMDPs.
  arXiv  Detail & Related papers  (2026-05-11T16:34:28Z)
• OGPO: Sample Efficient Full-Finetuning of Generative Control Policies [53.42266064673132]
  Generative control policies (GCPs) have emerged as effective parameterizations for robot learning.<n>This work introduces Off-policy Generative Policy Optimization (OGPO), a sample-efficient for finetuning GCPs.<n>OGPO achieves state-of-the-art performance on manipulation tasks spanning multi-task settings, high-precision insertion, and dexterous control.
  arXiv  Detail & Related papers  (2026-05-04T18:36:40Z)
• GRASP: Gradient Realignment via Active Shared Perception for Multi-Agent Collaborative Optimization [41.52202306408042]
  Non-stationarity arises from concurrent policy updates and leads to persistent environmental fluctuations.<n>We propose Realignment via Active Shared Perception (GRASP), a novel framework that defines generalized Bellman equilibrium as a stable objective for policy evolution.
  arXiv  Detail & Related papers  (2026-04-01T10:26:22Z)
• Compose Your Policies! Improving Diffusion-based or Flow-based Robot Policies via Test-time Distribution-level Composition [52.232968183793986]
  General Policy Composition (GPC) is a training-free method that enhances performance by combining the distributional scores of multiple pre-trained policies.<n>GPC consistently improves performance and adaptability across a diverse set of tasks.
  arXiv  Detail & Related papers  (2025-10-01T16:05:53Z)
• Off-Policy Evaluation for Large Action Spaces via Policy Convolution [60.6953713877886]
  Policy Convolution family of estimators uses latent structure within actions to strategically convolve the logging and target policies. Experiments on synthetic and benchmark datasets demonstrate remarkable mean squared error (MSE) improvements when using PC.
  arXiv  Detail & Related papers  (2023-10-24T01:00:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.