OpenLoopEvolve: A Verifiable Self-Evolution Framework for Loop Policies in Long-Horizon Complex Tasks

• URL: http://arxiv.org/abs/2608.09380v1
• Date: Mon, 10 Aug 2026 09:57:26 GMT
• Title: OpenLoopEvolve: A Verifiable Self-Evolution Framework for Loop Policies in Long-Horizon Complex Tasks
• Abstract summary: This paper presents OpenLoopEvolve (OLE), a self-evolution framework centered on the Loop Policy.<n>OLE represents an agent's observation, planning, memory, action, verification, recovery, stopping, and budget control as portable policy assets with versions and lineages.<n>Online mode triggers candidate generation based on feedback from continuous operation, whereas the offline mode searches for candidate policies from archived traces and failure evidence.
• Score: 17.773038952370865
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-horizon complex tasks require agents to repeatedly observe states, formulate plans, invoke tools, verify results, and recover from failures in continuously changing environments. However, such control experience often remains confined to a single context or a fixed prompt, and is difficult to accumulate and reuse across historical traces. This paper presents OpenLoopEvolve (OLE), a self-evolution framework centered on the Loop Policy. OLE represents an agent's observation, planning, memory, action, verification, recovery, stopping, and budget control as portable policy assets with versions and lineages, and provides online and offline evolution modes that can be selected according to practical needs: the online mode triggers candidate generation based on feedback from continuous operation, whereas the offline mode searches for candidate policies from archived traces and failure evidence. Both modes share an evolution mechanism consisting of autonomous proposals by a large language model, Champion--Challenger paired evaluation, and robust release. Policies released online are activated at a subsequent task boundary, monitored using subsequent feedback, and rolled back to their parent versions when degradation conditions are met. On the simulated business benchmark YC-Bench, both modes improve aggregate task performance, task success rate, and risk metrics relative to a fixed initial Loop Policy. The results indicate that treating the Loop Policy as a governable asset can support the accumulation, comparison, release, and reuse of control experience and improve agent performance on long-horizon complex tasks.

Related papers

• Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement [22.481667463025214]
  We propose Dynamic Retrieval-based Policy Generation (DRPG), a framework that integrates memory-based retrieval with a dynamic policy generator.<n>We evaluate DRPG across six benchmarks spanning text-to-answer, question answering, medical diagnosis, and Python programming.<n>We find that DRPG's policy generation is robust to retrieval strategy, operates effectively without prior policy continuity, and can leverage smaller or cross-family models as cost-efficient policy generators.
  arXiv  Detail & Related papers  (2026-09-15T08:06:17Z)
• Discovering Diverse Planning Policies for Multimodal Embodied Agents with Quality-Diversity Optimization [14.306388130938236]
  We propose a Quality-Diversity framework for discovering diverse planning policies for multimodal embodied agents.<n>The proposed method treats planning-policy templates as evolvable individuals and organizes them into a behavior-indexed archive.<n>In the offline stage, rollout trajectories are summarized into structured success and failure experiences, which guide policy variation.<n>In the online stage, the system rolls back to the latest checkpoint and switches to a behaviorally distinct archive policy to resume execution.
  arXiv  Detail & Related papers  (2026-08-09T06:41:53Z)
• MEMENTO: Memory-Guided Memetic Code-as-Policy Evolution [3.5615347765658396]
  Long-horizon embodied tasks require policies that execute many dependent actions before task success can be observed.<n>We introduce MEMENTO, a memory-guided single-elite memetic framework for code-as-policy evolution.<n>We evaluate MEMENTO on two long- horizon embodied domains: Robosuite Franka Tower-of-Hanoi manipulation and AI2- THOR household interaction.
  arXiv  Detail & Related papers  (2026-07-24T18:16:35Z)
• EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments [64.33452656100393]
  We introduce Autonomous Policy Evolution, a controlled evaluation setting in which a harness-model agent repeatedly edits an executable policy system under a fixed interaction budget.<n>We instantiate this setting in EvoPolicyGym, a benchmark built from compact interactive RL environments.<n>On the EvoPolicyGym suite, GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments.
  arXiv  Detail & Related papers  (2026-07-02T17:10:13Z)
• OLIVIA: Online Learning via Inference-time Action Adaptation for Decision Making in LLM ReAct Agents [74.20327254615854]
  Large language model agents interleave reasoning, action selection, and observation to solve sequential decision-making tasks.<n>Existing inference-time adaptation methods for LLM agents mainly rely on prompting or retrieval.<n>We propose OLIVIA, an inference-time action adaptation framework for ReAct-style agents.
  arXiv  Detail & Related papers  (2026-05-11T19:28:20Z)
• AgentSwing: Adaptive Parallel Context Management Routing for Long-Horizon Web Agents [115.18197165538943]
  Large language models (LLMs) evolve into autonomous agents for long-horizon information-seeking.<n>Existing context management methods typically commit to a single fixed strategy throughout the entire trajectory.<n>We introduce a probabilistic framework that characterizes long-horizon success through two complementary dimensions: search efficiency and terminal precision.
  arXiv  Detail & Related papers  (2026-03-29T02:59:27Z)
• PACEvolve: Enabling Long-Horizon Progress-Aware Consistent Evolution [64.15555230987222]
  PACEvolve is a framework designed to robustly govern the agent's context and search dynamics.<n>We demonstrate that PACEvolve provides a systematic path to consistent, long-horizon self-improvement.
  arXiv  Detail & Related papers  (2026-01-15T18:25:23Z)
• Decision Mamba: A Multi-Grained State Space Model with Self-Evolution Regularization for Offline RL [57.202733701029594]
  We propose Decision Mamba, a novel multi-grained state space model (SSM) with a self-evolving policy learning strategy.<n>To address these challenges, we propose Decision Mamba, a novel multi-grained state space model (SSM) with a self-evolving policy learning strategy.<n>To mitigate the overfitting issue on noisy trajectories, a self-evolving policy is proposed by using progressive regularization.
  arXiv  Detail & Related papers  (2024-06-08T10:12:00Z)
• Offline Reinforcement Learning from Datasets with Structured Non-Stationarity [50.35634234137108]
  Current Reinforcement Learning (RL) is often limited by the large amount of data needed to learn a successful policy. We address a novel Offline RL problem setting in which, while collecting the dataset, the transition and reward functions gradually change between episodes but stay constant within each episode. We propose a method based on Contrastive Predictive Coding that identifies this non-stationarity in the offline dataset, accounts for it when training a policy, and predicts it during evaluation.
  arXiv  Detail & Related papers  (2024-05-23T02:41:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.