Continual Harness: Online Adaptation for Self-Improving Foundation Agents

• URL: http://arxiv.org/abs/2605.09998v1
• Date: Mon, 11 May 2026 05:21:33 GMT
• Title: Continual Harness: Online Adaptation for Self-Improving Foundation Agents
• Abstract summary: Continual Harness is a reset-free self-improving harness for embodied agents.<n>It drives sustained in-game milestone progress on Pokemon Red without resetting the environment between training iterations.
• Score: 21.599574507129365
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Coding harnesses such as Claude Code and OpenHands wrap foundation models with tools, memory, and planning, but no equivalent exists for embodied agents' long-horizon partial-observability decision-making. We first report our Gemini Plays Pokemon (GPP) experiments. With iterative human-in-the-loop harness refinement, GPP became the first AI system to complete Pokemon Blue, Yellow Legacy on hard mode, and Crystal without a lost battle. In the hardest stages, the agent itself began iterating on its strategy through long-context memory, surfacing emergent self-improvement signals alongside human-in-the-loop refinement. Continual Harness removes the human fully from this loop: a reset-free self-improving harness for embodied agents that formalizes and automates what we observed. Starting from only a minimal environment interface, the agent alternates between acting and refining its own prompt, sub-agents, skills, and memory, drawing on any past trajectory data. Prompt-optimization methods require episode resets; Continual Harness adapts online within a single run. On Pokemon Red and Emerald across frontier models, Continual Harness starting from scratch substantially reduces button-press cost relative to the minimalist baseline and recovers a majority of the gap to a hand-engineered expert harness, with capability-dependent gains, despite starting from the same raw interface with no curated knowledge, no hand-crafted tools, and no domain scaffolding. We then close the loop with the model itself: an online process-reward co-learning loop, in which an open-source agent's rollouts through the refining harness are relabeled by a frontier teacher and used to update the model, drives sustained in-game milestone progress on Pokemon Red without resetting the environment between training iterations.

Related papers

• Harness Continual Learning: Continual Adaptation Beyond Model Parameters [9.54786133734043]
  We formulate a new continual learning paradigm in which the harness evolves around a frozen foundation model.<n>A candidate commits a harness only after checking current improvement, historical retention, and validity.<n>Experiments demonstrate capability accumulation and failure recovery, with relative gains exceeding 10% over corresponding baselines in multiple settings.
  arXiv  Detail & Related papers  (2026-08-19T15:12:31Z)
• Zetta $ζ$: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence [35.97081339163319]
  Embodied agents are increasingly used to close the gap left by end-to-end policy models.<n>Existing harnesses remain largely open-loop, following fixed skills during rollout and reflecting only after an episode completes.<n>We present Zetta, a closed-loop embodied harness that evolves code-based runtime critics and recovery skills online.
  arXiv  Detail & Related papers  (2026-08-17T13:49:43Z)
• EvoHIL: Self-Evolving Reward and Flow-Matched Policy Optimization for Robust Human-in-the-Loop Reinforcement Learning [14.891400906901433]
  EvoHIL is a unified framework that adapts the reward model, action generator, and visual do main within a staged human-in-the-loop learning process.<n>It improves task success, agreement with human-confirmation labels, motion smoothness, and completion time relative to human-in-the-loop and imitation baselines.
  arXiv  Detail & Related papers  (2026-08-04T16:12:05Z)
• Living-Harness Is an Interactive-Agent Evolver [15.081919387954578]
  Large language model (LLM) agents may recover from a failure within an episode or after a retry, yet the same execution failure can recur in later tasks.<n>We propose $textbfLiving-Harness, a self-evolving agent harness that converts each completed trajectory and its evaluator signals into posterior evidence for bounded harness updates.
  arXiv  Detail & Related papers  (2026-07-29T08:20:11Z)
• HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry [35.87794858139959]
  HarnessX is a foundry for composable, adaptive, and evolvable agent harnesses.<n>It assembles typed harness primitives via a substitution algebra, adapts them through AEGIS.<n>It closes the harness-model loop by turning trajectories into both harness updates and model training signal.
  arXiv  Detail & Related papers  (2026-06-12T08:27:11Z)
• Dynamic Mixture of Latent Memories for Self-Evolving Agents [57.20419419302731]
  MoLEM is a generative mixture of latent memory framework based on a dynamic mixture-of-experts (MoE)<n>We train the framework on continual-learning sequences spanning math, science, and code domains.<n>After the full continual-learning sequence, our method improves the average accuracy by 10.40% over the Vanilla pretrained baseline.
  arXiv  Detail & Related papers  (2026-05-21T03:35:10Z)
• CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narratives [117.85963914353904]
  CausalCine is an interactive autoregressive framework that transforms multi-shot video generation into an online directing process.<n>CausalCine generates causally across shot changes, accepts dynamic prompts on the fly, and reuses context without regenerating previous shots.
  arXiv  Detail & Related papers  (2026-05-12T17:59:51Z)
• RADAR: Closed-Loop Robotic Data Generation via Semantic Planning and Autonomous Causal Environment Reset [48.645870795753105]
  We introduce Robust Autonomous Data Acquisition for Robotics (RADAR)<n>RADAR is a fully autonomous, closed-loop data generation engine that completely removes human intervention from the collection cycle.<n>In simulation, our framework achieves up to 90% success rates on complex, long-horizon tasks.
  arXiv  Detail & Related papers  (2026-03-12T11:18:52Z)
• Continual Learning for non-stationary regression via Memory-Efficient Replay [1.5749416770494706]
  We propose the first prototype-based generative replay framework designed for online task-free continual regression.<n>Our approach defines an adaptive output-space discretization model, enabling prototype-based generative replay for continual regression without storing raw data.
  arXiv  Detail & Related papers  (2026-02-10T12:22:59Z)
• UI-S1: Advancing GUI Automation via Semi-online Reinforcement Learning [78.86567400365392]
  We present Semi-online Reinforcement Learning, a novel paradigm that simulates online RL on offline trajectories.<n>To capture long-term training signals, Semi-online RL introduces discounted future returns into the reward computation.<n>Experiments show that ours Semi-online RL achieves SOTA performance among 7B models across four dynamic benchmarks.
  arXiv  Detail & Related papers  (2025-09-15T03:24:08Z)
• Continual Diffuser (CoD): Mastering Continual Offline Reinforcement Learning with Experience Rehearsal [54.93261535899478]
  In real-world applications, such as robotic control of reinforcement learning, the tasks are changing, and new tasks arise in a sequential order.<n>This situation poses the new challenge of plasticity-stability trade-off for training an agent who can adapt to task changes and retain acquired knowledge.<n>We propose a rehearsal-based continual diffusion model, called Continual diffuser (CoD), to endow the diffuser with the capabilities of quick adaptation (plasticity) and lasting retention (stability)
  arXiv  Detail & Related papers  (2024-09-04T08:21:47Z)
• Automating Reinforcement Learning with Example-based Resets [19.86233948960312]
  Existing reinforcement learning algorithms assume an episodic setting in which the agent resets to a fixed initial state distribution at the end of each episode. We propose an extension to conventional reinforcement learning towards greater autonomy by introducing an additional agent that learns to reset in a self-supervised manner. We apply our method to learn from scratch on a suite of simulated and real-world continuous control tasks and demonstrate that the reset agent successfully learns to reduce manual resets.
  arXiv  Detail & Related papers  (2022-04-05T08:12:42Z)
• Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble [135.6115462399788]
  Deep offline reinforcement learning has made it possible to train strong robotic agents from offline datasets. State-action distribution shift may lead to severe bootstrap error during fine-tuning. We propose a balanced replay scheme that prioritizes samples encountered online while also encouraging the use of near-on-policy samples.
  arXiv  Detail & Related papers  (2021-07-01T16:26:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.