OpenForgeRL: Train Harness-native Agents in Any Environment

• URL: http://arxiv.org/abs/2607.21557v2
• Date: Fri, 24 Jul 2026 02:46:29 GMT
• Title: OpenForgeRL: Train Harness-native Agents in Any Environment
• Abstract summary: We present OpenForgeRL, an open-source framework for training harness-based agents end-to-end in diverse environments.<n>By decoupling inference, OpenForgeRL allows researchers to easily train, study, and improve agents directly in the real harnesses and environments they are deployed with.
• Score: 60.23309689179493
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern AI agents rely on elaborate inference harnesses such as Claude Code, Codex, and OpenClaw to drive multi-turn reasoning, tool use, and access to external systems. While powerful, these complex harnesses also make agents hard to train end-to-end with open infrastructure, whose SFT/RL stacks cannot natively express stateful, multi-process harness inference. To address this, we present OpenForgeRL, an open-source framework for training harness-based agents end-to-end in diverse environments. OpenForgeRL achieves this with a lightweight proxy that serves the harness's model calls while recording them as training data for a standard RL codebase (e.g., veRL), and a Kubernetes orchestrator that runs each rollout in its own remote container, together enabling training on any harness in any environment at scale. By decoupling training and inference, OpenForgeRL allows researchers to easily train, study, and improve agents directly in the real harnesses and environments they are deployed with. We validate our framework across diverse, complex harnesses and environments, spanning tool/claw-based agents and multimodal GUI browser- and computer-use agents. Using only hundreds to a few thousand tasks, OpenForgeClaw reaches 31.7 pass^3 and 55.9 pass@3 on ClawEval and 33.7 on QwenClawBench. OpenForgeGUI reaches 37.7 on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager. Both outperform open baselines of similar size on nearly all benchmarks, and in the GUI setting match or surpass models several times larger. Beyond benchmarks, we analyze how harness choice (e.g., ZeroClaw, OpenClaw, Codex) and RL shape agent behavior. We find that some harnesses are substantially harder to learn than others, and that RL improves agentic reliability, such as self-verification, tool coverage, and completing multi-step plans, though critical abilities such as error recovery remain weak.

Related papers

• LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents [56.902747731541695]
  We present LEGO-RL, a framework that bridges native coding-agent harnesses with scalable policy-gradient optimization.<n>We evaluate LEGO-RL by training the sparse MoE model Qwen3.5-35B-A3B with GSPO across three native coding-agent harnesses.
  arXiv  Detail & Related papers  (2026-08-18T05:34:35Z)
• SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL [38.05711358650287]
  Computer use agents (CUAs) are emerging as a powerful interface for automating complex digital through visual perception and GUI execution.<n>Online reinforcement learning with verifiable rewards (RLVR) has emerged as a key direction for scaling their capabilities.<n>We introduce ScaleCUA, a unified framework that scales online RL for CUAs via verifiable task synthesis and efficient training.
  arXiv  Detail & Related papers  (2026-07-13T07:32:35Z)
• ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning [91.51460129144654]
  We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling.<n>We show that ParaVT improves over the Qwen3-VL baseline by +7.9% on average, with PARA-GRPO lifting training-time format compliance from 0.13 to 0.64.
  arXiv  Detail & Related papers  (2026-05-19T18:01:26Z)
• OpenClaw-RL: Train Any Agent Simply by Talking [54.06773485601523]
  Next-state signals are universal, and policy can learn from all of them simultaneously.<n>Personal conversations, terminal executions, GUI interactions, SWE tasks, and tool-call traces are not separate training problems.<n>OpenClaw-RL enables an agent to improve simply by being used, recovering conversational signals from user re-queries, corrections, and explicit feedback.
  arXiv  Detail & Related papers  (2026-03-10T18:59:01Z)
• INTELLECT-3: Technical Report [5.3998786788822]
  INTELLECT-3 is a Mixture-of-Experts model (12B active) trained with large-scale reinforcement learning.<n>We open-source the model together with the full infrastructure stack used to create it, including RL frameworks.<n>We introduce prime-rl, an open framework for large-scale asynchronous reinforcement learning.
  arXiv  Detail & Related papers  (2025-12-18T03:57:01Z)
• RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments [111.87296453908199]
  We introduce Reinforcement Learning with Adaptive Verifiable Environments (RLVE)<n>RLVE enables each verifiable environment to dynamically adapt its problem difficulty distribution to the policy model's capabilities as training progresses.<n>We show that environment scaling, i.e., expanding the collection of training environments, consistently improves reasoning capabilities.
  arXiv  Detail & Related papers  (2025-11-10T17:18:35Z)
• RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control [7.259696592534715]
  Deep Reinforcement Learning (RL) can yield capable agents and control policies in several domains but is commonly plagued by prohibitively long training times. We present RLtools, a dependency-free, header-only, pure C++ library for deep supervised and reinforcement learning.
  arXiv  Detail & Related papers  (2023-06-06T09:26:43Z)
• ElegantRL-Podracer: Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning [141.58588761593955]
  We present a library ElegantRL-podracer for cloud-native deep reinforcement learning. It efficiently supports millions of cores to carry out massively parallel training at multiple levels. At a low-level, each pod simulates agent-environment interactions in parallel by fully utilizing nearly 7,000 GPU cores in a single GPU.
  arXiv  Detail & Related papers  (2021-12-11T06:31:21Z)
• Godot Reinforcement Learning Agents [10.413185820687021]
  The Godot RL Agents interface allows the design, creation and learning of agent behaviors in challenging 2D and 3D environments. The framework is a versatile tool that allows researchers and game designers the ability to create environments with discrete, continuous and mixed action spaces.
  arXiv  Detail & Related papers  (2021-12-07T11:24:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.