ClawGym II: Exploring Black-Box RL on Agent Harness

• URL: http://arxiv.org/abs/2608.16798v1
• Date: Mon, 17 Aug 2026 16:53:03 GMT
• Title: ClawGym II: Exploring Black-Box RL on Agent Harness
• Abstract summary: Agent harnesses have substantially improved performance on long-horizon tasks by coordinating agent interactions with the environment.<n>We present a unified black-box RL framework for stable and scalable optimization of general agents through complex harnesses.
• Score: 82.92963070856416
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Agent harnesses have substantially improved performance on long-horizon tasks by coordinating agent interactions with the environment. However, reinforcement learning through complex harnesses remains largely unexplored, as scaling such training to long-horizon agent tasks introduces fundamental challenges. In this work, we present a unified black-box RL framework for stable and scalable optimization of general agents through complex harnesses. Concretely, we first build a sandbox-based execution infrastructure that isolates task environments and harnesses within temporary sandboxes for large-scale concurrent rollouts. We then decouple policy optimization from opaque harness execution and place a serving proxy at the model boundary to capture model calls. To reconstruct multi-turn trajectories and improve training efficiency, we organize the captured calls into prefix trees and further adapt both critic-based PPO and critic-free GRPO to optimize over the recovered tree structure. Meanwhile, we maintain training-inference consistency throughout the optimization process. Finally, we introduce mix-harness training, allowing a single model to be jointly optimized by heterogeneous harnesses. With Qwen3-30A3B, black-box RL improves Pass@1 on ClawGym-Bench by 9.98 and 14.81 points through OpenClaw and Claude Code, respectively, while remaining stable over 200-400 optimization steps. Moreover, the framework yields consistent gains on more challenging tasks such as JobBench and OfficeQA. Overall, our framework enables effective, stable, and scalable optimization of general agents through black-box harnesses, supporting unified training across heterogeneous execution systems.

Related papers

• ForgeTrain: Forging Production-Grade Training Frameworks via Harness-Driven AI Development [31.93593134525649]
  Training large models still relies on general-purpose frameworks such as Megatron-LM.<n>We propose Forge Engineering: building a dedicated implementation from scratch for each scenario.<n>We instantiate this paradigm for training frameworks as ForgeTrain, which holds a trusted framework as a golden reference.
  arXiv  Detail & Related papers  (2026-09-12T01:47:57Z)
• ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search [54.43696074323815]
  ZGCM-1 is a fully open 7B foundation model trained from scratch with extreme data, system, and algorithmic efficiency.<n>We establish an AI-native R&D workflow where agent swarms autonomously manage cluster operations.<n>ZGCM-1-7B is competitive across 7B model family on general benchmarks.
  arXiv  Detail & Related papers  (2026-09-11T17:18:04Z)
• Self-Evolving Embodied Agents via Skill-Harness Evolution [53.307052568223945]
  Embodied agents are increasingly built as systems around foundation models.<n>While supervised fine-tuning and reinforcement learning can adapt agents to new environments, they require additional data, rewards, and training runs.<n>We propose SHAPER, a self-evolving framework for train-free embodied adaptation that keeps model parameters frozen.
  arXiv  Detail & Related papers  (2026-08-11T18:55:58Z)
• Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents [37.91666123749406]
  Co-Harness is a framework that jointly optimize the agent harness and model parameters during post-training.<n>An autonomous case study shows that Co-Harness can recover from system crashes, improve inference efficiency, and discover ensemble strategies without human intervention.
  arXiv  Detail & Related papers  (2026-07-17T02:39:57Z)
• Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning [47.00572734047258]
  We present Single-rollout Asynchronous Optimization (SAO) to address the stability and off-policy challenges in asynchronous RL.<n>SAO replaces group-wise sampling with single-rollout sampling, that is, using one rollout per prompt.<n>SAO is successfully deployed in the agentic RL pipeline for training the open GLM-5.2 model.
  arXiv  Detail & Related papers  (2026-07-08T15:02:19Z)
• A Comparative Study of Bayesian Contextual Bandits for Real-Time Warehouse Sorter Optimization [1.8807636852384417]
  In this study, we use an inbound receiving sorter at a high-volume e-commerce warehouse as our primary use case.<n>To address this real-time sorter diversion optimization challenge, we conducted a comparative study of three candidate hybrid machine learning frameworks.<n>Our results demonstrate that while tree-based reward models offer slightly better predictive power, the BCB framework achieved overall higher performance with 2.03% reward uplift over the baseline.
  arXiv  Detail & Related papers  (2026-06-22T22:07:36Z)
• Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control [50.28263951510334]
  We propose a hybrid hierarchical control framework for reactive dexterous grasping.<n>The proposed framework explicitly decouples high-level spatial intent from low-level joint execution.<n>We extensively validate the proposed framework through a rigorous simulation-to-reality pipeline.
  arXiv  Detail & Related papers  (2026-05-05T04:49:38Z)
• ClawGym: A Scalable Framework for Building Effective Claw Agents [47.47444724291439]
  We present ClawGym, a scalable framework that supports the full lifecycle of Claw-style personal agent development.<n>ClawGym-SynData is a dataset of 13.5K filtered tasks synthesized from persona-driven intents and skill-grounded operations.<n>We then train a family of capable Claw-style models, termed ClawGym-Agents, through supervised fine-tuning on black-box rollout trajectories.<n>We further construct ClawGym-Bench, a benchmark of 200 instances through automated filtering and human-LLM review.
  arXiv  Detail & Related papers  (2026-04-29T17:12:22Z)
• HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving [51.268878540511054]
  We propose HAD, an end-to-end planning framework with a Hierarchical Diffusion Policy.<n>We show that HAD achieves new state-of-the-art performance on both NAVSIM and HUGSIM.
  arXiv  Detail & Related papers  (2026-04-04T04:12:47Z)
• ARLArena: A Unified Framework for Stable Agentic Reinforcement Learning [75.73135757250806]
  Agentic reinforcement learning (ARL) has rapidly gained attention as a promising paradigm for training agents to solve complex, multi-step interactive tasks.<n>Despite encouraging early results, ARL remains highly unstable, often leading to training collapse.<n>In this paper, we first propose ARLArena, a stable training recipe and systematic analysis framework that examines training stability in a controlled and reproducible setting.
  arXiv  Detail & Related papers  (2026-02-25T03:43:34Z)
• DiRL: An Efficient Post-Training Framework for Diffusion Language Models [54.405206032785706]
  Diffusion Language Models (dLLMs) have emerged as promising alternatives to Auto-Regressive (AR) models.<n>Existing methods suffer from computational inefficiency and objective mismatches between training and inference.<n>We introduce DiRL, an efficient post-training framework that tightly integrates FlexAttention-accelerated blockwise training with LMDeploy-optimized inference.
  arXiv  Detail & Related papers  (2025-12-23T08:33:19Z)
• DSPO: Stable and Efficient Policy Optimization for Agentic Search and Reasoning [10.663331399616956]
  We introduce textbfDynamic-filter textbfSequence-level textbfPolicy textbfOptimization (DSPO), an improved RL algorithm for robust agent training.<n>We train our model purely through RL to interleave multi-turn search and reasoning, obviating the need for supervised demonstration data.
  arXiv  Detail & Related papers  (2025-10-10T10:53:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.