CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents

• URL: http://arxiv.org/abs/2605.25624v1
• Date: Mon, 25 May 2026 09:28:03 GMT
• Title: CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents
• Abstract summary: Reinforcement learning with verifiable rewards (RLVR) has driven breakthroughs in domains such as math, tool-use, and software engineering.<n>But its extension to computer-use agents (CUAs) has been bottlenecked by the scarcity of scalable training data with deterministic rewards.<n>We present CUA-Gym, a scalable pipeline that co-generates task instructions, environment states, and reward functions.
• Score: 64.53357162213601
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning with verifiable rewards (RLVR) has driven breakthroughs in domains such as math, tool-use, and software engineering, yet its extension to computer-use agents (CUAs) has been bottlenecked by the scarcity of scalable training data with deterministic rewards. Constructing such data for CUAs requires consistent task instruction, executable environment, and verifiable reward. However, hand-curated benchmarks achieve high reward fidelity but cover few applications and LLM-as-judge-based datasets scale broadly but lack reliable verification. We present CUA-Gym, a scalable pipeline that co-generates task instructions, environment states, and reward functions. Concretely, a Generator agent constructs the initial and golden environment states, and a separate Discriminator agent writes the reward function from the task specification. An orchestrator agent drives the two through iterative rounds upon execution. Generated tuples then pass a final filter combining LLM majority voting and agent rollouts, ensuring quality beyond the per-task adversarial loop. To address the scarcity of training environments, we further synthesize CUA-Gym-Hub, a broad suite of high-fidelity mock web applications grounded in real-world software-use distributions, expanding the scale of CUA RLVR data by magnitude. Using this pipeline, we construct CUA-Gym, a dataset of 32,112 verified RLVR training tuples grounded in 110 environments. Trained with GSPO on CUA-Gym, our CUA-Gym-A3B and CUA-Gym-A17B achieve 62.1% and 72.6% on OSWorld-Verified, outperforming prior open-source CUAs at comparable scales, with performance scaling smoothly in both data volume and environment diversity. The same checkpoints also improve on the held-out WebArena benchmark, indicating transfer beyond the training environments. We will open-source the full synthesis pipeline, dataset, CUA-Gym-Hub environments, and models.

Related papers

• AgentBrew: Offline Tool-Use Agent Learning from Raw Real-World Trajectories [36.7302669545557]
  Real-world applications provide no pre-defined tasks or verifiers, no faithful simulators, and limited budget for large-scale environment interaction.<n>We propose textbfAgentBrew, an offline training framework that learns effective tool-use policies from a single batch of raw interaction trajectories.<n>AgentBrew improves Qwen3-32B by +8.7 Acc / +9.7 Score on average, surpassing Qwen3-235B (+2.3 / +4.4) and outperforming rejection sampling.
  arXiv  Detail & Related papers  (2026-09-05T03:04:23Z)
• SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL [38.05711358650287]
  Computer use agents (CUAs) are emerging as a powerful interface for automating complex digital through visual perception and GUI execution.<n>Online reinforcement learning with verifiable rewards (RLVR) has emerged as a key direction for scaling their capabilities.<n>We introduce ScaleCUA, a unified framework that scales online RL for CUAs via verifiable task synthesis and efficient training.
  arXiv  Detail & Related papers  (2026-07-13T07:32:35Z)
• ClawGym: A Scalable Framework for Building Effective Claw Agents [47.47444724291439]
  We present ClawGym, a scalable framework that supports the full lifecycle of Claw-style personal agent development.<n>ClawGym-SynData is a dataset of 13.5K filtered tasks synthesized from persona-driven intents and skill-grounded operations.<n>We then train a family of capable Claw-style models, termed ClawGym-Agents, through supervised fine-tuning on black-box rollout trajectories.<n>We further construct ClawGym-Bench, a benchmark of 200 instances through automated filtering and human-LLM review.
  arXiv  Detail & Related papers  (2026-04-29T17:12:22Z)
• CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation [51.72529978689561]
  Agent is a large-scale agentic reinforcement learning system that develops kernel expertise through three components.<n>Agent delivers 100%, 100%, and 92% faster rate over torchcompile on KernelBench.
  arXiv  Detail & Related papers  (2026-02-27T18:58:05Z)
• SWE-Universe: Scale Real-World Verifiable Environments to Millions [84.63665266236963]
  SWE-Universe is a framework for automatically constructing real-world software engineering (SWE) verifiable environments from GitHub pull requests (PRs)<n>We propose a building agent powered by an efficient custom-trained model to overcome the prevalent challenges of automatic building.<n>We demonstrate the profound value of our environments through large-scale agentic mid-training and reinforcement learning.
  arXiv  Detail & Related papers  (2026-02-02T17:20:30Z)
• VibeTensor: System Software for Deep Learning, Fully Generated by AI Agents [42.56489784841984]
  "fully generated" refers to code provenance: implementation changes were produced and applied as agent-proposed diffs.<n>We describe the architecture, summarize the workflow used to produce and validate the system, and evaluate the artifact.
  arXiv  Detail & Related papers  (2026-01-21T19:29:00Z)
• Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Control [61.155940786140455]
  Reinforcement learning (RL) has shown promising results in active flow control (AFC)<n>Current AFC benchmarks rely on external computational fluid dynamics (CFD) solvers, are not fully differentiable, and provide limited 3D and multi-agent support.<n>We introduce FluidGym, the first standalone, fully differentiable benchmark suite for RL in AFC.
  arXiv  Detail & Related papers  (2026-01-21T14:13:44Z)
• Towards Agentic Self-Learning LLMs in Search Environment [36.158823302039195]
  We study whether self-learning can scale LLM-based agents without relying on human-curated datasets or predefined rule-based rewards.<n>We find that rewards from a Generative Reward Model (GRM) outperform rigid rule-based signals for open-domain learning.<n>We propose textbfAgentic Self-Learning (ASL), a fully closed-loop, multi-role reinforcement learning framework.
  arXiv  Detail & Related papers  (2025-10-16T03:11:56Z)
• OpenCUA: Open Foundations for Computer-Use Agents [74.61449905487565]
  Vision-language models have demonstrated impressive capabilities as computer-use agents (CUAs)<n>We propose OpenCUA, a comprehensive open-source framework for scaling CUA data and foundation models.<n>Our end-to-end agent models demonstrate strong performance across CUA benchmarks.
  arXiv  Detail & Related papers  (2025-08-12T17:52:32Z)
• FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression [55.992528247880685]
  Decentralized training faces significant challenges regarding system design and efficiency. We present FusionLLM, a decentralized training system designed and implemented for training large deep neural networks (DNNs) We show that our system and method can achieve 1.45 - 9.39x speedup compared to baseline methods while ensuring convergence.
  arXiv  Detail & Related papers  (2024-10-16T16:13:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.