T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks

• URL: http://arxiv.org/abs/2609.11042v1
• Date: Thu, 10 Sep 2026 03:42:01 GMT
• Title: T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
• Abstract summary: T1 is a Mixture-of-Experts model of 122B total trained with reinforcement learning.<n>It operates a real shell in a cloud sandbox for up to 300+ tool-call turns per task, rewarded by executing each task's own verifier.<n>On Terminal-Bench 2.1, our post-train pipeline raises initial base model from 43.8% to T1 with 64.0% resolved.
• Score: 27.750923745153145
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agent usage is shifting toward long-horizon tasks such as coding and scientific discovery, among which terminal tasks are especially important. We introduce T1, a Mixture-of-Experts model of 122B total trained with reinforcement learning, operating a real shell in a cloud sandbox for up to 300+ tool-call turns per task, rewarded by executing each task's own verifier. We provide a comprehensive recipe: First, an aggressively warm-started to stabilize actor-critic training, with a dense process reward scoring trajectories by the absolute number of passing verifiers. Second, stable optimization through TITO construction, training on the exact sampled token identifiers with drift repair at turn boundaries, and rollout routing replay, recording the sampler's per-token expert choices at every MoE layer and replaying them during training. Third, fully out-of-distribution training corpus: isolated seeds and synthesized tasks disjoint from Terminal-Bench 2.1 ensures gains reflect genuine capability transfer over benchmark overfitting. Together, TITO and R3 cut the training-to-inference log-probability difference from 0.021 to 0.013, with exactly aligned zero token drift in the loss region. On Terminal-Bench 2.1, our post-train pipeline raises initial base model from 43.8% to T1 with 64.0% resolved. On Long-Horizon Terminal Bench, T1 reaches 27.9% and surpasses GPT-5.4 and GLM-5.1.

Related papers

• PROOF-Gen: From Optimized Data to Better Distillation [6.158612515104146]
  Post-training pipelines that drive tool-calling agents re-run this stage on a daily or weekly cadence, paying the frontier-teacher cost each cycle.<n>On 2-bench, 57% of teacher trials fail, two-thirds of them near-misses.<n>We introduce PROOF-Gen, which recovers golden trajectories from these failures via per-scenario prompt optimization.
  arXiv  Detail & Related papers  (2026-08-24T23:33:56Z)
• Recursive Synthesis for Long-Horizon Terminal Tasks [50.39352383646813]
  Recursive Synthetic Terminal Tasks (RST) is a framework for constructing long-horizon terminal-agent tasks at scale.<n>RST produces 37,484 synthesized terminal-agent tasks at roughly $0.05 per task.<n>Fine-tuning on trajectories improves Qwen3.5-27B and Qwen3.5-122B-A10B by up to 10 points on Terminal-Bench 2, Terminal-Bench Hard, and Long-Horizon Terminal Bench.
  arXiv  Detail & Related papers  (2026-08-05T23:24:26Z)
• Cross-Benchmark Generalization in Long-Horizon Agents [1.0742728354283815]
  We post-train an open-weight mixture-of-experts model on 363 long-horizon Model Context Protocol (MCP) tasks across 27 categories.<n>No externalbenchmark task or grader entered training and no external score informed the reward, training hyper parameters, trained-checkpoint selection, or stopping.<n>Both software benchmarks improve despite the training collection containing no software-engineering tasks.
  arXiv  Detail & Related papers  (2026-07-31T18:05:36Z)
• SETA: Scaling Environments for Terminal Agents [49.686781072283814]
  We propose SETA, a scalable framework for generating verifiable terminal environments for reinforcement learning.<n>We release SETA-Env, the largest open-source verifiable terminal RL dataset to date, containing over 4,500 environments.
  arXiv  Detail & Related papers  (2026-07-12T19:40:27Z)
• What Makes Interaction Trajectories Effective for Training Terminal Agents? [55.62817294510983]
  Terminal-Lego is a scalable pipeline that transforms real-world issues into environment-verified agentic tasks.<n>Students fine-tuned on trajectories from DeepSeek-V3.2, a lower-scoring agent, exhibit significantly stronger generalization.<n>With only 15.3k Terminal-Lego trajectories, Qwen3-32B achieves a 24.3% score on Terminal-Bench 2.0, rivaling previous SOTA performance established with over 30x the data volume.
  arXiv  Detail & Related papers  (2026-06-02T10:37:47Z)
• Terminal-World: Scaling Terminal-Agent Environments via Agent Skills [52.39713754337834]
  We introduce Terminal-World, a fully automated pipeline that uses agent skills as the central synthesis primitive.<n>We construct 5,723 training environments and train Terminal-World-8B/14B/32B, evaluated across 6 benchmarks.<n>Terminal-World-32B surpasses Nemotron-Terminal-32B on Terminal-Bench 2.0 by +4.5 Pass@1 (31.5) and achieves 43.8 Pass@3.
  arXiv  Detail & Related papers  (2026-05-20T08:14:51Z)
• On Data Engineering for Scaling LLM Terminal Capabilities [62.14352406328365]
  Training data strategies behind state-of-the-art terminal agents remain largely undisclosed.<n>We address this gap through a systematic study of data engineering practices for terminal agents.<n>Our pipeline yields Terminal-Corpus, a large-scale open-source dataset for terminal tasks.
  arXiv  Detail & Related papers  (2026-02-24T18:51:04Z)
• Endless Terminals: Scaling RL Environments for Terminal Agents [39.60665149203152]
  Endless Terminals is a fully autonomous pipeline that procedurally generates terminal-use tasks without human annotation.<n>We train agents using vanilla PPO with binary episode level rewards and a minimal interaction loop.<n>These improvements transfer to human-curated benchmarks.
  arXiv  Detail & Related papers  (2026-01-23T04:39:55Z)
• Agentic Reinforcement Learning for Real-World Code Repair [7.512134741776294]
  We tackle the challenge of training reliable code-fixing agents in real repositories.<n>We developed a verifiable pipeline with success defined as post-fix build validation.<n>We introduced a scalable simplified pipeline for large-scale reinforcement learning.
  arXiv  Detail & Related papers  (2025-10-24T23:25:02Z)
• Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Model [100.86587937568832]
  Ring-1T is the first open-source, state-of-the-art thinking model with a trillion-scale parameter.<n>It features 1 trillion total parameters and activates approximately 50 billion per token.
  arXiv  Detail & Related papers  (2025-10-21T17:46:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.