Cross-Benchmark Generalization in Long-Horizon Agents

• URL: http://arxiv.org/abs/2608.00181v1
• Date: Fri, 31 Jul 2026 18:05:36 GMT
• Title: Cross-Benchmark Generalization in Long-Horizon Agents
• Abstract summary: We post-train an open-weight mixture-of-experts model on 363 long-horizon Model Context Protocol (MCP) tasks across 27 categories.<n>No externalbenchmark task or grader entered training and no external score informed the reward, training hyper parameters, trained-checkpoint selection, or stopping.<n>Both software benchmarks improve despite the training collection containing no software-engineering tasks.
• Score: 1.0742728354283815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: For reinforcement learning (RL) in self-contained environments, a policy can get rewards by exploiting environment-specific regularities (tool schemas, grader parsing, task templates) rather than by acquiring transferable skill, and an in-distribution holdout shares those regularities. We argue that the discriminating question is behavioral, namely how a trained agent acts, and that cross-benchmark transfer is the right place to look for it. We post-train an open-weight mixture-of-experts model (Qwen3.5-122B-A10B) on 363 long-horizon Model Context Protocol (MCP) tasks across 27 categories, using a two-stage SFT-then-RL pipeline. Toolathlon performance informed the initial base-family and SFT-teacher choices, but no external-benchmark task or grader entered training and no external score informed the reward, training hyperparameters, trained-checkpoint selection, or stopping. At greedy pass@1, the trained model improves over the base on five reported external evaluations: Toolathlon (+9.6 pp), $τ^2$-Bench (+5.3 pp), BFCL-V4 (+3.5 pp), SWE-Bench Pro (+5.8 pp), and Terminal-Bench 2 (+2.8 pp). Both software-engineering benchmarks improve despite the training collection containing no software-engineering tasks. An exploratory paired-trajectory analysis identifies four recurring behavioral differences (more careful local-goal formation, building goal-relevant working state, keeping parent goals stable through local repairs, and verifying completion) that appear in analogous forms across office workflows and code. These results provide descriptive evidence that long-horizon multi-tool post-training can change ways of working that transfer beyond its training domain.

Related papers

• T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks [27.750923745153145]
  T1 is a Mixture-of-Experts model of 122B total trained with reinforcement learning.<n>It operates a real shell in a cloud sandbox for up to 300+ tool-call turns per task, rewarded by executing each task's own verifier.<n>On Terminal-Bench 2.1, our post-train pipeline raises initial base model from 43.8% to T1 with 64.0% resolved.
  arXiv  Detail & Related papers  (2026-09-10T03:42:01Z)
• NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness [76.6324710887112]
  NeoHorse-1 is a family of agent-native models developed to explore this path through agentic post-training.<n>Our system combines a heterogeneous model pool with intelligent routing, recording the predicted capability demand, selected service tier, and subsequent interaction for each user turn.
  arXiv  Detail & Related papers  (2026-09-08T03:14:54Z)
• TRACE: A Self-Evolving Skill Bank for Consistent, Limit-Aware LLM Agents [41.82249291285459]
  TRACE iteratively improves a skill-based agent's behavioral knowledge without modifying model weights.<n>On GPT-5.5, TRACE improves consistency (Pass3) by 34.6 points, from 59.9% to 94.5%, while shrinking the gap between potential and reliable performance to just 4.0 points.<n>These results show that TRACE converts high model potential into stable, consistent performance gain.
  arXiv  Detail & Related papers  (2026-08-24T04:36:03Z)
• LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents [56.902747731541695]
  We present LEGO-RL, a framework that bridges native coding-agent harnesses with scalable policy-gradient optimization.<n>We evaluate LEGO-RL by training the sparse MoE model Qwen3.5-35B-A3B with GSPO across three native coding-agent harnesses.
  arXiv  Detail & Related papers  (2026-08-18T05:34:35Z)
• Post-Training on Office Work Improves Software Engineering: A Behavioral Account of Cross-Domain Transfer [1.0742728354283815]
  Long-horizon tasks require agents to maintain coherent state and goals across nested and branching work.<n>We call this capability goal execution (GDE): the repeated application of four behaviors, namely selecting goals, constructing task-relevant state, maintaining fidelity to higher-level objectives, and verifying completion against the environment.<n>We test this by post-training Qwen3.5-122B-A10B on 363 Long-Horizon Multi-Tool Agent (LHMTA) tasks drawn from office. The collection contained no software-directed tasks, yet the model's pass@1 improved by 5.8 points on SWE-Bench Pro.
  arXiv  Detail & Related papers  (2026-08-03T02:22:02Z)
• TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents [51.523913302114266]
  TRACE (Turn-level Reward Assignment via Credit Estimation) is a dense credit-assignment method for agentic reinforcement learning.<n>It represents rollouts as state transitions at tool-call boundaries, obtains gold-answer log-probabilities from a frozen reference model, transforms them into log-ratio state values, and derives per-action rewards as Temporal-Difference changes in those values.<n>It substantially improves base-model tool-use ability using pure RL, without a cold-start supervised fine-tuning stage, an agentic mid-training stage, or training on live-web data.
  arXiv  Detail & Related papers  (2026-07-15T16:16:42Z)
• KAT-Coder-V2.5 Technical Report [61.907486544595336]
  KAT-Coder-V2.5 is a coding-focused agentic model trained to act autonomously inside real, executable repositories.<n>It reconstructs multilingual repositories into sandboxed environments with fail-to-pass and pass-to-pass verification at scale.<n>We further scale reinforcement learning with harness randomization, a reliability-hardened sandbox, an asymmetric actor--critic PPO with hindsight-augmented value estimation, and a harness-oriented reward framework.
  arXiv  Detail & Related papers  (2026-07-06T08:14:02Z)
• What Makes Interaction Trajectories Effective for Training Terminal Agents? [55.62817294510983]
  Terminal-Lego is a scalable pipeline that transforms real-world issues into environment-verified agentic tasks.<n>Students fine-tuned on trajectories from DeepSeek-V3.2, a lower-scoring agent, exhibit significantly stronger generalization.<n>With only 15.3k Terminal-Lego trajectories, Qwen3-32B achieves a 24.3% score on Terminal-Bench 2.0, rivaling previous SOTA performance established with over 30x the data volume.
  arXiv  Detail & Related papers  (2026-06-02T10:37:47Z)
• SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understanding [41.98672557723593]
  We introduce SWEQA-Pro, a benchmark constructed from diverse, long-tail repositories with executable environments.<n>We also propose a scalable synthetic data pipeline that powers a two-stage training recipe: Supervised Fine-Tuning (SFT) followed by Reinforcement Learning from AI Feedback (RLAIF)<n> Empirically, a Qwen3-8B model trained with our recipe surpasses GPT-4o by 2.3 points on SWE-QA-Pro and substantially narrows the gap to state-of-the-art models.
  arXiv  Detail & Related papers  (2026-03-17T05:12:48Z)
• EnterpriseBench Corecraft: Training Generalizable Agents on High-Fidelity RL Environments [0.10934862523101825]
  We show that training AI agents on high-fidelity reinforcement learning environments produces capabilities that generalize beyond the training distribution.<n>We introduce CoreCraft, the first environment in EnterpriseBench, Surge AI's suite of agentic RL environments.
  arXiv  Detail & Related papers  (2026-02-18T04:35:46Z)
• Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation [65.3648667980258]
  Vision-language model (VLM) based GUI agents show promise for automating complex tasks, but face significant challenges in applying reinforcement learning (RL)<n>We propose DART, a Decoupled Agentic RL Training framework for GUI agents, which coordinates heterogeneous modules in a highly decoupled manner.<n>On the OSWorld benchmark, DART-GUI-7B achieves a 42.13% task success rate, a 14.61% absolute gain over the base model, and 7.34% higher than open-source SOTA.
  arXiv  Detail & Related papers  (2025-09-28T13:19:20Z)
• Agentic Reinforcement Learning with Implicit Step Rewards [92.26560379363492]
  Large language models (LLMs) are increasingly developed as autonomous agents using reinforcement learning (agentic RL)<n>We introduce implicit step rewards for agentic RL (iStar), a general credit-assignment strategy that integrates seamlessly with standard RL algorithms.<n>We evaluate our method on three challenging agent benchmarks, including WebShop and VisualSokoban, as well as open-ended social interactions with unverifiable rewards in SOTOPIA.
  arXiv  Detail & Related papers  (2025-09-23T16:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.