Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces

• URL: http://arxiv.org/abs/2603.06713v1
• Date: Thu, 05 Mar 2026 20:29:12 GMT
• Title: Scaling Agentic Capabilities, Not Context: Efficient Reinforcement Finetuning for Large Toolspaces
• Abstract summary: ATLAS is a reinforcement finetuning framework that enables small language models to operate effectively in large-scale toolspace environments.<n>First, we treat context control and execution structure as learnable decisions.<n>Second, we propose reinforcement finetuning, which decomposes task success into structured, task-aligned criteria.
• Score: 5.870572929943109
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic systems operating over large tool ecosystems must plan and execute long-horizon workflows under weak or non-verifiable supervision. While frontier models mitigate these challenges through scale and large context budgets, small language models (SLMs) remain brittle: eager tool loading saturates context, execution errors compound over time, and sparse rewards limit learning. We introduce ATLAS, a reinforcement finetuning framework that enables SLMs to operate effectively in large-scale toolspace environments by learning how to acquire context and how to execute actions. Our approach makes two key contributions. First, we treat context control and execution structure as learnable decisions, combining iterative tool loading with programmatic tool orchestration to bound context growth and stabilize long-horizon trajectories. Second, we propose rubric-based reinforcement finetuning, which decomposes task success into structured, task-aligned criteria and enables scalable training using small judge models. Across MCP benchmarks, these design choices yield large and consistent gains over generic RL baselines, allowing a 4B SLM to approach frontier-agent performance under far tighter parameter and context budgets.

Related papers

• ToolVerse: Unlocking Massive Environments and Long-Horizon Tasks for Agentic Reinforcement Learning [6.90417084681273]
  ToolVerse is a framework that scales up agentic RL environments and enables agents to perform complex long-horizon reasoning.<n>First, ToolVerse automatically builds the massive executable agent training environments from nearly 400 real-world Model Context Protocols.<n>Third, to alleviate the credit assigment problem in long-horizon agentic RL, we propose a fine-grained Turn-Aware Relative Advantage algorithm.
  arXiv  Detail & Related papers  (2026-07-17T06:12:04Z)
• From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents [90.3279207163486]
  EvoSOP is a framework that empowers agents to extract SOPs from execution trajectories and iteratively optimize the toolset.<n>We show that EvoSOP significantly boosts task success rates while substantially reducing the number of interaction rounds.
  arXiv  Detail & Related papers  (2026-07-08T12:09:49Z)
• AHD Agent: Agentic Reinforcement Learning for Automatic Heuristic Design [5.7547035595428255]
  Automatic design (AHD) has emerged as a promising paradigm for solving NP-hard optimization problems.<n>Existing AHD frameworks typically treat large language models (LLMs) as passive generators within fixed contexts.<n>We propose AHD Agent, a novel tool-integrated, multi-turn framework that empowers LLMs to proactively decide whether to generate tools or retrieve targeted evidence.
  arXiv  Detail & Related papers  (2026-05-09T07:36:45Z)
• Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning [32.56971325545821]
  Language model (LM)-based agents have demonstrated promising capabilities in automating complex tasks from natural language instructions.<n>We propose an enhanced multi-agent framework that decomposes automation into three roles: a planner for high-level decision-making, an actor for task execution, and a memory manager for contextual reasoning.
  arXiv  Detail & Related papers  (2026-05-04T02:58:05Z)
• Long-Horizon Plan Execution in Large Tool Spaces through Entropy-Guided Branching [21.203929965634853]
  Large Language Models (LLMs) have significantly advanced tool-augmented agents, enabling autonomous reasoning via API interactions.<n> executing multi-step tasks within massive tool libraries remains challenging due to two critical bottlenecks.<n>We first introduce SLATE, a large-scale context-aware benchmark designed for the automated assessment of tool-integrated agents.<n>We propose Entropy-Guided Branching (EGB), an uncertainty-aware search algorithm that dynamically expands decision branches where predictive entropy is high.
  arXiv  Detail & Related papers  (2026-04-13T23:14:32Z)
• AgentSwing: Adaptive Parallel Context Management Routing for Long-Horizon Web Agents [115.18197165538943]
  Large language models (LLMs) evolve into autonomous agents for long-horizon information-seeking.<n>Existing context management methods typically commit to a single fixed strategy throughout the entire trajectory.<n>We introduce a probabilistic framework that characterizes long-horizon success through two complementary dimensions: search efficiency and terminal precision.
  arXiv  Detail & Related papers  (2026-03-29T02:59:27Z)
• MagicAgent: Towards Generalized Agent Planning [73.21129030631421]
  We present textbfMagicAgent, a series of foundation models specifically designed for generalized agent planning.<n>We introduce a lightweight and scalable synthetic data framework that generates high-quality trajectories across diverse planning tasks.<n>We show that MagicAgent-32B and MagicAgent-30B-A3B achieve superior performance across diverse open-source benchmarks.
  arXiv  Detail & Related papers  (2026-02-22T01:39:16Z)
• ToolSelf: Unifying Task Execution and Self-Reconfiguration via Tool-Driven Intrinsic Adaptation [60.25542764389203]
  Agentic systems powered by Large Language Models (LLMs) have demonstrated remarkable potential in tackling complex, long-horizon tasks.<n>Existing approaches, relying on manual orchestration or runtime-based patches, often struggle with poor generalization and fragmented optimization.<n>We propose ToolSelf, a novel paradigm enabling tool-driven self-readjustment.
  arXiv  Detail & Related papers  (2026-02-08T09:27:18Z)
• ASTRA: Automated Synthesis of agentic Trajectories and Reinforcement Arenas [13.919124676472022]
  ASTRA is an end-to-end framework for training tool-augmented language model agents.<n>ASTRA integrates scalable data synthesis and verifiable reinforcement learning.<n> Experiments on multiple agentic tool-use benchmarks demonstrate that ASTRA-trained models achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-01-29T11:22:23Z)
• Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models [78.73992315826035]
  We introduce Youtu-LLM, a lightweight language model that harmonizes high computational efficiency with native agentic intelligence.<n>Youtu-LLM is pre-trained from scratch to systematically cultivate reasoning and planning capabilities.
  arXiv  Detail & Related papers  (2025-12-31T04:25:11Z)
• GAP: Graph-Based Agent Planning with Parallel Tool Use and Reinforcement Learning [20.75113227786218]
  Graph-based Agent Planning (GAP) is a novel framework that explicitly models inter-task dependencies through graph-based planning.<n>Our approach trains agent foundation models to decompose complex tasks into dependency-aware sub-task graphs.<n>This dependency-aware orchestration achieves substantial improvements in both execution efficiency and task accuracy.
  arXiv  Detail & Related papers  (2025-10-29T09:35:55Z)
• Scaling Long-Horizon LLM Agent via Context-Folding [46.685552398338295]
  We introduce Context-Folding, a framework that empowers agents to actively manage their working context.<n>An agent can procedurally branch into a sub-trajectory to handle a subtask and then fold it upon completion, collapsing the intermediate steps while retaining a concise summary of the outcome.
  arXiv  Detail & Related papers  (2025-10-13T22:00:58Z)
• Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Management [19.980762483472354]
  We introduce summarization-based context management to training.<n>We instantiate this framework with underlineSUmmarization augmented underlinePolicy underlineOptimization (textttSUPO)<n>Our results establish summarization-based context management as a principled and scalable approach for training RL agents beyond a fixed context length limit.
  arXiv  Detail & Related papers  (2025-10-08T07:29:22Z)
• Structured Agent Distillation for Large Language Model [56.38279355868093]
  We propose Structured Agent Distillation, a framework that compresses large LLM-based agents into smaller student models.<n>Our method segments trajectories into [REASON] and [ACT] spans, applying segment-specific losses to align each component with the teacher's behavior.<n>Experiments on ALFWorld, HotPotQA-ReAct, and WebShop show that our approach consistently outperforms token-level and imitation learning baselines.
  arXiv  Detail & Related papers  (2025-05-20T02:01:55Z)
• TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use [72.32614703504122]
  Large language models (LLMs) achieve remarkable advancements by leveraging tools to interact with environments.<n>Standard supervised fine-tuning approach, which relies on large-scale datasets, often overlooks task-specific characteristics in tool use.<n>We propose TL-Training, a task-feature-based framework that mitigates the effects of suboptimal training data.
  arXiv  Detail & Related papers  (2024-12-20T02:21:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.