AdaTIR: Adaptive Tool-Integrated Reasoning via Difficulty-Aware Policy Optimization

• URL: http://arxiv.org/abs/2601.14696v1
• Date: Wed, 21 Jan 2026 06:18:46 GMT
• Title: AdaTIR: Adaptive Tool-Integrated Reasoning via Difficulty-Aware Policy Optimization
• Abstract summary: Current agents exhibit cognitive offloading, redundantly invoking external tools even for simple tasks.<n>We propose a framework that shifts the paradigm from static tool invocation to difficulty-aware reasoning internalization.<n>AdaTIR successfully internalizes reasoning, outperforming baselines by 4.8% on AIME 2024 even when tool access is strictly disabled.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tool-Integrated Reasoning (TIR) has significantly enhanced the capabilities of Large Language Models (LLMs), yet current agents tend to exhibit cognitive offloading, redundantly invoking external tools even for simple tasks. In this paper, we suggest that true agentic intelligence requires not just tool invocation, but the adaptive wisdom to discern when to use them. We propose AdaTIR, a framework that shifts the paradigm from static tool invocation to difficulty-aware reasoning internalization. By introducing a difficulty-aware efficiency reward, AdaTIR dynamically adjusts tool budgets based on task complexity--internalizing reasoning for simple tasks while selectively invoking tools for complex tasks. Furthermore, we identify a sign reversal problem where tool penalties outweigh correctness rewards, mistakenly penalizing correct rollouts with negative advantages. To resolve this, we propose Clipped Advantage Shaping (CAS), which ensures that correctness remains the primary objective while using efficiency as a secondary constraint. Empirical results demonstrate that AdaTIR reduces tool calls by up to 97.6% on simple tasks and 28.2% on complex challenges while maintaining or enhancing accuracy. Notably, AdaTIR successfully internalizes reasoning, outperforming baselines by 4.8% on AIME 2024 even when tool access is strictly disabled.

Related papers

• Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act [38.40533345940731]
  Large language model (LLM) agents increasingly interleave natural language reasoning with external tools.<n>We study when and why RL-trained agents learn shortcut tool-selection policies.
  arXiv  Detail & Related papers  (2026-09-14T19:31:04Z)
• HiDiffTIR: Hierarchical Difficulty-Aware Policy Optimization for Multi-Turn Tool-Integrated Reasoning [95.29291773706224]
  Tool-Integrated Reasoning is a capability for LLM agents to solve complex tasks by interacting with external tools iteratively.<n>Existing approaches typically assign uniform trajectory-level advantages and treat all correct tool calls equally.<n>This can lead to imprecise learning signals that do not adequately distinguish between trivial and challenging tool-use patterns.
  arXiv  Detail & Related papers  (2026-08-22T09:14:15Z)
• Learning When Not to Act: Mitigating Tool Abuse in Agentic Reinforcement Learning [26.34952204312613]
  Agentic reinforcement learning can induce tool abuse, where models overuse external tools even for solvable queries by internal reasoning.<n>We propose EAPO, an Efficient Agentic Policy Optimization framework that learns selective tool use.<n>Compared with GRPO, EAPO improves average performance by 10.45%, 7.27%, and 9.69%, while reducing average tool calls by 18.33%, 18.33%, and 24.59%, respectively.
  arXiv  Detail & Related papers  (2026-06-01T11:58:55Z)
• Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning [59.74608632210439]
  We show how to inject natural tool-use behavior into a strong thinking model without sacrificing its no-tool reasoning ability.<n>Our recipe yields models that achieve state-of-the-art performance in a wide range of benchmarks among open-source models.
  arXiv  Detail & Related papers  (2026-05-07T14:23:21Z)
• Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models [27.250148827297604]
  HDPO is a framework that reframes tool efficiency from a competing scalar objective to a strictly conditional one.<n>Our resulting model, Metis, reduces tool invocations by orders of magnitude while simultaneously elevating reasoning accuracy.
  arXiv  Detail & Related papers  (2026-04-09T17:59:57Z)
• Evolving from Tool User to Creator via Training-Free Experience Reuse in Multimodal Reasoning [16.12114923351562]
  We propose a training-free framework that transforms agents from tool users to tool creators.<n>This approach harvests reasoning experiences and distills them into reusable assets.<n>We also introduce a memory consolidation mechanism to maintain the tool library.
  arXiv  Detail & Related papers  (2026-02-02T11:37:45Z)
• Structured Uncertainty guided Clarification for LLM Agents [126.26213027785813]
  LLM agents extend large language models with tool-calling capabilities, but ambiguous user instructions often lead to incorrect invocations and task failures.<n>We introduce a principled formulation of structured uncertainty over tool-call parameters, modeling joint tool-argument clarification as a POMDP with Expected Value of Perfect Information (EVPI) objective for optimal question selection and aspect-based cost modeling to prevent redundancy.<n>Our SAGE-Agent leverages this structured uncertainty to achieve superior efficiency: increasing coverage on ambiguous tasks by 7-39% while reducing clarification questions by 1.5-2.7$times$ compared to strong prompting and uncertainty-based baselines.
  arXiv  Detail & Related papers  (2025-11-11T21:50:44Z)
• Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learning [68.89572566071575]
  Tool-Integrated Reasoning (TIR) enables large language models (LLMs) to improve their internal reasoning ability by integrating external tools.<n>We propose Tool-Light, a framework designed to encourage LLMs to perform TIR efficiently and accurately.<n> Experimental results on 10 datasets demonstrate the effectiveness of Tool-Light.
  arXiv  Detail & Related papers  (2025-09-27T12:53:37Z)
• AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learning [17.086082843274003]
  Large Language Models (LLMs) evolve into powerful Large Reasoning Models (LRMs)<n>Tool-Integrated Reasoning (TIR) further extends their capabilities by incorporating external tools.<n>Inspired by the human ability to adaptively select tools, we introduce AutoTIR, a reinforcement learning framework.
  arXiv  Detail & Related papers  (2025-07-29T14:12:28Z)
• Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection Learning [63.2198957755528]
  We propose Tool-MVR, a novel Tool-Augmented LLM that achieves comprehensive System 2 reasoning through two key innovations.<n>Specifically, we first introduce Multi-Agent Meta-Verification (MAMV), a systematic pipeline that rigorously validates APIs, queries, and reasoning trajectories.<n>Second, we propose Exploration-based Reflection Learning (EXPLORE), which enhances tool reflection capabilities by leveraging tool feedback.
  arXiv  Detail & Related papers  (2025-06-05T04:35:49Z)
• Acting Less is Reasoning More! Teaching Model to Act Efficiently [87.28134636548705]
  Tool-integrated reasoning augments large language models with the ability to invoke external tools to solve tasks.<n>Current approaches typically optimize only for final correctness without considering the efficiency or necessity of external tool use.<n>We propose a framework that encourages models to produce accurate answers with minimal tool calls.<n>Our approach reduces tool calls by up to 68.3% and improves tool productivity by up to 215.4%, while maintaining comparable answer accuracy.
  arXiv  Detail & Related papers  (2025-04-21T05:40:05Z)
• AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarification [25.27444694706659]
  We present AskToAct, which exploits the structural mapping between queries and their tool invocation solutions.<n>By systematically removing key parameters from queries while retaining them as ground truth, we enable automated construction of high-quality training data.<n>Our framework exhibits robust performance across different model architectures and successfully generalizes to entirely unseen APIs without additional training.
  arXiv  Detail & Related papers  (2025-03-03T12:55:49Z)
• Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger [49.81945268343162]
  We propose MeCo, an adaptive decision-making strategy for external tool use.<n>MeCo quantifies metacognitive scores by capturing high-level cognitive signals in the representation space.<n>MeCo is fine-tuning-free and incurs minimal cost.
  arXiv  Detail & Related papers  (2025-02-18T15:45:01Z)
• SMART: Self-Aware Agent for Tool Overuse Mitigation [58.748554080273585]
  Current Large Language Model (LLM) agents demonstrate strong reasoning and tool use capabilities, but often lack self-awareness.<n>This imbalance leads to Tool Overuse, where models unnecessarily rely on external tools for tasks with parametric knowledge.<n>We introduce SMART (Strategic Model-Aware Reasoning with Tools), a paradigm that enhances an agent's self-awareness to optimize task handling and reduce tool overuse.
  arXiv  Detail & Related papers  (2025-02-17T04:50:37Z)
• Reducing Tool Hallucination via Reliability Alignment [31.761771794788462]
  Large Language Models (LLMs) have expanded their capabilities beyond language generation to interact with external tools, enabling automation and real-world applications.<n>Tool hallucinations, where models either select inappropriate tools or misuse them, pose significant challenges, leading to erroneous task execution, increased computational costs, and reduced system reliability.<n>We introduce RelyToolBench, which integrates specialized test cases and novel metrics to assess hallucination-aware task success and efficiency.<n>Finally, we propose Relign, a reliability alignment framework that expands the tool-use action space to include indecisive actions, allowing LLMs to defer tool use, seek clarification, or adjust tool selection
  arXiv  Detail & Related papers  (2024-12-05T13:10:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.