In-Context Reinforcement Learning for Tool Use in Large Language Models

• URL: http://arxiv.org/abs/2603.08068v1
• Date: Mon, 09 Mar 2026 08:06:18 GMT
• Title: In-Context Reinforcement Learning for Tool Use in Large Language Models
• Abstract summary: Large language models (LLMs) exhibit strong reasoning abilities, but their performance on complex tasks is often constrained by their internal knowledge.<n>We propose In-Context Reinforcement Learning (ICRL), an RL-only framework that eliminates the need for supervised fine-tuning.<n>ICRL achieves state-of-the-art performance, demonstrating its effectiveness as a scalable, data-efficient alternative to traditional SFT-based pipelines.
• Score: 68.66653829365187
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While large language models (LLMs) exhibit strong reasoning abilities, their performance on complex tasks is often constrained by the limitations of their internal knowledge. A compelling approach to overcome this challenge is to augment these models with external tools -- such as Python interpreters for mathematical computations or search engines for retrieving factual information. However, enabling models to use these tools effectively remains a significant challenge. Existing methods typically rely on cold-start pipelines that begin with supervised fine-tuning (SFT), followed by reinforcement learning (RL). These approaches often require substantial amounts of labeled data for SFT, which is expensive to annotate or synthesize. In this work, we propose In-Context Reinforcement Learning (ICRL), an RL-only framework that eliminates the need for SFT by leveraging few-shot prompting during the rollout stage of RL. Specifically, ICRL introduces in-context examples within the rollout prompts to teach the model how to invoke external tools. Furthermore, as training progresses, the number of in-context examples is gradually reduced, eventually reaching a zero-shot setting where the model learns to call tools independently. We conduct extensive experiments across a range of reasoning and tool-use benchmarks. Results show that ICRL achieves state-of-the-art performance, demonstrating its effectiveness as a scalable, data-efficient alternative to traditional SFT-based pipelines.

Related papers

• Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch [63.40752011615843]
  Training tool-augmented language models has emerged as a promising approach to enhancing their capabilities for complex tasks.<n>We propose a dynamic generalization-guided reward design for rule-based reinforcement learning.<n>We show that our models achieve over 7% performance improvement compared to both SFT and RL-with-SFT models.
  arXiv  Detail & Related papers  (2025-11-02T16:33:45Z)
• Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learning [68.89572566071575]
  Tool-Integrated Reasoning (TIR) enables large language models (LLMs) to improve their internal reasoning ability by integrating external tools.<n>We propose Tool-Light, a framework designed to encourage LLMs to perform TIR efficiently and accurately.<n> Experimental results on 10 datasets demonstrate the effectiveness of Tool-Light.
  arXiv  Detail & Related papers  (2025-09-27T12:53:37Z)
• Understanding Tool-Integrated Reasoning [9.235747697967984]
  We study why Tool-Integrated Reasoning makes Large Language Models (LLMs) more capable.<n>LLMs integrated with tools like Python code interpreters show great promise, but a principled theory explaining why this paradigm is effective has been missing.<n>We demonstrate that tools enable a strict expansion of the model's empirical and feasible support, breaking the capability ceiling of pure-text models.
  arXiv  Detail & Related papers  (2025-08-26T17:03:46Z)
• SPaRFT: Self-Paced Reinforcement Fine-Tuning for Large Language Models [51.74498855100541]
  Large language models (LLMs) have shown strong reasoning capabilities when fine-tuned with reinforcement learning (RL)<n>We propose textbfSPaRFT, a self-paced learning framework that enables efficient learning based on the capability of the model being trained.
  arXiv  Detail & Related papers  (2025-08-07T03:50:48Z)
• Replacing thinking with tool usage enables reasoning in small language models [2.357055571094446]
  Recent advances have established a new machine learning paradigm based on scaling up compute at inference time as well as at training time.<n>In this paper, we propose to format these tokens as a multi-turn interaction trace with a stateful tool.<n>At each turn, the new state of the tool is appended to the context of the model, whose job is to generate the tokens necessary to control the tool via a custom DSL.
  arXiv  Detail & Related papers  (2025-07-07T14:49:18Z)
• Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning [93.30252692375886]
  Rule-based reinforcement learning can be used to enhance tool-calling in large language models.<n>Tool-N1-7B/14B clearly outperform GPT-4o on several major benchmarks.
  arXiv  Detail & Related papers  (2025-04-25T02:55:21Z)
• Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger [49.81945268343162]
  We propose MeCo, an adaptive decision-making strategy for external tool use.<n>MeCo quantifies metacognitive scores by capturing high-level cognitive signals in the representation space.<n>MeCo is fine-tuning-free and incurs minimal cost.
  arXiv  Detail & Related papers  (2025-02-18T15:45:01Z)
• iTool: Reinforced Fine-Tuning with Dynamic Deficiency Calibration for Advanced Tool Use [56.31110409360567]
  Augmenting large language models with external tools is a promising approach to enhance their capabilities.<n>We show that training gains significantly decay as synthetic data increases.<n>We propose an iterative reinforced fine-tuning strategy designed to alleviate this limitation.
  arXiv  Detail & Related papers  (2025-01-15T04:52:34Z)
• AXOLOTL: Fairness through Assisted Self-Debiasing of Large Language Model Outputs [20.772266479533776]
  AXOLOTL is a novel post-processing framework that operates agnostically across tasks and models. It identifies biases, proposes resolutions, and guides the model to self-debias its outputs. This approach minimizes computational costs and preserves model performance.
  arXiv  Detail & Related papers  (2024-03-01T00:02:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.