Chain-of-Tools: Utilizing Massive Unseen Tools in the CoT Reasoning of Frozen Language Models

• URL: http://arxiv.org/abs/2503.16779v1
• Date: Fri, 21 Mar 2025 01:26:12 GMT
• Title: Chain-of-Tools: Utilizing Massive Unseen Tools in the CoT Reasoning of Frozen Language Models
• Authors: Mengsong Wu, Tong Zhu, Han Han, Xiang Zhang, Wenbiao Shao, Wenliang Chen,
• Abstract summary: Tool learning can further broaden the usage scenarios of large language models (LLMs)<n>We present a new Tool Learning method Chain-of-Tools.<n>It makes full use of the powerful semantic representation capability of frozen LLMs to finish tool calling in CoT reasoning.
• Score: 8.573278807410507
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tool learning can further broaden the usage scenarios of large language models (LLMs). However most of the existing methods either need to finetune that the model can only use tools seen in the training data, or add tool demonstrations into the prompt with lower efficiency. In this paper, we present a new Tool Learning method Chain-of-Tools. It makes full use of the powerful semantic representation capability of frozen LLMs to finish tool calling in CoT reasoning with a huge and flexible tool pool which may contain unseen tools. Especially, to validate the effectiveness of our approach in the massive unseen tool scenario, we construct a new dataset SimpleToolQuestions. We conduct experiments on two numerical reasoning benchmarks (GSM8K-XL and FuncQA) and two knowledge-based question answering benchmarks (KAMEL and SimpleToolQuestions). Experimental results show that our approach performs better than the baseline. We also identify dimensions of the model output that are critical in tool selection, enhancing the model interpretability. Our code and data are available at: https://github.com/fairyshine/Chain-of-Tools .

Related papers

• Self-Training Large Language Models for Tool-Use Without Demonstrations [15.17750971071501]
  Large language models (LLMs) remain prone to factual inaccuracies and computational errors.<n>Recent work augmented LLMs with tools to mitigate these shortcomings, but often requires curated gold tool-use demonstrations.<n>This paper investigates whether LLMs can learn to use tools without demonstrations.
  arXiv  Detail & Related papers  (2025-02-09T12:06:10Z)
• Efficient and Scalable Estimation of Tool Representations in Vector Space [34.767193045989515]
  We present a framework for generating synthetic data for tool retrieval applications and an efficient data-driven tool retrieval strategy using small encoder models. We create ToolBank, a new tool retrieval dataset that reflects real human user usages. With these new methods, we achieve improvements of up to 27.28 in Recall@K on the ToolBench dataset and 30.5 in Recall@K on ToolBank.
  arXiv  Detail & Related papers  (2024-09-02T19:39:24Z)
• Enhancing Tool Retrieval with Iterative Feedback from Large Language Models [9.588592185027455]
  Large language models (LLMs) can effectively handle a certain amount of tools through in-context learning or fine-tuning. In real-world scenarios, the number of tools is typically extensive and irregularly updated, emphasizing the necessity for a dedicated tool retrieval component. We propose to enhance tool retrieval with iterative feedback from the large language model.
  arXiv  Detail & Related papers  (2024-06-25T11:12:01Z)
• Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agents [56.822238860147024]
  Augmenting large language models with external tools has emerged as a promising approach to extend their utility.<n>Previous methods manually parse tool documentation and create in-context demonstrations, transforming tools into structured formats for LLMs to use in their step-by-step reasoning.<n>We propose AutoTools, a framework that enables LLMs to automate the tool-use workflow.
  arXiv  Detail & Related papers  (2024-05-26T11:40:58Z)
• Towards Completeness-Oriented Tool Retrieval for Large Language Models [60.733557487886635]
  Real-world systems often incorporate a wide array of tools, making it impractical to input all tools into Large Language Models. Existing tool retrieval methods primarily focus on semantic matching between user queries and tool descriptions. We propose a novel modelagnostic COllaborative Learning-based Tool Retrieval approach, COLT, which captures not only the semantic similarities between user queries and tool descriptions but also takes into account the collaborative information of tools.
  arXiv  Detail & Related papers  (2024-05-25T06:41:23Z)
• Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmark [8.573278807410507]
  This paper presents a new tool learning dataset Seal-Tools. Seal-Tools contains self-instruct API-like tools. It also includes instances which demonstrate the practical application of tools.
  arXiv  Detail & Related papers  (2024-05-14T06:50:19Z)
• What Are Tools Anyway? A Survey from the Language Model Perspective [67.18843218893416]
  Language models (LMs) are powerful yet mostly for text generation tasks. We provide a unified definition of tools as external programs used by LMs. We empirically study the efficiency of various tooling methods.
  arXiv  Detail & Related papers  (2024-03-18T17:20:07Z)
• EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction [56.02100384015907]
  EasyTool is a framework transforming diverse and lengthy tool documentation into a unified and concise tool instruction. It can significantly reduce token consumption and improve the performance of tool utilization in real-world scenarios.
  arXiv  Detail & Related papers  (2024-01-11T15:45:11Z)
• MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use [79.87054552116443]
  Large language models (LLMs) have garnered significant attention due to their impressive natural language processing (NLP) capabilities. We introduce MetaTool, a benchmark designed to evaluate whether LLMs have tool usage awareness and can correctly choose tools. We conduct experiments involving eight popular LLMs and find that the majority of them still struggle to effectively select tools.
  arXiv  Detail & Related papers  (2023-10-04T19:39:26Z)
• Large Language Models as Tool Makers [85.00361145117293]
  We introduce a closed-loop framework, referred to as LLMs A s Tool Makers (LATM), where LLMs create their own reusable tools for problem-solving. Our approach consists of two phases: 1) tool making: an LLM acts as the tool maker that crafts tools for a set of tasks. 2) tool using: another LLM acts as the tool user, which applies the tool built by the tool maker for problem-solving.
  arXiv  Detail & Related papers  (2023-05-26T17:50:11Z)
• Toolformer: Language Models Can Teach Themselves to Use Tools [62.04867424598204]
  Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale. We show that LMs can teach themselves to use external tools via simple APIs and achieve the best of both worlds. We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction.
  arXiv  Detail & Related papers  (2023-02-09T16:49:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.