ToolNet: Connecting Large Language Models with Massive Tools via Tool Graph

• URL: http://arxiv.org/abs/2403.00839v1
• Date: Thu, 29 Feb 2024 02:04:00 GMT
• Title: ToolNet: Connecting Large Language Models with Massive Tools via Tool Graph
• Authors: Xukun Liu, Zhiyuan Peng, Xiaoyuan Yi, Xing Xie, Lirong Xiang, Yuchen Liu, Dongkuan Xu
• Abstract summary: Existing in-context learning approaches simply format tools into a list of plain text descriptions and input them to large language models. This paper proposes ToolNet, a plug-and-play framework that scales up the number of tools to thousands with a moderate increase in token consumption.
• Score: 43.95759808077083
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While achieving remarkable progress in a broad range of tasks, large language models (LLMs) remain significantly limited in properly using massive external tools. Existing in-context learning approaches simply format tools into a list of plain text descriptions and input them to LLMs, from which, LLMs generate a sequence of tool calls to solve problems step by step. Such a paradigm ignores the intrinsic dependency between tools and offloads all reasoning loads to LLMs, making them restricted to a limited number of specifically designed tools. It thus remains challenging for LLMs to operate on a library of massive tools, casting a great limitation when confronted with real-world scenarios. This paper proposes ToolNet, a plug-and-play framework that scales up the number of tools to thousands with a moderate increase in token consumption. ToolNet organizes tools into a directed graph. Each node represents a tool, and weighted edges denote tool transition. Starting from an initial tool node, an LLM navigates in the graph by iteratively choosing the next one from its successors until the task is resolved. Extensive experiments show that ToolNet can achieve impressive results in challenging multi-hop tool learning datasets and is resilient to tool failures.

Related papers

• NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of Large Language Models [10.344854970262984]
  We introduce NesTools to bridge the gap in comprehensive nested tool learning evaluations. NesTools comprises a novel automatic data generation method to construct large-scale nested tool calls. With manual review and refinement, the dataset is in high quality and closely aligned with real-world scenarios.
  arXiv  Detail & Related papers  (2024-10-15T17:33:43Z)
• ToolGen: Unified Tool Retrieval and Calling via Generation [34.34787641393914]
  We introduce ToolGen, a paradigm shift that integrates tool knowledge directly into the large language models' parameters. We show that ToolGen achieves superior results in both tool retrieval and autonomous task completion. ToolGen paves the way for more versatile, efficient, and autonomous AI systems.
  arXiv  Detail & Related papers  (2024-10-04T13:52:32Z)
• Chain of Tools: Large Language Model is an Automatic Multi-tool Learner [54.992464510992605]
  Automatic Tool Chain (ATC) is a framework that enables the large language models (LLMs) to act as a multi-tool user. To scale up the scope of the tools, we next propose a black-box probing method. For a comprehensive evaluation, we build a challenging benchmark named ToolFlow.
  arXiv  Detail & Related papers  (2024-05-26T11:40:58Z)
• Towards Completeness-Oriented Tool Retrieval for Large Language Models [60.733557487886635]
  Real-world systems often incorporate a wide array of tools, making it impractical to input all tools into Large Language Models. Existing tool retrieval methods primarily focus on semantic matching between user queries and tool descriptions. We propose a novel modelagnostic COllaborative Learning-based Tool Retrieval approach, COLT, which captures not only the semantic similarities between user queries and tool descriptions but also takes into account the collaborative information of tools.
  arXiv  Detail & Related papers  (2024-05-25T06:41:23Z)
• EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction [56.02100384015907]
  EasyTool is a framework transforming diverse and lengthy tool documentation into a unified and concise tool instruction. It can significantly reduce token consumption and improve the performance of tool utilization in real-world scenarios.
  arXiv  Detail & Related papers  (2024-01-11T15:45:11Z)
• MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use [82.24774504584066]
  Large language models (LLMs) have garnered significant attention due to their impressive natural language processing (NLP) capabilities. We introduce MetaTool, a benchmark designed to evaluate whether LLMs have tool usage awareness and can correctly choose tools. We conduct experiments involving eight popular LLMs and find that the majority of them still struggle to effectively select tools.
  arXiv  Detail & Related papers  (2023-10-04T19:39:26Z)
• CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models [74.22729793816451]
  Large Language Models (LLMs) have made significant progress in utilizing tools, but their ability is limited by API availability. We propose CREATOR, a novel framework that enables LLMs to create their own tools using documentation and code realization. We evaluate CREATOR on MATH and TabMWP benchmarks, respectively consisting of challenging math competition problems.
  arXiv  Detail & Related papers  (2023-05-23T17:51:52Z)
• ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings [25.5476046472217]
  Augmenting large language models with external tools has emerged as a promising approach to solving complex problems. Recent in-context learning paradigm alleviates these issues, but the limited context length only allows for a few shots of demonstrations. We propose an alternative approach, $textbfToolkenGPT$, which combines the benefits of both sides.
  arXiv  Detail & Related papers  (2023-05-19T09:54:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.