ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via
Tool Embeddings
- URL: http://arxiv.org/abs/2305.11554v4
- Date: Mon, 15 Jan 2024 23:52:21 GMT
- Title: ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via
Tool Embeddings
- Authors: Shibo Hao, Tianyang Liu, Zhen Wang, Zhiting Hu
- Abstract summary: Augmenting large language models with external tools has emerged as a promising approach to solving complex problems.
Recent in-context learning paradigm alleviates these issues, but the limited context length only allows for a few shots of demonstrations.
We propose an alternative approach, $textbfToolkenGPT$, which combines the benefits of both sides.
- Score: 25.5476046472217
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Augmenting large language models (LLMs) with external tools has emerged as a
promising approach to solving complex problems. However, traditional methods,
which finetune LLMs with tool demonstration data, can be both costly and
restricted to a predefined set of tools. Recent in-context learning paradigm
alleviates these issues, but the limited context length only allows for a few
shots of demonstrations, leading to suboptimal understandings of the tools.
Moreover, when there are numerous tools to choose from, in-context learning
could completely fail to work. In this paper, we propose an alternative
approach, $\textbf{ToolkenGPT}$, which combines the benefits of both sides. Our
approach represents each $\underline{tool}$ as a to$\underline{ken}$
($\textit{toolken}$) and learns an embedding for it, enabling tool calls in the
same way as generating a regular word token. Once a toolken is triggered, the
LLM is prompted to complete arguments for the tool to execute. ToolkenGPT
offers the flexibility to plug in an arbitrary number of tools by expanding the
set of toolkens on the fly. In addition, it improves tool use by allowing
extensive demonstration data for learning the toolken embeddings. In diverse
domains, including numerical reasoning, knowledge-based question answering, and
embodied plan generation, our approach effectively augments LLMs with tools and
substantially outperforms various latest baselines. ToolkenGPT demonstrates the
promising ability to use relevant tools from a large tool set in complex
scenarios.
Related papers
- Enhancing Tool Retrieval with Iterative Feedback from Large Language Models [9.588592185027455]
Large language models (LLMs) can effectively handle a certain amount of tools through in-context learning or fine-tuning.
In real-world scenarios, the number of tools is typically extensive and irregularly updated, emphasizing the necessity for a dedicated tool retrieval component.
We propose to enhance tool retrieval with iterative feedback from the large language model.
arXiv Detail & Related papers (2024-06-25T11:12:01Z) - Chain of Tools: Large Language Model is an Automatic Multi-tool Learner [54.992464510992605]
Automatic Tool Chain (ATC) is a framework that enables the large language models (LLMs) to act as a multi-tool user.
To scale up the scope of the tools, we next propose a black-box probing method.
For a comprehensive evaluation, we build a challenging benchmark named ToolFlow.
arXiv Detail & Related papers (2024-05-26T11:40:58Z) - COLT: Towards Completeness-Oriented Tool Retrieval for Large Language Models [60.733557487886635]
We propose a novel modelagnostic COllaborative Learning-based Tool Retrieval approach, COLT.
COLT captures semantic similarities between user queries and tool descriptions.
It also takes into account the collaborative information of tools.
arXiv Detail & Related papers (2024-05-25T06:41:23Z) - What Are Tools Anyway? A Survey from the Language Model Perspective [67.18843218893416]
Language models (LMs) are powerful yet mostly for text generation tasks.
We provide a unified definition of tools as external programs used by LMs.
We empirically study the efficiency of various tooling methods.
arXiv Detail & Related papers (2024-03-18T17:20:07Z) - ToolNet: Connecting Large Language Models with Massive Tools via Tool
Graph [43.95759808077083]
Existing in-context learning approaches simply format tools into a list of plain text descriptions and input them to large language models.
This paper proposes ToolNet, a plug-and-play framework that scales up the number of tools to thousands with a moderate increase in token consumption.
arXiv Detail & Related papers (2024-02-29T02:04:00Z) - Look Before You Leap: Towards Decision-Aware and Generalizable
Tool-Usage for Large Language Models [28.19932548630398]
We propose a decision-aware and generalizable tool-usage framework (DEER)
Specifically, we first construct the tool-usage samples with multiple decision branches via an automatic generation pipeline.
Our proposed DEER is effective and significantly outperforms baselines across various datasets.
arXiv Detail & Related papers (2024-02-26T16:11:03Z) - EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction [56.02100384015907]
EasyTool is a framework transforming diverse and lengthy tool documentation into a unified and concise tool instruction.
It can significantly reduce token consumption and improve the performance of tool utilization in real-world scenarios.
arXiv Detail & Related papers (2024-01-11T15:45:11Z) - ControlLLM: Augment Language Models with Tools by Searching on Graphs [97.62758830255002]
We present ControlLLM, a novel framework that enables large language models (LLMs) to utilize multi-modal tools for solving real-world tasks.
Our framework comprises three key components: (1) a textittask decomposer that breaks down a complex task into clear subtasks with well-defined inputs and outputs; (2) a textitThoughts-on-Graph (ToG) paradigm that searches the optimal solution path on a pre-built tool graph; and (3) an textitexecution engine with a rich toolbox that interprets the solution path and runs the
arXiv Detail & Related papers (2023-10-26T21:57:21Z) - MetaTool Benchmark for Large Language Models: Deciding Whether to Use
Tools and Which to Use [82.24774504584066]
Large language models (LLMs) have garnered significant attention due to their impressive natural language processing (NLP) capabilities.
We introduce MetaTool, a benchmark designed to evaluate whether LLMs have tool usage awareness and can correctly choose tools.
We conduct experiments involving eight popular LLMs and find that the majority of them still struggle to effectively select tools.
arXiv Detail & Related papers (2023-10-04T19:39:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.