PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models

• URL: http://arxiv.org/abs/2607.05441v1
• Date: Fri, 03 Jul 2026 18:47:06 GMT
• Title: PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models
• Abstract summary: PORTS is a novel odds ratio preference optimization method for training retrievers aimed at tool selection.<n>Our approach fine-tunes a retriever to find helpful tools by optimizing the correlation between the selection probabilities and the downstream performances.<n>With low computational demands, our alignment process facilitates generalization to new queries and tools.
• Score: 9.904299892454594
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Integrating external tools with Large Language Models (LLMs) has emerged as a promising paradigm for accomplishing complex tasks. Since LLMs still struggle to effectively manage large tool collections, researchers have begun exploring retrieval-based methods to pre-select the most relevant options, addressing input length and latency constraints. However, existing retrievers are often misaligned with tool-calling LLMs due to their separate training processes. This paper presents PORTS, a novel odds ratio preference optimization method for training retrievers aimed at tool selection. Using a perplexity-inspired preference signal from a frozen LLM, our approach fine-tunes a retriever to find helpful tools by optimizing the correlation between the selection probabilities and the downstream performances while jointly enforcing a contrastive semantic loss between documentation strings. The versatility of PORTS and its ability to significantly improve tool selection accuracy are demonstrated through extensive experiments on six datasets, two encoder models, and three LLMs with diverse prior knowledge. With low computational demands, our alignment process facilitates generalization to new queries and tools, proving valuable for practical applications with evolving toolsets.

Related papers

• IAPO: Input Attribution-Aware Policy Optimization for Tool Use in Small Multimodal Agents [12.019312046941396]
  This paper investigates reinforcement learning (RL) methods for improving tool-calling capabilities in small language models (SLM)<n>We propose IAPO, an RL algorithm for improving tool use in multimodal SLM by aligning the model's attribution across input components with that of a stronger teacher.<n>Experiments on Qwen2.5-VL-3B show that the proposed method improves visual question answering accuracy by an average of 3% across six test sets compared with existing visual tool use work, by helping the model attend to the most relevant input evidence.
  arXiv  Detail & Related papers  (2026-06-10T04:30:37Z)
• RaTA-Tool: Retrieval-based Tool Selection with Multimodal Large Language Models [57.15854852525046]
  We introduce RaTA-Tool, a novel framework for open-world multimodal tool selection.<n>Our approach enables an MLLM to convert a multimodal query into a structured task description and subsequently retrieve the most appropriate tool.<n>To further improve alignment between task descriptions and tool selection, we incorporate a preference-based optimization stage.
  arXiv  Detail & Related papers  (2026-04-16T12:47:09Z)
• ToolDreamer: Instilling LLM Reasoning Into Tool Retrievers [33.08308979741825]
  Existing retrieval models rank tools based on the similarity between a user query and a tool description (TD)<n>This leads to suboptimal retrieval as user requests are often poorly aligned with the language of TD.<n>We propose ToolDreamer, a framework to condition retriever models to fetch tools based on hypothetical (synthetic) TD.
  arXiv  Detail & Related papers  (2025-10-22T17:26:05Z)
• Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning [63.31585771716123]
  Large language models (LLMs) have shown remarkable reasoning capabilities via large-scale reinforcement learning (RL)<n>We introduce Tool-Star, an RL-based framework designed to empower LLMs to autonomously invoke multiple external tools during stepwise reasoning.<n>Tool-Star integrates six types of tools and incorporates systematic designs in both data synthesis and training.
  arXiv  Detail & Related papers  (2025-05-22T09:00:19Z)
• Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger [49.81945268343162]
  We propose MeCo, an adaptive decision-making strategy for external tool use.<n>MeCo quantifies metacognitive scores by capturing high-level cognitive signals in the representation space.<n>MeCo is fine-tuning-free and incurs minimal cost.
  arXiv  Detail & Related papers  (2025-02-18T15:45:01Z)
• TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use [72.32614703504122]
  Large language models (LLMs) achieve remarkable advancements by leveraging tools to interact with environments.<n>Standard supervised fine-tuning approach, which relies on large-scale datasets, often overlooks task-specific characteristics in tool use.<n>We propose TL-Training, a task-feature-based framework that mitigates the effects of suboptimal training data.
  arXiv  Detail & Related papers  (2024-12-20T02:21:36Z)
• Efficient and Scalable Estimation of Tool Representations in Vector Space [34.767193045989515]
  We present a framework for generating synthetic data for tool retrieval applications and an efficient data-driven tool retrieval strategy using small encoder models. We create ToolBank, a new tool retrieval dataset that reflects real human user usages. With these new methods, we achieve improvements of up to 27.28 in Recall@K on the ToolBench dataset and 30.5 in Recall@K on ToolBank.
  arXiv  Detail & Related papers  (2024-09-02T19:39:24Z)
• Towards Completeness-Oriented Tool Retrieval for Large Language Models [60.733557487886635]
  Real-world systems often incorporate a wide array of tools, making it impractical to input all tools into Large Language Models. Existing tool retrieval methods primarily focus on semantic matching between user queries and tool descriptions. We propose a novel modelagnostic COllaborative Learning-based Tool Retrieval approach, COLT, which captures not only the semantic similarities between user queries and tool descriptions but also takes into account the collaborative information of tools.
  arXiv  Detail & Related papers  (2024-05-25T06:41:23Z)
• Look Before You Leap: Towards Decision-Aware and Generalizable Tool-Usage for Large Language Models [26.28459880766842]
  We propose a decision-aware and generalizable tool-usage framework (DEER) Specifically, we first construct the tool-usage samples with multiple decision branches via an automatic generation pipeline. Our proposed DEER is effective and significantly outperforms baselines across various datasets.
  arXiv  Detail & Related papers  (2024-02-26T16:11:03Z)
• MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use [79.87054552116443]
  Large language models (LLMs) have garnered significant attention due to their impressive natural language processing (NLP) capabilities.<n>We introduce MetaTool, a benchmark designed to evaluate whether LLMs have tool usage awareness and can correctly choose tools.<n>We conduct experiments involving eight popular LLMs and find that the majority of them still struggle to effectively select tools.
  arXiv  Detail & Related papers  (2023-10-04T19:39:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.