Outcome-Aware Tool Selection for Semantic Routers: Latency-Constrained Learning Without LLM Inference

• URL: http://arxiv.org/abs/2603.13426v1
• Date: Fri, 13 Mar 2026 02:13:05 GMT
• Title: Outcome-Aware Tool Selection for Semantic Routers: Latency-Constrained Learning Without LLM Inference
• Abstract summary: Outcome-Aware Tool Selection (OATS) interpolates tool embeddings toward the centroid of queries where they historically succeed.<n>On MetaTool (199tools, 4,287queries), this improves NDCG@5 from 0.869 to 0.940.<n>All methods are evaluated on the same held-out 30% test split.
• Score: 12.135271159221178
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semantic routers in LLM inference gateways select tools in the critical request path, where every millisecond of added latency compounds across millions of requests. We propose Outcome-Aware Tool Selection (OATS), which interpolates tool embeddings toward the centroid of queries where they historically succeed -- an offline process that adds no parameters, latency, or GPU cost at serving time. On MetaTool (199~tools, 4,287~queries), this improves NDCG@5 from 0.869 to 0.940; on ToolBench (2,413~APIs), from 0.834 to 0.848. We also evaluate two learned extensions: a 2,625-parameter MLP re-ranker and a 197K-parameter contrastive adapter. The MLP re-ranker hurts or matches baseline when outcome data is sparse relative to the tool set; the contrastive adapter provides comparable gains on MetaTool (NDCG@5: 0.931). All methods are evaluated on the same held-out 30\% test split. The practical takeaway is to start with the zero-cost refinement and add learned components only when data density warrants it. All mechanisms run within single-digit millisecond CPU budgets.

Related papers

• Hybrid Semantic Tool Discovery for Enterprise MCP Gateway: Architecture and Implementation [1.6774701449274187]
  Large language model (LLM) agents invoke external tools to retrieve and reason over information beyond pretrained knowledge.<n>The Model Context Protocol (MCP) standardizes how such tools are surfaced, and a proxy MCP server aggregates many backend servers behind a single endpoint.<n>We present SCOUT, which reframes tool exposure as a context-selection problem, injecting only tools relevant to the current step.
  arXiv  Detail & Related papers  (2026-08-25T02:33:44Z)
• Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls [43.88014476513854]
  In domains such as cloud networking, even frontier models correctly complete fewer than half of tool calls.<n>We propose a unified probe-guided framework with two complementary approaches.<n>We show that our method substantially improves parameter generation, raising the average exact match from 19.7% to 59.6%.
  arXiv  Detail & Related papers  (2026-08-04T03:36:41Z)
• ToolAtlas: Learning Once, Reusing Everywhere with Tool-Side Memory [49.08868447320197]
  Large language model (LLM) agents increasingly rely on external tools served by shared providers.<n>Existing approaches improve tool use on the agent side through parameter updates, prompt refinement, or agent-side memory.<n>We introduce ToolAtlas, a graph-based framework that builds a persistent provider-side tool memory of tool capabilities.
  arXiv  Detail & Related papers  (2026-07-13T06:01:41Z)
• TRACE: Business Rule-Grounded Reasoning Curriculum for Knowledge-Preserving Parametric Tool Retrieval in Enterprise LLMs [1.4680035572775534]
  Parametric retrieval enables LLMs to retrieve tools implicitly by assigning each API a unique virtual token and training the model to generate it via constrained beam search.<n>Toolsense shows that this regime has two critical drawbacks: it destroys parametric tool knowledge during training, and its beamsearch decoding is too slow for real-time deployment.<n>We introduce TRACE (Tool Retrieval via Augmented Chain-of-thought and Enterprise rules), a two-stage curriculum that resolves this dissociation.
  arXiv  Detail & Related papers  (2026-06-22T23:15:35Z)
• Semantic Tool Discovery for Large Language Models: A Vector-Based Approach to MCP Tool Selection [1.2633421050364024]
  Large Language Models (LLMs) with tool-calling capabilities have demonstrated remarkable potential in executing complex tasks through external tool integration.<n>The Model Context Protocol (MCP) has emerged as a standardized framework for connecting LLMs to diverse toolsets, with individual MCP servers potentially exposing dozens to hundreds of tools.<n>We present a semantic tool discovery architecture that addresses these challenges through vector-based retrieval.
  arXiv  Detail & Related papers  (2026-03-19T18:10:35Z)
• Try, Check and Retry: A Divide-and-Conquer Framework for Boosting Long-context Tool-Calling Performance of LLMs [78.58042751671393]
  Tool-DC is a Divide-and-Conquer framework for boosting tool-calling performance of Large Language Models.<n>Tool-DC involves two variants: 1) the training-free Tool-DC (TF), which is plug-and-play and flexible; 2) the training-based Tool-DC (TB), which is more inference-efficient.<n>Extensive experiments show that both Tool-DC methods outperform their counterparts by a clear margin.
  arXiv  Detail & Related papers  (2026-03-12T03:30:01Z)
• PROTEUS: SLA-Aware Routing via Lagrangian RL for Multi-LLM Serving Systems [1.0978496459260902]
  We present PROTEUS, a router that accepts accuracy targets tau as runtime input.<n>A single trained model serves the full accuracy spectrum without retraining.<n>We evaluate on RouterBench (11 models, 405K queries) and SPROUT (14 models, 45K queries)
  arXiv  Detail & Related papers  (2026-01-27T09:38:16Z)
• AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning [79.65732142949014]
  Agentic reinforcement learning has advanced large language models (LLMs) to reason through long chain-of-thought trajectories.<n>Existing approaches assume a fixed inventory of tools, limiting LLM agents' adaptability to new or evolving toolsets.<n>We present AutoTool, a framework that equips LLM agents with dynamic tool-selection capabilities throughout their reasoning trajectories.
  arXiv  Detail & Related papers  (2025-12-15T12:38:04Z)
• TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use [74.47746287181383]
  Large language model (LLM)-based agents increasingly rely on tool use to complete real-world tasks.<n>We introduce TRAJECT-Bench, a trajectory-aware benchmark to comprehensively evaluate LLMs' tool use capability.
  arXiv  Detail & Related papers  (2025-10-06T07:30:25Z)
• RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Use [50.52940111891476]
  Large language models excel at basic reasoning but struggle with tasks that require interaction with external tools.<n>We present RLFactory, a plug-and-play reinforcement learning framework for multi-round tool use.
  arXiv  Detail & Related papers  (2025-08-31T16:47:31Z)
• Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection Learning [63.2198957755528]
  We propose Tool-MVR, a novel Tool-Augmented LLM that achieves comprehensive System 2 reasoning through two key innovations.<n>Specifically, we first introduce Multi-Agent Meta-Verification (MAMV), a systematic pipeline that rigorously validates APIs, queries, and reasoning trajectories.<n>Second, we propose Exploration-based Reflection Learning (EXPLORE), which enhances tool reflection capabilities by leveraging tool feedback.
  arXiv  Detail & Related papers  (2025-06-05T04:35:49Z)
• MCP-Zero: Active Tool Discovery for Autonomous LLM Agents [13.005899769943442]
  We introduce MCP-Zero, an active agent framework that restores tool discovery autonomy to LLMs themselves.<n>Instead of overwhelming models with all available tools, MCP-Zero enables agents to actively identify capability gaps, and request specific tools on-demand.<n>We construct MCP-tools, a comprehensive dataset of 308 MCP servers and 2,797 tools from the official Model-Context-Protocol repository.
  arXiv  Detail & Related papers  (2025-06-01T15:48:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.