Switchcraft: AI Model Router for Agentic Tool Calling

• URL: http://arxiv.org/abs/2605.07112v1
• Date: Fri, 08 May 2026 01:41:31 GMT
• Title: Switchcraft: AI Model Router for Agentic Tool Calling
• Abstract summary: We present Switchcraft, the first (to the best of our knowledge) model router optimized for agentic tool calling.<n>Switchcraft achieves 82.9% accuracy -- matching or exceeding the best individual model -- while reducing inference cost by 84%.<n>We find that larger models do not consistently outperform smaller ones on tool-use tasks, and that nominally cheaper models can incur higher total cost due to token-intensive reasoning.
• Score: 2.8769287530821948
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic AI systems that invoke external tools are powerful but costly, leading developers to default to large models and overspend inference budgets. Model routing can mitigate this, but existing routers are designed for chat completion rather than tool use. We present Switchcraft, the first (to the best of our knowledge) model router optimized for agentic tool calling. Switchcraft operates inline, selecting the lowest-cost model subject to correctness. We construct an evaluation framework on five function-calling benchmarks and train a DistilBERT-based classifier, deployed under a latency budget. Switchcraft achieves 82.9% accuracy -- matching or exceeding the best individual model -- while reducing inference cost by 84%, saving over $3,600 per million queries. We find that larger models do not consistently outperform smaller ones on tool-use tasks, and that nominally cheaper models can incur higher total cost due to token-intensive reasoning. Our work enables cost-aware agentic AI deployment without sacrificing correctness.

Related papers

• Agentic Routing: The Harness-Native Data Flywheel [35.444792826526]
  Report studies singleton and multi-model routing on agentic benchmarks including DRACO and PinchBench.<n>It argues that agentic routing is not merely cost control, but a data engine for agent-native training.
  arXiv  Detail & Related papers  (2026-07-13T11:05:55Z)
• SWE-Router: Routing in Multi-turn Agentic Software Engineering Tasks [11.50404356970464]
  Large language models (LLMs) embedded in multi-turn agentic harnesses are reshaping software engineering (SWE)<n>But routing every task to a frontier model is wasteful when many issues admit cheap fixes.<n>We introduce SWE--a value-based temporal approach that lets a cheap model run for a few exploratory turns and reads the resulting partial trajectory.<n>We show that SWE- greatly improves the cost efficiency of SWE tasks, while maintaining the majority of the performances of the stronger model.
  arXiv  Detail & Related papers  (2026-06-30T01:46:26Z)
• Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference [0.0]
  "Pyramid MoA" is a hierarchical Mixture-of-Agents architecture that uses a lightweight Router to dynamically escalate queries only when necessary.<n>We demonstrate that the system introduces negligible latency overhead (+0.82s) and allows for a tunable trade-off between performance and budget.
  arXiv  Detail & Related papers  (2026-02-23T04:47:47Z)
• When Routing Collapses: On the Degenerate Convergence of LLM Routers [46.01380774114097]
  As user's cost budget increases, routers systematically default to the most capable and most expensive model.<n>We propose Equi, a decision-aware router that directly learns model rankings.<n>On RouterBench, Equi reduces cost by about 17% at GPT-4-level performance compared to the strongest prior router.
  arXiv  Detail & Related papers  (2026-02-03T12:51:55Z)
• Models Under SCOPE: Scalable and Controllable Routing via Pre-hoc Reasoning [28.165465162107253]
  We propose SCOPE, a routing framework that goes beyond model selection by predicting their cost and performance.<n>SCOPE makes reasoning-based predictions by retrieving how models behave on similar problems, rather than relying on fixed model names.<n>It can boost accuracy by up to 25.7% when performance is the priority, or cut costs by up to 95.1% when efficiency matters most.
  arXiv  Detail & Related papers  (2026-01-29T21:09:36Z)
• RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents [91.0187958746262]
  RouteMoA is an efficient mixture-of-agents framework with dynamic routing.<n>It employs a lightweight scorer to perform initial screening by predicting coarse-grained performance from the query.<n>It refines these scores through lightweight self- and cross-assessment based on existing model outputs, providing posterior correction without additional inference.
  arXiv  Detail & Related papers  (2026-01-26T04:22:22Z)
• One Model to Critique Them All: Rewarding Agentic Tool-Use via Efficient Reasoning [54.580646706013965]
  Reward models (RMs) play a critical role in aligning large language models with human preferences.<n>We introduce ToolRM, a family of lightweight generative RMs tailored for general tool-use scenarios.<n>To build these models, we propose a novel pipeline that constructs pairwise preference data using rule-based scoring and multidimensional sampling.
  arXiv  Detail & Related papers  (2025-10-30T06:08:27Z)
• xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning [104.63494870852894]
  We present x, a tool-calling-based routing system in which a learned router can either answer directly or invoke one or more external models.<n>Our implementation encompasses the full reinforcement learning framework, including reward and cost accounting.<n>Across diverse benchmarks, x achieves strong cost-performance trade-offs.
  arXiv  Detail & Related papers  (2025-10-09T16:52:01Z)
• RADAR: Reasoning-Ability and Difficulty-Aware Routing for Reasoning LLMs [51.88834210085435]
  We present RADAR (Reasoning-Ability and Difficulty-Aware Routing), a lightweight, interpretable, and scalable routing framework.<n>Inspired by psychometrics, RADAR learns an item response model from model responses with different budgets to different queries.<n>We conduct extensive experiments on 8 widely used reasoning benchmarks, demonstrating the superior performance of RADAR compared to state-of-the-art routing methods.
  arXiv  Detail & Related papers  (2025-09-29T19:33:44Z)
• RouteLLM: Learning to Route LLMs with Preference Data [41.687640419561504]
  Large language models (LLMs) exhibit impressive capabilities across a wide range of tasks, yet the choice of which model to use often involves a trade-off between performance and cost.<n>We propose several efficient router models that dynamically select between a stronger and a weaker LLM during inference.<n>We develop a training framework for these routers leveraging human preference data and data augmentation techniques to enhance performance.
  arXiv  Detail & Related papers  (2024-06-26T18:10:22Z)
• Exploring Sparse Expert Models and Beyond [51.90860155810848]
  Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost. We propose a simple method called expert prototyping that splits experts into different prototypes and applies $k$ top-$1$ routing. This strategy improves the model quality but maintains constant computational costs, and our further exploration on extremely large-scale models reflects that it is more effective in training larger models.
  arXiv  Detail & Related papers  (2021-05-31T16:12:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.