RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models

• URL: http://arxiv.org/abs/2603.06616v1
• Date: Fri, 20 Feb 2026 08:23:03 GMT
• Title: RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models
• Abstract summary: We formulate LLM routing as the $$-VOR problem to minimize expected set size while controlling the misrouting risk.<n>We propose RACER, extending base routers to output model sets that can be subsequently aggregated for improved output.<n>We theoretically prove that RACER achieves rigorous distribution-free risk control on unseen test data in a post-hoc and model-agnostic manner.
• Score: 23.345746575717545
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Efficiently routing queries to the optimal large language model (LLM) is crucial for optimizing the cost-performance trade-off in multi-model systems. However, most existing routers rely on single-model selection, making them susceptible to misrouting. In this work, we formulate LLM routing as the $α$-VOR problem to minimize expected set size while controlling the misrouting risk, and propose a novel method -- RACER, extending base routers to output model sets that can be subsequently aggregated for improved output. In particular, RACER constructs nested model sets via augmented scoring and utilizes finite-sample concentration bounds to calibrate a threshold that allows for both variable set sizes and abstention. We theoretically prove that RACER achieves rigorous distribution-free risk control on unseen test data in a post-hoc and model-agnostic manner. Extensive experiments verify our theoretical guarantees and demonstrate that RACER consistently enhances downstream accuracy across a wide range of benchmarks.

Related papers

• An Efficient and Modular Framework for Targeted Harm Mitigation in LLMS [3.37470725921294]
  Large Language Models (LLMs) are powerful zero-shot learners but remain prone to misalignment with human preferences.<n>We propose a modular correction framework that augments pretrained LLMs with Activated LoRA adapters and a context-aware routing mechanism.
  arXiv  Detail & Related papers  (2026-09-12T00:24:49Z)
• Before Thinking, Learn to Decide: Proactive Routing for Efficient Visual Reasoning [87.18590662029432]
  We propose PRP, a Proactive Routing Paradigm that enables early decision-making by jointly evaluating the competence of both the draft and target models.<n>Our draft Rating Learning (DRL) equips the draft model with an internal confidence estimator, while Joint Rating Learning (JRL) predicts how well the target model can handle a given query.<n>These ratings enable fine-grained, instance-level textbfProactive Routing and substantially accelerate inference without compromising overall performance.
  arXiv  Detail & Related papers  (2026-06-29T12:30:24Z)
• RouteLMT: Learned Sample Routing for Hybrid LLM Translation Deployment [57.588738943463646]
  Large Language Models (LLMs) have achieved remarkable performance in Machine Translation (MT)<n> deploying them at scale remains prohibitively expensive.<n>We propose textbfRouteLMT, an in-model router that predicts the expected gain by probing the small translators prompt-token representation.
  arXiv  Detail & Related papers  (2026-04-24T13:02:45Z)
• Efficient and Interpretable Multi-Agent LLM Routing via Ant Colony Optimization [58.59491516762626]
  We propose AMRO-S, an efficient and interpretable routing framework for Multi-Agent Systems (MAS)<n>AMRO-S models MAS routing as a semantic-conditioned path selection problem, enhancing routing performance through three key mechanisms.<n>Extensive experiments on five public benchmarks and high-concurrency stress tests demonstrate that AMRO-S consistently improves the quality--cost trade-off over strong routing baselines.
  arXiv  Detail & Related papers  (2026-03-13T12:26:05Z)
• Tiny-Critic RAG: Empowering Agentic Fallback with Parameter-Efficient Small Language Models [7.704706624419061]
  Retrieval-Augmented Generation grounds Large Language Models (LLMs) to mitigate factual hallucinations.<n>We propose Tiny-Critic RAG, a parameter-efficient Small Language Model (SLM) via Low-Rank Adaptation (LoRA)<n>Tiny-Critic RAG achieves routing accuracy comparable to GPT-4o-mini while reducing latency by an order of magnitude.
  arXiv  Detail & Related papers  (2026-03-01T00:16:31Z)
• Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning [20.41220110321494]
  We propose Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning.<n>STEER is a domain-agnostic framework that performs fine-grained, step-level routing between smaller and larger language models.<n>Our results establish model-internal confidence as a robust, domain-agnostic signal for model routing.
  arXiv  Detail & Related papers  (2025-11-09T02:33:08Z)
• CP-Router: An Uncertainty-Aware Router Between LLM and LRM [15.980590561603256]
  Large Language Models (LLMs) produce unnecessarily lengthy outputs even for simple queries.<n>We propose CP-, a training-free and model-agnostic routing framework that dynamically selects between an LLM and an LRM.<n>CP- efficiently reduces token usage while maintaining or even improving accuracy compared to using LRM alone.
  arXiv  Detail & Related papers  (2025-05-26T13:33:31Z)
• How Robust Are Router-LLMs? Analysis of the Fragility of LLM Routing Capabilities [62.474732677086855]
  Large language model (LLM) routing has emerged as a crucial strategy for balancing computational costs with performance.<n>We propose the DSC benchmark: Diverse, Simple, and Categorized, an evaluation framework that categorizes router performance across a broad spectrum of query types.
  arXiv  Detail & Related papers  (2025-03-20T19:52:30Z)
• Confident or Seek Stronger: Exploring Uncertainty-Based On-device LLM Routing From Benchmarking to Generalization [61.02719787737867]
  Large language models (LLMs) are increasingly deployed and democratized on edge devices.<n>One promising solution is uncertainty-based SLM routing, offloading high-stakes queries to stronger LLMs when resulting in low-confidence responses on SLM.<n>We conduct a comprehensive investigation into benchmarking and generalization of uncertainty-driven routing strategies from SLMs to LLMs over 1500+ settings.
  arXiv  Detail & Related papers  (2025-02-06T18:59:11Z)
• Reward-Guided Speculative Decoding for Efficient LLM Reasoning [80.55186052123196]
  We introduce Reward-Guided Speculative Decoding (RSD), a novel framework aimed at improving the efficiency of inference in large language models (LLMs)<n>RSD incorporates a controlled bias to prioritize high-reward outputs, in contrast to existing speculative decoding methods that enforce strict unbiasedness.<n>RSD delivers significant efficiency gains against decoding with the target model only, while achieving significant better accuracy than parallel decoding method on average.
  arXiv  Detail & Related papers  (2025-01-31T17:19:57Z)
• Risk-Controlling Model Selection via Guided Bayesian Optimization [35.53469358591976]
  We find a configuration that adheres to user-specified limits on certain risks while being useful with respect to other conflicting metrics. Our method identifies a set of optimal configurations residing in a designated region of interest. We demonstrate the effectiveness of our approach on a range of tasks with multiple desiderata, including low error rates, equitable predictions, handling spurious correlations, managing rate and distortion in generative models, and reducing computational costs.
  arXiv  Detail & Related papers  (2023-12-04T07:29:44Z)
• Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
  Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios. We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
  arXiv  Detail & Related papers  (2022-04-13T12:43:12Z)
• Modeling the Second Player in Distributionally Robust Optimization [90.25995710696425]
  We argue for the use of neural generative models to characterize the worst-case distribution. This approach poses a number of implementation and optimization challenges. We find that the proposed approach yields models that are more robust than comparable baselines.
  arXiv  Detail & Related papers  (2021-03-18T14:26:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.