When Routing Collapses: On the Degenerate Convergence of LLM Routers

• URL: http://arxiv.org/abs/2602.03478v1
• Date: Tue, 03 Feb 2026 12:51:55 GMT
• Title: When Routing Collapses: On the Degenerate Convergence of LLM Routers
• Abstract summary: As user's cost budget increases, routers systematically default to the most capable and most expensive model.<n>We propose Equi, a decision-aware router that directly learns model rankings.<n>On RouterBench, Equi reduces cost by about 17% at GPT-4-level performance compared to the strongest prior router.
• Score: 46.01380774114097
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM routing aims to achieve a favorable quality--cost trade-off by dynamically assigning easy queries to smaller models and harder queries to stronger ones. However, across both unimodal and multimodal settings, we uncover a pervasive yet underexplored failure mode in existing routers: as the user's cost budget increases, routers systematically default to the most capable and most expensive model even when cheaper models already suffice. As a result, current routers under-utilize small models, wasting computation and monetary cost and undermining the core promise of routing; we term this phenomenon routing collapse. We attribute routing collapse to an objective--decision mismatch: many routers are trained to predict scalar performance scores, whereas routing decisions ultimately depend on discrete comparisons among candidate models. Consequently, small prediction errors can flip relative orderings and trigger suboptimal selections. To bridge this gap, we propose EquiRouter, a decision-aware router that directly learns model rankings, restoring the role of smaller models and mitigating routing collapse. On RouterBench, EquiRouter reduces cost by about 17\% at GPT-4-level performance compared to the strongest prior router. Our code is available at https://github.com/AIGNLAI/EquiRouter.

Related papers

• LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers [31.402829906814365]
  No single large language model (LLM) is optimal across all queries and budget constraints, making model routing essential for cost-effective deployment.<n>We present a unified formulation of LLM routing as a sequential decision process characterized by five components.<n>We develop an automated pipeline for constructing routing supervision and evaluating routers jointly on response quality and inference cost.
  arXiv  Detail & Related papers  (2026-08-07T06:46:58Z)
• SWE-Router: Routing in Multi-turn Agentic Software Engineering Tasks [11.50404356970464]
  Large language models (LLMs) embedded in multi-turn agentic harnesses are reshaping software engineering (SWE)<n>But routing every task to a frontier model is wasteful when many issues admit cheap fixes.<n>We introduce SWE--a value-based temporal approach that lets a cheap model run for a few exploratory turns and reads the resulting partial trajectory.<n>We show that SWE- greatly improves the cost efficiency of SWE tasks, while maintaining the majority of the performances of the stronger model.
  arXiv  Detail & Related papers  (2026-06-30T01:46:26Z)
• Agent-as-a-Router: Agentic Model Routing for Coding Tasks [54.00153674547733]
  Agent-as-a-router (AC) formalizes routing as a C-A-F loop (Context->Action->FeedbackContext)<n>AC achieves the lowest cumulative regret on in-distribution tasks and generalizes to out-of-distribution agentic-programming tasks.
  arXiv  Detail & Related papers  (2026-06-22T06:37:31Z)
• MTRouter: Cost-Aware Multi-Turn LLM Routing with History-Model Joint Embeddings [47.963523156965174]
  We study cost-aware multi-turn routing for large language models.<n>We propose MT, which encodes the interaction history and candidate models into joint history-model embeddings.<n>Experiments show that MT improves the performance-cost trade-off.
  arXiv  Detail & Related papers  (2026-04-26T04:42:21Z)
• Route to Rome Attack: Directing LLM Routers to Expensive Models via Adversarial Suffix Optimization [10.831044672402177]
  Existing routing attacks depend on either white-box access or prompts, rendering them ineffective in real-world black-box scenarios.<n>We propose R$2$A, which aims to mislead black-box LLM routers to expensive models via adversarial suffix optimization.<n>Experiments on multiple open-source and commercial routing systems demonstrate that R$2$A significantly increases the routing rate to expensive models on queries of different distributions.
  arXiv  Detail & Related papers  (2026-04-16T13:51:48Z)
• Efficient and Interpretable Multi-Agent LLM Routing via Ant Colony Optimization [58.59491516762626]
  We propose AMRO-S, an efficient and interpretable routing framework for Multi-Agent Systems (MAS)<n>AMRO-S models MAS routing as a semantic-conditioned path selection problem, enhancing routing performance through three key mechanisms.<n>Extensive experiments on five public benchmarks and high-concurrency stress tests demonstrate that AMRO-S consistently improves the quality--cost trade-off over strong routing baselines.
  arXiv  Detail & Related papers  (2026-03-13T12:26:05Z)
• Routing, Cascades, and User Choice for LLMs [9.28138618885869]
  We study the effect of LLM routing with respect to user behavior.<n>We propose a game between an LLM provider with two models and a user who can re-prompt or abandon tasks.<n>The user's goal is to maximize their utility minus the delay from using the model, while the provider minimizes the cost of servicing the user.
  arXiv  Detail & Related papers  (2026-02-10T15:39:31Z)
• Dr.LLM: Dynamic Layer Routing in LLMs [55.11953638340419]
  Dr.LLM is a retrofittable framework that equips pretrained models with lightweight per-layer routers deciding to skip, execute, or repeat a block.<n>On ARC (logic) and DART (math), Dr.LLM improves accuracy by up to +3.4%p while saving 5 layers per example on average.
  arXiv  Detail & Related papers  (2025-10-14T17:51:26Z)
• ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outliers [14.831117443453165]
  Large language model (LLM) query routers are critical to modern AI platforms.<n>We propose Prox, which applies an exponentially tilted aggregation mechanism to balance bias and variance in nonparametric routers.
  arXiv  Detail & Related papers  (2025-10-10T20:28:14Z)
• xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning [104.63494870852894]
  We present x, a tool-calling-based routing system in which a learned router can either answer directly or invoke one or more external models.<n>Our implementation encompasses the full reinforcement learning framework, including reward and cost accounting.<n>Across diverse benchmarks, x achieves strong cost-performance trade-offs.
  arXiv  Detail & Related papers  (2025-10-09T16:52:01Z)
• Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs [69.2486294522259]
  BaRP is a Bandit Routing-feedback with Preferences approach that trains under the same partial-feedback restriction as deployment.<n> Framed as a contextual bandit over prompt features and a user preference vector, our method simulates an online feedback setting during training and adapts its routing decisions to each new prompt.
  arXiv  Detail & Related papers  (2025-10-08T18:24:59Z)
• Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learning [27.70756702796812]
  We present textbf generalization-R1, a reinforcement learning framework that formulates multi-LLM routing and aggregation as a sequential decision process.<n>To facilitate learning, we employ a lightweight rule-based reward comprising format rewards, final outcome rewards, and a novel cost reward for optimizing the balance between performance and cost.
  arXiv  Detail & Related papers  (2025-06-10T17:56:45Z)
• How Robust Are Router-LLMs? Analysis of the Fragility of LLM Routing Capabilities [62.474732677086855]
  Large language model (LLM) routing has emerged as a crucial strategy for balancing computational costs with performance.<n>We propose the DSC benchmark: Diverse, Simple, and Categorized, an evaluation framework that categorizes router performance across a broad spectrum of query types.
  arXiv  Detail & Related papers  (2025-03-20T19:52:30Z)
• OmniRouter: Budget and Performance Controllable Multi-LLM Routing [31.60019342381251]
  Large language models (LLMs) deliver superior performance but require substantial computational resources and operate with relatively low efficiency.<n>We introduce Omni, a controllable routing framework for multi-LLM serving.<n>Experiments show that Omni achieves up to 6.30% improvement in response accuracy while simultaneously reducing computational costs by at least 10.15%.
  arXiv  Detail & Related papers  (2025-02-27T22:35:31Z)
• MasRouter: Learning to Route LLMs for Multi-Agent Systems [14.029698552632107]
  Multi-agent systems powered by Large Language Models (LLMs) have been demonstrated to push the boundaries of LLM capabilities.<n>Current routing methods effectively reduce overhead in single-agent scenarios by customizing LLM selection for each query.<n>We first introduce the problem of Multi-Agent Routing System (MASR), which integrates all components of MAS into a unified routing framework.<n>Mas is (1) high-performing, achieving a $1.8%sim8.2%$ improvement over the state-of-the-art method on MBPP; (2) economical, reducing overhead by up to $52.07%$ compared to S
  arXiv  Detail & Related papers  (2025-02-16T14:00:59Z)
• Universal Model Routing for Efficient LLM Inference [69.86195589350264]
  Model routing is a technique for reducing the inference cost of large language models (LLMs)<n>We propose UniRoute, a new approach to the problem of dynamic routing, where new, previously unobserved LLMs are available at test time.<n>We show that these are estimates of a theoretically optimal routing rule, and quantify their errors via an excess risk bound.
  arXiv  Detail & Related papers  (2025-02-12T20:30:28Z)
• HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts [34.08858035082419]
  This work introduces HyperRout, which dynamically generates the router's parameters through a fixed hypernetwork and trainable embeddings. Experiments across a wide range of tasks demonstrate the superior performance and efficiency gains of HyperRout.
  arXiv  Detail & Related papers  (2023-12-12T07:40:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.