How Often Should a Recommender Call an LLM? Value-Weighted Routing, Monitoring, and Seasonal Robustness

• URL: http://arxiv.org/abs/2607.25068v1
• Date: Mon, 27 Jul 2026 20:52:15 GMT
• Title: How Often Should a Recommender Call an LLM? Value-Weighted Routing, Monitoring, and Seasonal Robustness
• Abstract summary: We present a fully synthetic simulation of a retail merchandising pipeline that routes items using only estimated difficulty and estimated value, never ground truth.<n>The study has three stages. First, a value-weighted threshold router is compared with a difficulty-only and a random baseline on a synthetic catalog.<n>Second, a decision logger and monitor expose a failure mode hidden by aggregate metrics showing that the aggregate result is driven almost entirely by between-category differences.
• Score: 0.15229257192293197
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Routing decisions between a cheap heuristic and an expensive large language model (LLM) are typically framed as a difficulty problem: send the hard cases to the expensive path. We argue this framing is incomplete because difficulty and business value are distinct axes - a difficult cheap item and a difficult costly item do not have the same cost of error. We present Value Router, a fully synthetic simulation of a retail merchandising pipeline that routes items using only estimated difficulty and estimated value, never ground truth. The study has three stages. First, a value-weighted threshold router is compared with a difficulty-only and a random baseline on a synthetic catalog with an inverse correlation between category volume and value. Value-weighting matches the difficulty-only baseline's recall of true high-value items (60%) while achieving substantially higher precision (98.3% vs. 94.3%). Second, a decision logger and monitor expose a failure mode hidden by aggregate metrics showing that the aggregate result is driven almost entirely by between-category differences rather than per-item discrimination. Third, a simulated Black Friday demand surge (2.5 volume with a shift toward higher-value categories) compares a static router, a seasonally tuned router, and two slow-path budget policies. All results are from a controlled synthetic simulation with experimenter-defined ground truth and illustrate design principles for cost-aware routing systems rather than validated real-world claims.

Related papers

• Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation [64.51088225051959]
  Heterogeneous AI systems can improve quality and efficiency by routing queries to the specialist who can answer most effectively at the lowest cost.<n>We formalize this tradeoff as an instance of Pandora's Box, the classical problem of optimal search with costly inspection.<n>Under a Gaussian signal model, the resulting policies have closed-form value-of-information expressions that determine, for each specialist and input, whether refining the value estimate is worth its cost.
  arXiv  Detail & Related papers  (2026-08-20T17:54:37Z)
• VDAR-Router: Adaptive LLMs Routing via Verbalized Query Difficulty Analysis Retrieval [16.067071673823417]
  We propose VDAR-the, a difficulty-aware retrieval-based routing framework.<n>For each input query, VDAR-the first generates an explicit difficulty analysis.<n>It then retrieves historical examples with similar difficulty profiles.
  arXiv  Detail & Related papers  (2026-07-20T16:00:45Z)
• Cluster, Route, Escalate: Cascaded Framework for Cost-Aware LLM Serving [69.5155152169752]
  We propose a two-stage cascaded solution for large language models (LLMs)<n>Stage 1 clusters incoming queries and assigns each cluster to its most cost-effective model.<n>Stage 2 adds a quality estimation (QE) cascade; when an output from Stage 1 is judged low-quality, the query is escalated to a stronger model.<n>On the test datasets, the cascaded system retains 97-99% of the strongest model's accuracy while reducing Time Per Output Token (TPOT)<n>It requires only task-correctness labels and adapts to changes in the model pool without manual reconfiguration.
  arXiv  Detail & Related papers  (2026-06-25T18:29:24Z)
• Unsolvability Ceiling in Multi-LLM Routing: An Empirical Study of Evaluation Artifacts [0.07812854697536452]
  We present a large-scale study of multi-tier LLM routing with 206,000 query-model pairs across six benchmarks.<n>We show that a substantial portion of reported unsolvability stems from evaluation artifacts.
  arXiv  Detail & Related papers  (2026-05-08T07:49:24Z)
• MTRouter: Cost-Aware Multi-Turn LLM Routing with History-Model Joint Embeddings [47.963523156965174]
  We study cost-aware multi-turn routing for large language models.<n>We propose MT, which encodes the interaction history and candidate models into joint history-model embeddings.<n>Experiments show that MT improves the performance-cost trade-off.
  arXiv  Detail & Related papers  (2026-04-26T04:42:21Z)
• The Price Reversal Phenomenon: When Cheaper Reasoning Models End Up Costing More [76.93600828673503]
  Listed API pricing is an unreliable proxy for actual cost.<n>Removing thinking token costs reduces ranking reversals by 70%.<n>Our findings call for cost-aware model selection and transparent per-request cost monitoring.
  arXiv  Detail & Related papers  (2026-03-25T06:07:39Z)
• xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning [104.63494870852894]
  We present x, a tool-calling-based routing system in which a learned router can either answer directly or invoke one or more external models.<n>Our implementation encompasses the full reinforcement learning framework, including reward and cost accounting.<n>Across diverse benchmarks, x achieves strong cost-performance trade-offs.
  arXiv  Detail & Related papers  (2025-10-09T16:52:01Z)
• Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs [69.2486294522259]
  BaRP is a Bandit Routing-feedback with Preferences approach that trains under the same partial-feedback restriction as deployment.<n> Framed as a contextual bandit over prompt features and a user preference vector, our method simulates an online feedback setting during training and adapts its routing decisions to each new prompt.
  arXiv  Detail & Related papers  (2025-10-08T18:24:59Z)
• Meta-Router: Bridging Gold-standard and Preference-based Evaluations in Large Language Model Routing [15.724480880994259]
  A large language model (LLM) router selects the most appropriate model from a pool of candidates for each query.<n> preference-based data, collected via crowdsourcing or LLM-as-a-judge systems, are cheaper and more scalable, yet often biased in reflecting the true quality of responses.<n>We develop an integrative causal router training framework that corrects preference-data bias, address imbalances between two data sources, and improve routing robustness and efficiency.
  arXiv  Detail & Related papers  (2025-09-29T21:44:00Z)
• SynapseRoute: An Auto-Route Switching Framework on Dual-State Large Language Model [12.929385845055137]
  We show that approximately 58% of medical questions can be accurately answered by the non-thinking mode alone.<n>We propose SynapseRoute, a machine learning-based dynamic routing framework that intelligently assigns input queries to either thinking or non-thinking modes.
  arXiv  Detail & Related papers  (2025-07-03T17:33:58Z)
• How Robust Are Router-LLMs? Analysis of the Fragility of LLM Routing Capabilities [62.474732677086855]
  Large language model (LLM) routing has emerged as a crucial strategy for balancing computational costs with performance.<n>We propose the DSC benchmark: Diverse, Simple, and Categorized, an evaluation framework that categorizes router performance across a broad spectrum of query types.
  arXiv  Detail & Related papers  (2025-03-20T19:52:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.