RealRoute: Dynamic Query Routing System via Retrieve-then-Verify Paradigm

• URL: http://arxiv.org/abs/2604.20860v1
• Date: Mon, 02 Mar 2026 17:57:46 GMT
• Title: RealRoute: Dynamic Query Routing System via Retrieve-then-Verify Paradigm
• Abstract summary: We introduce RealRoute, a framework that shifts the paradigm from predictive routing to a robust Retrieve-then-Verify mechanism.<n> Experiments show that RealRoute significantly outperforms predictive baselines in the multi-hop Rag reasoning task.
• Score: 19.239852494559212
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the success of Retrieval-Augmented Generation (RAG) in grounding LLMs with external knowledge, its application over heterogeneous sources (e.g., private databases, global corpora, and APIs) remains a significant challenge. Existing approaches typically employ an LLM-as-a-Router to dispatch decomposed sub-queries to specific sources in a predictive manner. However, this "LLM-as-a-Router" strategy relies heavily on the semantic meaning of different data sources, often leading to routing errors when source boundaries are ambiguous. In this work, we introduce RealRoute System, a framework that shifts the paradigm from predictive routing to a robust Retrieve-then-Verify mechanism. RealRoute ensures \textit{evidence completeness through parallel, source-agnostic retrieval, followed by a dynamic verifier that cross-checks the results and synthesizes a factually grounded answer}. Our demonstration allows users to visualize the real-time "re-routing" process and inspect the verification chain across multiple knowledge silos. Experiments show that RealRoute significantly outperforms predictive baselines in the multi-hop Rag reasoning task. The RealRoute system is released as an open-source toolkit with a user-friendly web interface. The code is available at the URL: https://github.com/Joseph1951210/RealRoute.

Related papers

• Evidence for Shared Routing Geometry and Dynamics in Sparse Mixture-of-Experts [0.0]
  We show that routing-relevant states across layers share a common geometric structure that is obscured by layer-specific coordinate systems.<n>A matched-rank comparison shows that residual representations are often easier to predict across layers, while router-control states preserve the model's expert choices much more faithfully.
  arXiv  Detail & Related papers  (2026-09-02T10:17:12Z)
• Agent-as-a-Router: Agentic Model Routing for Coding Tasks [54.00153674547733]
  Agent-as-a-router (AC) formalizes routing as a C-A-F loop (Context->Action->FeedbackContext)<n>AC achieves the lowest cumulative regret on in-distribution tasks and generalizes to out-of-distribution agentic-programming tasks.
  arXiv  Detail & Related papers  (2026-06-22T06:37:31Z)
• From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routing [79.19755531338872]
  Existing routing methods treat a model's single response to a query as its capability label for training.<n>We show that this assumption introduces systematic noise into routing supervision, making learned routing policies less reliable.<n>We propose DARS (Distribution-Aware Routing Supervision), a framework that constructs routing supervision from a distributional view of model behavior.
  arXiv  Detail & Related papers  (2026-06-05T05:42:00Z)
• TwinRouterBench: Fast Static and Live Dynamic Evaluation for Realistic Agentic LLM Routing [24.63595849324157]
  We introduce TwinBench, a step-level routing benchmark with two tracks.<n>The static track provides 970 router-visible prefixes from 520 instances across SWE-bench, BFCL, mtRAG, QMSum, and PinchBench.<n>The dynamic track supplies a harness that runs routers on the full 500-case SWE-bench Verified suite.
  arXiv  Detail & Related papers  (2026-05-14T08:58:59Z)
• LatentRouter: Can We Choose the Right Multimodal Model Before Seeing Its Answer? [69.71754384259167]
  Multimodal large language models (MLLMs) have heterogeneous strengths across OCR, chart understanding, spatial reasoning, visual question answering, cost, and latency.<n>We propose Latent, a router that formulates MLLM routing as counterfactual multimodal utility prediction.<n> Experiments on MMR-Bench and VL-Bench show that Latent outperforms fixed-model, feature-level, and learned-router baselines.
  arXiv  Detail & Related papers  (2026-05-11T22:42:12Z)
• Route Before Retrieve: Activating Latent Routing Abilities of LLMs for RAG vs. Long-Context Selection [57.3886742625188]
  Pre-Route is a proactive routing framework that performs structured reasoning before answering.<n>Our study shows three key findings: (i) LLMs possess latent routing ability that can be reliably elicited with guidelines; (ii) linear probes reveal that structured prompts sharpen the separability of the "optimal routing" in representation space; and (iii) distillation transfers this reasoning structure to smaller models for lightweight deployment.
  arXiv  Detail & Related papers  (2026-05-11T09:10:55Z)
• The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Project [30.96691028676722]
  vLLM Semantic Router project has released a series of work spanning signal-driven routing, context-length pool routing, router performance engineering, policy conflict detection, low-latency embedding models, category-aware semantic caching, user-feedback-driven routing adaptation, hallucination detection, and hierarchical content-safety classification for privacy and jailbreak protection.<n>This paper distills those results into the Workload-Pool-Pool architecture, a three-dimensional framework for LLM inference optimization.
  arXiv  Detail & Related papers  (2026-03-22T18:30:11Z)
• Efficient and Interpretable Multi-Agent LLM Routing via Ant Colony Optimization [58.59491516762626]
  We propose AMRO-S, an efficient and interpretable routing framework for Multi-Agent Systems (MAS)<n>AMRO-S models MAS routing as a semantic-conditioned path selection problem, enhancing routing performance through three key mechanisms.<n>Extensive experiments on five public benchmarks and high-concurrency stress tests demonstrate that AMRO-S consistently improves the quality--cost trade-off over strong routing baselines.
  arXiv  Detail & Related papers  (2026-03-13T12:26:05Z)
• DeepSieve: Information Sieving via LLM-as-a-Knowledge-Router [57.28685457991806]
  DeepSieve is an agentic RAG framework that incorporates information sieving via LLM-as-a-knowledge-router.<n>Our design emphasizes modularity, transparency, and adaptability, leveraging recent advances in agentic system design.
  arXiv  Detail & Related papers  (2025-07-29T17:55:23Z)
• IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theory [26.39979967537193]
  Large language models (LLMs) have demonstrated exceptional performance across a wide range of natural language tasks.<n>While powerful models deliver better results, they come at a high cost, whereas smaller models are more cost-effective but less capable.<n>We propose IRT-Merci, a multi-LLM routing framework that efficiently routes user queries to the most suitable LLM.
  arXiv  Detail & Related papers  (2025-06-01T15:14:58Z)
• RAGRouter: Learning to Route Queries to Multiple Retrieval-Augmented Language Models [45.58601993849455]
  Retrieval-Augmented Generation (RAG) significantly improves the performance of Large Language Models (LLMs) on knowledge-intensive tasks.<n>We observe that external documents dynamically affect LLMs' ability to answer queries, while existing routing methods exhibit suboptimal performance in RAG scenarios.<n>We propose RAG, a parametric RAG-aware routing design, which leverages document embeddings and RAG capability embeddings with contrastive learning to capture knowledge representation shifts.
  arXiv  Detail & Related papers  (2025-05-29T03:44:56Z)
• Learning to Route Queries Across Knowledge Bases for Step-wise Retrieval-Augmented Reasoning [60.84901522792042]
  Multimodal Retrieval-Augmented Generation (MRAG) has shown promise in mitigating hallucinations in Multimodal Large Language Models (MLLMs)<n>We propose R1, a novel MRAG framework that learns to decide when and where to retrieve knowledge based on the evolving reasoning state.<n>R1- can adaptively and effectively leverage diverse KBs, reducing unnecessary retrievals and improving both efficiency and accuracy.
  arXiv  Detail & Related papers  (2025-05-28T08:17:57Z)
• Universal Model Routing for Efficient LLM Inference [69.86195589350264]
  Model routing is a technique for reducing the inference cost of large language models (LLMs)<n>We propose UniRoute, a new approach to the problem of dynamic routing, where new, previously unobserved LLMs are available at test time.<n>We show that these are estimates of a theoretically optimal routing rule, and quantify their errors via an excess risk bound.
  arXiv  Detail & Related papers  (2025-02-12T20:30:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.