Pre-retrieval Query Clustering for Adaptive Top-k Document Retrieval in RAG Systems

• URL: http://arxiv.org/abs/2609.13489v2
• Date: Tue, 15 Sep 2026 01:11:04 GMT
• Title: Pre-retrieval Query Clustering for Adaptive Top-k Document Retrieval in RAG Systems
• Abstract summary: RAG systems commonly retrieve a fixed number of documents to ground generation.<n>simple queries suffer over-retrieval (adding noise and cost) while complex queries are under-retrieved.<n>We propose a practical, general framework for query-adaptive retrieval depth.
• Score: 7.440012812118169
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: RAG systems commonly retrieve a fixed number of documents (top-k) to ground generation, but this static approach is brittle: simple queries suffer over-retrieval (adding noise and cost) while complex queries are under-retrieved, causing recall failures that cascade into incorrect answers. Motivated by the question of how many documents must be retrieved to answer an arbitrary query reliably, we propose a practical, general framework for query-adaptive retrieval depth. Offline, we estimate per-query retrieval difficulty by measuring NDCG under the default retriever and deriving a query-specific saturation point k* from the NDCG-k curve. Because computing these signals online is expensive, we cluster a large set of queries in embedding space and summarize each cluster with a recommended retrieval depth that targets high coverage (e.g., ~95%) using a mean-plus-variance rule. At runtime, the system assigns an incoming query to a cluster and selects the corresponding top-k in constant time. Compared with post-retrieval confidence methods that rely on clustering retrieved documents, our approach is pre-retrieval and query-centric, making it robust in heterogeneous, case-like corpora and applicable across domains such as legal, healthcare, finance, and enterprise search. Finally, this framework has been tested in full-traffic queries that improved $F_1$ by over 36% while reducing token usage by 14% on low-complexity clusters without accuracy loss.

Related papers

• ORDER: Task-Conditioned Routing for Retrieval-Augmented Generation [3.6028767235872965]
  In this paper, we introduce a query-conditioned RAG framework that jointly adapts indexing and retrieval to the incoming query.<n>We evaluate our framework on large-scale, heterogeneous historical archives and show that conditioning both indexing and retrieval on the query consistently outperforms both naive baselines and strong state-of-the-art RAG systems.
  arXiv  Detail & Related papers  (2026-09-15T11:23:16Z)
• Retriever Portfolios: A Principled Approach to Adaptive RAG [30.668406304237404]
  We propose a method that automatically selects a small, diverse subset of retrievers from a large pool of candidates, to cover different regions of the target query distribution.<n>Across multiple QA benchmarks, our learned portfolios and router pipeline consistently outperform single-retriever and naive multi-retriever baselines.
  arXiv  Detail & Related papers  (2026-05-29T11:43:05Z)
• Query Decomposition for RAG: Balancing Exploration-Exploitation [83.79639293409802]
  RAG systems address complex user requests by decomposing them into subqueries, retrieving potentially relevant documents for each, and then aggregating them to generate an answer.<n>We formulate query decomposition and document retrieval in an exploitation-exploration setting, where retrieving one document at a time builds a belief about the utility of a given sub-queries.<n>Our main finding is that estimating document relevance using rank information and human judgments yields a 35% gain in document-level precision, 15% increase in alpha-nDCG, and better performance on the downstream task of long-form generation.
  arXiv  Detail & Related papers  (2025-10-21T13:37:11Z)
• Evaluating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queries [53.99620546358492]
  Real-world use cases often present RAG systems with complex queries for which relevant information is missing from the corpus or is incomplete.<n>Existing RAG benchmarks rarely reflect realistic task complexity for multi-hop or out-of-scope questions.<n>We present the first pipeline for automatic, difficulty-controlled creation of un$underlinec$heatable, $underliner$ealistic, $underlineu$nanswerable, and $underlinem$ulti-hop.
  arXiv  Detail & Related papers  (2025-10-13T21:38:04Z)
• Cluster-based Adaptive Retrieval: Dynamic Context Selection for RAG Applications [8.946586077722822]
  Cluster-based Adaptive Retrieval (CAR) is an algorithm that determines the optimal number of documents by analyzing the clustering patterns of ordered query-document similarity distances.<n>CAR consistently picks the optimal retrieval depth and achieves the highest TES score, outperforming every fixed top-k baseline.
  arXiv  Detail & Related papers  (2025-10-02T05:11:12Z)
• Learning to Route: A Rule-Driven Agent Framework for Hybrid-Source Retrieval-Augmented Generation [55.47971671635531]
  Large Language Models (LLMs) have shown remarkable performance on general Question Answering (QA)<n>Retrieval-Augmented Generation (RAG) addresses this limitation by enriching LLMs with external knowledge.<n>Existing systems primarily rely on unstructured documents, while largely overlooking relational databases.
  arXiv  Detail & Related papers  (2025-09-30T22:19:44Z)
• MODE: Mixture of Document Experts for RAG [0.532836690371986]
  MODE replaces fine-grained nearest-neighbor search with cluster-and-route retrieval.<n>Documents are embedded, grouped into semantically coherent clusters, and represented by cached centroids.<n>On HotpotQA and SQuAD corpora with 100-500 chunks, MODE matches or exceeds a dense-retrieval baseline in answer quality.
  arXiv  Detail & Related papers  (2025-08-27T17:45:16Z)
• Chain of Retrieval: Multi-Aspect Iterative Search Expansion and Post-Order Search Aggregation for Full Paper Retrieval [68.71038700559195]
  Chain of Retrieval(COR) is a novel iterative framework for full-paper retrieval.<n>We present SCIBENCH, a benchmark providing both complete and segmented contexts of full papers for queries and candidates.
  arXiv  Detail & Related papers  (2025-07-14T08:41:53Z)
• Tree-Based Text Retrieval via Hierarchical Clustering in RAGFrameworks: Application on Taiwanese Regulations [0.0]
  We propose a hierarchical clustering-based retrieval method that eliminates the need to predefine k.<n>Our approach maintains the accuracy and relevance of system responses while adaptively selecting semantically relevant content.<n>Our framework is simple to implement and easily integrates with existing RAG pipelines, making it a practical solution for real-world applications under limited resources.
  arXiv  Detail & Related papers  (2025-06-16T15:34:29Z)
• Optimization of Retrieval-Augmented Generation Context with Outlier Detection [0.0]
  We focus on methods to reduce the size and improve the quality of the prompt context required for question-answering systems. Our goal is to select the most semantically relevant documents, treating the discarded ones as outliers. It was found that the greatest improvements were achieved with increasing complexity of the questions and answers.
  arXiv  Detail & Related papers  (2024-07-01T15:53:29Z)
• Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity [59.57065228857247]
  Retrieval-augmented Large Language Models (LLMs) have emerged as a promising approach to enhancing response accuracy in several tasks, such as Question-Answering (QA) We propose a novel adaptive QA framework, that can dynamically select the most suitable strategy for (retrieval-augmented) LLMs based on the query complexity. We validate our model on a set of open-domain QA datasets, covering multiple query complexities, and show that ours enhances the overall efficiency and accuracy of QA systems.
  arXiv  Detail & Related papers  (2024-03-21T13:52:30Z)
• Optimizing Test-Time Query Representations for Dense Retrieval [34.61821330771046]
  TOUR improves query representations guided by test-time retrieval results. We leverage a cross-encoder re-ranker to provide fine-grained pseudo labels over retrieval results. TOUR consistently improves direct re-ranking by up to 2.0% while running 1.3-2.4x faster.
  arXiv  Detail & Related papers  (2022-05-25T11:39:42Z)
• Session-Aware Query Auto-completion using Extreme Multi-label Ranking [61.753713147852125]
  We take the novel approach of modeling session-aware query auto-completion as an e Multi-Xtreme Ranking (XMR) problem. We adapt a popular XMR algorithm for this purpose by proposing several modifications to the key steps in the algorithm. Our approach meets the stringent latency requirements for auto-complete systems while leveraging session information in making suggestions.
  arXiv  Detail & Related papers  (2020-12-09T17:56:22Z)
• Query Focused Multi-Document Summarization with Distant Supervision [88.39032981994535]
  Existing work relies heavily on retrieval-style methods for estimating the relevance between queries and text segments. We propose a coarse-to-fine modeling framework which introduces separate modules for estimating whether segments are relevant to the query. We demonstrate that our framework outperforms strong comparison systems on standard QFS benchmarks.
  arXiv  Detail & Related papers  (2020-04-06T22:35:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.