Your Retriever Already Knows: Distribution-Shape QPP for RAG Retrieval Sufficiency

• URL: http://arxiv.org/abs/2609.11646v1
• Date: Thu, 10 Sep 2026 14:52:16 GMT
• Title: Your Retriever Already Knows: Distribution-Shape QPP for RAG Retrieval Sufficiency
• Abstract summary: Motivated by a Czech nuclear-regulator deployment, we compare three Query Performance Prediction (QPP) paradigms for retrieval sufficiency in RAG.<n>Our 24 non-lexical features reach a weighted-average AUROC of 0.856 at 2 ms per query, ahead of a classic-QPP literature pool.<n>Adding the LLM judgment as one feature (hybrid) matches S1 on ViDoRe (0.863) but gains a statistically significant edge on SJB.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Standard Retrieval-Augmented Generation (RAG) pipelines often provide no reliable inference-time signal of whether retrieval succeeded; on ambiguous or out-of-scope queries, generation may then hallucinate. Motivated by a Czech nuclear-regulator deployment where data sensitivity precludes third-party LLM APIs, we compare three Query Performance Prediction (QPP) paradigms for retrieval sufficiency in RAG: score-based features, a content-based LLM judge, and a hybrid. On the eight ViDoRe vision domains (14,514 queries), our 24 non-lexical features (GeneralQPP; 15 distribution-shape, 5 query-surface, 4 global) reach a weighted-average AUROC of 0.856 at 2 ms per query, ahead of a classic-QPP literature pool (Classic Full, 0.835) and well above a local Qwen3.5 LLM judge (0.649, +0.207 gap; $\sim$3000$\times$ faster and cheaper per query). Adding the LLM judgment as one feature (hybrid) matches S1 on ViDoRe (0.863) but gains a statistically significant edge on SÚJB (AUROC 0.911 at Hit@5, adversarial-detection 0.954; 1,510 queries, 500 synthetic adversarial), at LLM latency. Rankings agree across datasets (Spearman $ρ= 0.90$). Under Leave-One-Domain-Out, S1 drops to 0.706; a 13-feature LODO-stepwise subset (S1-Lean) recovers to 0.719 (+0.032 over the literature pool).

Related papers

• TriCalRAG: A Three-Strategy, Retrieval-Augmented Benchmark for On-Premise LLM-Based Root Cause Analysis in AIOps [1.8136615181861766]
  Cloud-hosted large language models (LLMs) are increasingly used for root cause analysis (RCA) in AIOps pipelines.<n>They introduce data privacy risk, network latency, and per-query cost that scale poorly with production log volumes.<n>We presentRAG, a benchmark evaluating open-weights served locally via vLLM against a classical LSTM-based log anomaly detector.
  arXiv  Detail & Related papers  (2026-09-13T19:52:46Z)
• Codebook Agent: Amortized Topology Design for LLM Multi-Agent Systems [67.04448659688579]
  We develop a query-independent 16-entry codebook that ranks the top decoded candidates in a single batched forward pass.<n>With no iterative search and no message passing at test time, Codebook Agent is the most accurate method on all six benchmarks.
  arXiv  Detail & Related papers  (2026-09-02T08:10:22Z)
• From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix [55.08143827481755]
  We consolidate traffic from over 200 internal applications onto a single model.<n>We train a separate GRPO expert per axis and merge them via two-stage SLERP.<n>The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.
  arXiv  Detail & Related papers  (2026-09-01T17:39:26Z)
• Sub-Billion, Super-Frontier: Small Language Models Rival Zero-Shot Frontier LLMs on General and Literary Relation Extraction [5.070542698701158]
  Large language models (LLMs) achieve strong relation extraction (RE)<n>We investigate how far small language models (SLMs) can close this gap across general-domain and literary text.
  arXiv  Detail & Related papers  (2026-06-21T17:24:31Z)
• Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs [48.83701310501069]
  Large language models (LLMs) show strong potential for neural architecture generation.<n>Existing approaches produce complete model implementations from scratch.<n>We propose Delta-Code Generation, where fine-tuned LLMs generate compact unified diffs.
  arXiv  Detail & Related papers  (2026-05-06T13:32:05Z)
• Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment [53.72927532626824]
  We show that subsets of just 50 examples (0.3% of data) can achieve over 0.93 Pearson correlation with full benchmark scores.<n>To better predict preferences, we trained regression models on these selected subsets, achieving 0.98 correlation.<n>This demonstrates that in regression modeling, well-curated subsets outpredict the full benchmark, showing quality over quantity.
  arXiv  Detail & Related papers  (2026-04-20T00:57:31Z)
• SelRoute: Query-Type-Aware Routing for Long-Term Conversational Memory Retrieval [0.0]
  We introduce SelRoute, a framework that routes each query to a specialized retrieval pipeline based on its query type.<n>On LongMemEval_M, SelRoute achieves Recall@5 of 0.800 with bge-base-en-v1.5 (109M parameters) and 0.786 with bge-small-en-v1.5 (109M parameters)<n>A zero-ML baseline using FTS5 alone achieves NDCG@5 of 0.692, already exceeding all published baselines on storage quality.
  arXiv  Detail & Related papers  (2026-04-02T18:02:59Z)
• LLM-Guided Probabilistic Fusion for Label-Efficient Document Layout Analysis [6.908972852063454]
  Document layout understanding remains data-intensive despite advances in semi-supervised learning.<n>We present a framework that enhances semi-supervised detection by fusing visual predictions with structural priors.<n>Our method demonstrates consistent gains across model scales.
  arXiv  Detail & Related papers  (2025-11-12T02:25:58Z)
• Guarded Query Routing for Large Language Models [2.2349474636722317]
  We study a guarded query routing problem, for which we first introduce the Guarded Query Routing Benchmark (GQR-Bench)<n>Our results show that WideMLP, enhanced with out-of-domain detection capabilities, yields the best trade-off between accuracy (88%) and speed (4ms)<n>Our findings challenge the automatic reliance on LLMs for (guarded) query routing and provide concrete recommendations for practical applications.
  arXiv  Detail & Related papers  (2025-05-20T15:46:59Z)
• LLM Robustness Against Misinformation in Biomedical Question Answering [50.98256373698759]
  The retrieval-augmented generation (RAG) approach is used to reduce the confabulation of large language models (LLMs) for question answering. We evaluate the effectiveness and robustness of four LLMs against misinformation in answering biomedical questions.
  arXiv  Detail & Related papers  (2024-10-27T16:23:26Z)
• LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond [135.8013388183257]
  We propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits. Most LLMs struggle on SummEdits, with performance close to random chance. The best-performing model, GPT-4, is still 8% below estimated human performance.
  arXiv  Detail & Related papers  (2023-05-23T21:50:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.