Less can be More: Relieving RAG Bottlenecks via Evidence Frontloading and Pressure-Adaptive Budgeting

• URL: http://arxiv.org/abs/2608.25115v1
• Date: Tue, 25 Aug 2026 20:10:06 GMT
• Title: Less can be More: Relieving RAG Bottlenecks via Evidence Frontloading and Pressure-Adaptive Budgeting
• Abstract summary: We show that upstream reranking can become the dominant bottleneck under high query rates or large reranking budgets.<n>Reducing the reranking budget can relieve this bottleneck, but it may also drop supporting evidence and degrade recall.<n>We propose textbftextsfPACE, a training-free framework that combines textitevidence frontloading with textitpressure-adaptive budgeting.
• Score: 3.685453775072903
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing methods for improving Retrieval-Augmented Generation (RAG) efficiency mainly optimize downstream LLM generation, such as context compression or serving optimization. However, RAG is an end-to-end system, and its bottleneck can shift between upstream reranking and downstream generation under different serving loads and reranking budgets.In this paper, we first empirically characterize this shifting-bottleneck behavior and show that upstream reranking can become the dominant bottleneck under high query rates or large reranking budgets. Reducing the reranking budget can relieve this bottleneck, but it may also drop supporting evidence and degrade recall. To address this problem, we propose \textbf{\textsf{PACE}} (\textbf{P}rioritized \textbf{A}daptive \textbf{C}overage of \textbf{E}vidence), a training-free framework that combines \textit{evidence frontloading} with \textit{pressure-adaptive budgeting}. \textsf{PACE} first reorders candidates by marginal evidence coverage, prioritizing documents that are query-relevant, complementary, and useful for forming multi-hop evidence chains. We show that this objective is monotone submodular, giving greedy selection a $(1-1/e)$ approximation guarantee. \textsf{PACE} then dynamically adjusts the reranking budget according to the relative pressure of the reranker and the LLM. Experiments on three multi-hop QA datasets and online serving simulations show that \textsf{PACE} improves evidence recall, reduces p95 latency under ranking-heavy workloads. More importantly, the two components together reveal that \textit{less can be more}: an evidence-dense top-ranked candidates enable higher final recall with fewer reranked documents.

Related papers

• TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents [54.88208747581851]
  Training-free visual token pruning can reduce this cost, but cache reuse introduces a fundamental constraint.<n>We propose textbfmethod, a training-free framework for emphtextbfTrajectory-textbfrobust textbfAdmission and textbfCoverage-aware textbfEvidence ordering.
  arXiv  Detail & Related papers  (2026-09-09T15:12:48Z)
• ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping [79.1905186547363]
  ResiSpec is a framework that strategically reforms the proposal distribution during verification.<n>It prevents candidate obsolescence and achieves up to 1.92$times$ speedup over state-of-the-art multi-candidate methods.
  arXiv  Detail & Related papers  (2026-08-25T11:25:55Z)
• Lightweight Chunk Selection for Mobile Retrieval-Augmented Generation [8.227745781139634]
  We study lightweight RAG chunk selection as an evidence-alignment problem.<n>Our selector combines three complementary feature sources: question hidden states that represent LLM-side query intent, MoE routing-derived expert signals that capture the generator's internal routing structure, and retrieved chunk embeddings that preserve candidate-side evidence geometry.<n>Experiments show that the proposed selector consistently improves rank-1 evidence selection over mobile-applicable baselines by an average of 2.5%.
  arXiv  Detail & Related papers  (2026-08-04T05:26:25Z)
• miniReranker: Efficient Multimodal Reranking through Visual Cache Reuse and Interaction Sparsity [21.54829080388454]
  Multimodal large language models (MLLMs) have recently shown strong potential as point-wise rerankers.<n>However, point-wise reranking suffers from substantial repeated computation across query--document pairs.<n>We propose a $textitvision-first$ formulation that improves both cache reuse efficiency and reranking performance.
  arXiv  Detail & Related papers  (2026-06-09T12:11:02Z)
• Learning to Solve, Forgetting to Retain: Correct-Set Turnover in RLVR [16.681943272535104]
  Reinforcement learning with verifiable rewards (RLVR) improves the ability of large language model, yet headline accuracy gains often conceal a hidden cost.<n>We propose textbfmethod, a retention-aware review mechanism that tracks mastered prompts and periodically reintroduces them to textbfremind the model of previous solutions.
  arXiv  Detail & Related papers  (2026-06-02T03:17:34Z)
• F-GRPO: Factorized Group-Relative Policy Optimization for Unified Candidate Generation and Ranking [79.49893545611779]
  Large Language Models (LLMs) can generate a subset and order it within a single autoregressive pass.<n>This flexibility introduces a new optimization challenge: the model must search an output space while receiving utility feedback only after the full ranked list is generated.<n>This credit assignment gap makes end-to-end optimization unstable and sample-inefficient.<n>We propose a unified framework that performs both within a single autoregressive rollout and optimize them end-to-end via factorized group-relative policy optimization.
  arXiv  Detail & Related papers  (2026-05-13T04:52:33Z)
• PAR$^2$-RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering [57.89576196160413]
  Large language models (LLMs) remain brittle on multi-hop question answering (MHQA)<n>We propose textbfPlanned Active Retrieval and Reasoning RAG (PAR$2$-RAG), a two-stage framework that separates emphcoverage from emphcommitment.
  arXiv  Detail & Related papers  (2026-03-30T23:52:54Z)
• Less is More for RAG: Information Gain Pruning for Generator-Aligned Reranking and Evidence Selection [26.57845149504473]
  RAG grounds large language models with external evidence, but under a limited context budget, the key challenge is deciding which retrieved passages should be injected.<n>We show that retrieval relevance metrics correlate weakly with end-to-end QA quality and can even become negatively correlated under multi-passage injection.<n>We propose textbfInformation Gain Pruning (IGP), a deployment-friendly reranking-and-pruning module that selects evidence using a generator-aligned utility signal and filters weak or harmful passages before truncation.
  arXiv  Detail & Related papers  (2026-01-24T17:14:10Z)
• High-Rank Structured Modulation for Parameter-Efficient Fine-Tuning [57.85676271833619]
  Low-rank Adaptation (LoRA) uses a low-rank update method to simulate full parameter fine-tuning.<n>We present textbfSMoA, a high-rank textbfStructured textbfMOdulation textbfAdapter that uses fewer trainable parameters while maintaining a higher rank.
  arXiv  Detail & Related papers  (2026-01-12T13:06:17Z)
• When, What, and How: Rethinking Retrieval-Enhanced Speculative Decoding [29.402164743559]
  ReSpec is a novel framework that transforms drafter switching into adaptive decision-making.<n>Experiments on Spec-Bench demonstrate that ReSpec state-of-the-art acceleration achieves over $33%$ and $25%$, respectively.
  arXiv  Detail & Related papers  (2025-11-03T06:57:16Z)
• Enhancing Retrieval-Augmented Generation with Two-Stage Retrieval: FlashRank Reranking and Query Expansion [0.0]
  RAG couples a retriever with a large language model (LLM) to ground generated responses in external evidence.<n>We propose a two-stage retrieval pipeline that integrates LLM-driven query expansion to improve candidate recall.<n>FlashRank is a fast marginal-utility reranker that dynamically selects an optimal subset of evidence under a token budget.
  arXiv  Detail & Related papers  (2025-10-17T15:08:17Z)
• On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention [53.22963042513293]
  Large language models (LLMs) excel at capturing global token dependencies via self-attention but face prohibitive compute and memory costs on lengthy inputs.<n>We first propose dual-state linear attention (A), a novel design that maintains two hidden states-one for preserving historical context and one for tracking recencythereby mitigating the short-range bias typical of linear-attention architectures.<n>We introduce DSLA-Serve, an online adaptive distillation framework that progressively replaces Transformer layers DSLA layers at inference time, guided by a sensitivity-based layer ordering.
  arXiv  Detail & Related papers  (2025-06-11T01:25:06Z)
• Gumbel Reranking: Differentiable End-to-End Reranker Optimization [61.16471123356738]
  RAG systems rely on rerankers to identify relevant documents.<n> fine-tuning these models remains challenging due to the scarcity of annotated query-document pairs.<n>We propose Gumbel Reranking, an end-to-end training framework for rerankers aimed at minimizing the training-inference gap.
  arXiv  Detail & Related papers  (2025-02-16T13:23:39Z)
• HiRE: High Recall Approximate Top-$k$ Estimation for Efficient LLM Inference [68.59839755875252]
  HiRE comprises of two novel components: (i) a compression scheme to cheaply predict top-$k$ rows/columns with high recall, followed by full computation restricted to the predicted subset, and (ii) DA-TOP-$k$: an efficient multi-device approximate top-$k$ operator. We demonstrate that on a one billion parameter model, HiRE applied to both the softmax as well as feedforward layers, achieves almost matching pretraining and downstream accuracy, and speeds up inference latency by $1.47times$ on a single TPUv5e device.
  arXiv  Detail & Related papers  (2024-02-14T18:04:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.