Less is More: Compact Clue Selection for Efficient Retrieval-Augmented Generation Reasoning

• URL: http://arxiv.org/abs/2502.11811v6
• Date: Thu, 23 Oct 2025 10:36:01 GMT
• Title: Less is More: Compact Clue Selection for Efficient Retrieval-Augmented Generation Reasoning
• Abstract summary: Current RAG retrievers are designed primarily for human readers, emphasizing complete, readable, and coherent paragraphs.<n>Existing methods often rely on reranking or summarization to identify key sentences, but may suffer from semantic breaks and unfaithfulness.<n>We propose CompSelect, a compact clue Selection mechanism for LLM-centric RAG applications.
• Score: 30.217454611494276
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current RAG retrievers are designed primarily for human readers, emphasizing complete, readable, and coherent paragraphs. However, LLMs benefit more from precise, compact, and well-structured input, which enhances reasoning quality and efficiency. Existing methods often rely on reranking or summarization to identify key sentences, but may suffer from semantic breaks and unfaithfulness. Thus, efficiently extracting and organizing answer-relevant clues from large-scale documents while reducing LLM reasoning costs remains a challenge for RAG. Inspired by Occam's razor, we frame LLM-centric retrieval as a MinMax optimization: maximizing the extraction of potential clues and reranking them for well-organization, while minimizing reasoning costs by truncating to the smallest sufficient clues set. In this paper, we propose CompSelect, a Compact clue Selection mechanism for LLM-centric RAG, consisting of a clue extractor, a reranker, and a truncator. (1) The clue extractor first uses answer-containing sentences as fine-tuning targets, aiming to extract sufficient potential clues; (2) The reranker is trained to prioritize effective clues based on real LLM feedback; (3) The truncator uses the truncated text containing the minimum sufficient clues for answering the question as fine-tuning targets, thereby enabling efficient RAG reasoning. Experiments on three QA datasets show that CompSelect improves QA performance by approximately 11\% and reduces Total Latency and Online Latency by approximately 17\% and 67\% compared to various baseline methods on both LLaMA3 and Qwen3. Further analysis confirms its robustness to unreliable retrieval and generalization across different scenarios, offering a scalable and cost-efficient solution for web-scale RAG applications.

Related papers

• ThoughtFold: Folding Reasoning Chains via Introspective Preference Learning [69.64972562984882]
  We propose ThoughtFold, a framework that leverages fine-grained preference learning to redundant explorations for efficient reasoning.<n>We show that ThoughtFold significantly enhances efficiency.<n>It reduces the token usage of DeepSeek-R1-Distill-Qwen-7B by approximately 56% while maintaining state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2026-06-02T11:21:27Z)
• Route Before Retrieve: Activating Latent Routing Abilities of LLMs for RAG vs. Long-Context Selection [57.3886742625188]
  Pre-Route is a proactive routing framework that performs structured reasoning before answering.<n>Our study shows three key findings: (i) LLMs possess latent routing ability that can be reliably elicited with guidelines; (ii) linear probes reveal that structured prompts sharpen the separability of the "optimal routing" in representation space; and (iii) distillation transfers this reasoning structure to smaller models for lightweight deployment.
  arXiv  Detail & Related papers  (2026-05-11T09:10:55Z)
• Your Dense Retriever is Secretly an Expeditious Reasoner [12.123445960145693]
  We propose Adaptive Query Reasoning (AdaQR), a hybrid query rewriting framework.<n>AdaQR reduces reasoning cost by 28% while preserving-or even improving-retrieval performance by 7%.
  arXiv  Detail & Related papers  (2025-09-27T16:50:03Z)
• From Long to Short: LLMs Excel at Trimming Own Reasoning Chains [48.692414597960244]
  O1/R1 style large reasoning models (LRMs) signal a substantial leap forward over conventional instruction-following LLMs.<n>Recent studies show that LRMs are prone to suffer from overthinking.<n>We propose a test-time scaling method, EDIT, which efficiently guides LRMs to identify the shortest correct reasoning paths at test time.
  arXiv  Detail & Related papers  (2025-09-07T19:00:44Z)
• REFRAG: Rethinking RAG based Decoding [67.4862300145604]
  REFRAG is an efficient decoding framework that compresses, senses, and expands to improve latency in RAG applications.<n>We provide rigorous validation of REFRAG across diverse long-context tasks, including RAG, multi-turn conversations, and long document summarization.
  arXiv  Detail & Related papers  (2025-09-01T03:31:44Z)
• Distilling a Small Utility-Based Passage Selector to Enhance Retrieval-Augmented Generation [110.610512800947]
  Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating retrieved information.<n>In RAG, the emphasis has shifted to utility, which considers the usefulness of passages for generating accurate answers.<n>Our approach focuses on utility-based selection rather than ranking, enabling dynamic passage selection tailored to specific queries without the need for fixed thresholds.<n>Our experiments demonstrate that utility-based selection provides a flexible and cost-effective solution for RAG, significantly reducing computational costs while improving answer quality.
  arXiv  Detail & Related papers  (2025-07-25T09:32:29Z)
• Fractured Chain-of-Thought Reasoning [61.647243580650446]
  We introduce Fractured Sampling, a unified inference-time strategy that interpolates between full CoT and solution-only sampling.<n>We show that Fractured Sampling consistently achieves superior accuracy-cost trade-offs, yielding steep log-linear scaling gains in Pass@k versus token budget.
  arXiv  Detail & Related papers  (2025-05-19T11:30:41Z)
• CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capability [55.46506909726119]
  We introduce $textbfCAFE$, a two-stage coarse-to-fine method to enhance multi-document question-answering capacities.<n>CAFE achieves up to 22.1% and 13.7% SubEM improvement over SFT and RAG methods on the Mistral model, respectively.
  arXiv  Detail & Related papers  (2025-05-15T08:05:12Z)
• Pseudo Relevance Feedback is Enough to Close the Gap Between Small and Large Dense Retrieval Models [29.934928091542375]
  Scaling dense retrievers to larger large language model (LLM) backbones has been a dominant strategy for improving their retrieval effectiveness.<n>We introduce PromptPRF, a feature-based pseudo-relevance feedback (PRF) framework that enables small LLM-based dense retrievers to achieve effectiveness comparable to much larger models.
  arXiv  Detail & Related papers  (2025-03-19T04:30:20Z)
• Stepwise Informativeness Search for Efficient and Effective LLM Reasoning [49.85349030928302]
  Recent studies show that Large Language Models (LLMs) tend to lose focus over the middle of long contexts.<n>We propose guiding LLMs to generate more accurate and concise step-by-step rationales.
  arXiv  Detail & Related papers  (2025-02-21T09:39:27Z)
• Gumbel Reranking: Differentiable End-to-End Reranker Optimization [61.16471123356738]
  RAG systems rely on rerankers to identify relevant documents.<n> fine-tuning these models remains challenging due to the scarcity of annotated query-document pairs.<n>We propose Gumbel Reranking, an end-to-end training framework for rerankers aimed at minimizing the training-inference gap.
  arXiv  Detail & Related papers  (2025-02-16T13:23:39Z)
• GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigation [62.63014905981601]
  Refusal-Aware Instruction Tuning (RAIT) aims to enhance Large Language Models (LLMs) by improving their ability to refuse responses to questions beyond their knowledge.<n>Effective RAIT must address two key challenges: firstly, effectively reject unknown questions to minimize hallucinations; secondly, avoid over-refusal to ensure questions that can be correctly answered are not rejected.<n>GraIT employs gradient-driven sample selection to effectively minimize hallucinations and (2) introduces an adaptive weighting mechanism during fine-tuning to reduce the risk of over-refusal.
  arXiv  Detail & Related papers  (2025-02-09T14:11:30Z)
• MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation [34.66546005629471]
  Large Language Models (LLMs) are essential tools for various natural language processing tasks but often suffer from generating outdated or incorrect information.<n>Retrieval-Augmented Generation (RAG) addresses this issue by incorporating external, real-time information retrieval to ground LLM responses.<n>To tackle this problem, we propose Multi-Agent Filtering Retrieval-Augmented Generation (MAIN-RAG)<n>MAIN-RAG is a training-free RAG framework that leverages multiple LLM agents to collaboratively filter and score retrieved documents.
  arXiv  Detail & Related papers  (2024-12-31T08:07:26Z)
• Bridging Relevance and Reasoning: Rationale Distillation in Retrieval-Augmented Generation [43.50677378728461]
  We propose RADIO, a novel and practical preference alignment framework with RAtionale DIstillatiOn.<n>We first propose a rationale extraction method that leverages the reasoning capabilities of Large Language Models (LLMs) to extract the rationales necessary for answering the query.<n> Subsequently, a rationale-based alignment process is designed to rerank the documents based on the extracted rationales, and fine-tune the reranker to align the preferences.
  arXiv  Detail & Related papers  (2024-12-11T16:32:41Z)
• JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking [81.88787401178378]
  We introduce JudgeRank, a novel agentic reranker that emulates human cognitive processes when assessing document relevance. We evaluate JudgeRank on the reasoning-intensive BRIGHT benchmark, demonstrating substantial performance improvements over first-stage retrieval methods. In addition, JudgeRank performs on par with fine-tuned state-of-the-art rerankers on the popular BEIR benchmark, validating its zero-shot generalization capability.
  arXiv  Detail & Related papers  (2024-10-31T18:43:12Z)
• ChunkRAG: Novel LLM-Chunk Filtering Method for RAG Systems [2.8692611791027893]
  Retrieval-Augmented Generation (RAG) systems generate inaccurate responses due to the retrieval of irrelevant or loosely related information.<n>We propose ChunkRAG, a framework that enhances RAG systems by evaluating and filtering retrieved information at the chunk level.
  arXiv  Detail & Related papers  (2024-10-25T14:07:53Z)
• Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling [9.44858963874474]
  Self-Consistency mitigates hallucinations in Large Language Models (LLMs) by sampling multiple reasoning paths.<n>We introduce Reasoning-Aware Self-Consistency (RASC), a novel framework that enhances sampling efficiency and reasoning faithfulness.
  arXiv  Detail & Related papers  (2024-08-30T05:14:59Z)
• Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting [68.90949377014742]
  Speculative RAG is a framework that leverages a larger generalist LM to efficiently verify multiple RAG drafts produced in parallel by a smaller, distilled specialist LM.<n>Our method accelerates RAG by delegating drafting to the smaller specialist LM, with the larger generalist LM performing a single verification pass over the drafts.<n>It notably enhances accuracy by up to 12.97% while reducing latency by 50.83% compared to conventional RAG systems on PubHealth.
  arXiv  Detail & Related papers  (2024-07-11T06:50:19Z)
• Graph Anomaly Detection with Noisy Labels by Reinforcement Learning [13.135788402192215]
  We propose a novel framework REGAD, i.e., REinforced Graph Anomaly Detector. Specifically, we aim to maximize the performance improvement (AUC) of a base detector by cutting noisy edges approximated through the nodes with high-confidence labels.
  arXiv  Detail & Related papers  (2024-07-08T13:41:21Z)
• Optimization of Retrieval-Augmented Generation Context with Outlier Detection [0.0]
  We focus on methods to reduce the size and improve the quality of the prompt context required for question-answering systems. Our goal is to select the most semantically relevant documents, treating the discarded ones as outliers. It was found that the greatest improvements were achieved with increasing complexity of the questions and answers.
  arXiv  Detail & Related papers  (2024-07-01T15:53:29Z)
• Noisy Pair Corrector for Dense Retrieval [59.312376423104055]
  We propose a novel approach called Noisy Pair Corrector (NPC) NPC consists of a detection module and a correction module. We conduct experiments on text-retrieval benchmarks Natural Question and TriviaQA, code-search benchmarks StaQC and SO-DS.
  arXiv  Detail & Related papers  (2023-11-07T08:27:14Z)
• Improving the Robustness of Summarization Systems with Dual Augmentation [68.53139002203118]
  A robust summarization system should be able to capture the gist of the document, regardless of the specific word choices or noise in the input. We first explore the summarization models' robustness against perturbations including word-level synonym substitution and noise. We propose a SummAttacker, which is an efficient approach to generating adversarial samples based on language models.
  arXiv  Detail & Related papers  (2023-06-01T19:04:17Z)
• Gleo-Det: Deep Convolution Feature-Guided Detector with Local Entropy Optimization for Salient Points [5.955667705173262]
  We propose to achieve fine constraint based on the requirement of repeatability while coarse constraint with guidance of deep convolution features. With the guidance of convolution features, we define the cost function from both positive and negative sides.
  arXiv  Detail & Related papers  (2022-04-27T12:40:21Z)
• Progressive End-to-End Object Detection in Crowded Scenes [96.92416613336096]
  Previous query-based detectors suffer from two drawbacks: first, multiple predictions will be inferred for a single object, typically in crowded scenes; second, the performance saturates as the depth of the decoding stage increases. We propose a progressive predicting method to address the above issues. Specifically, we first select accepted queries to generate true positive predictions, then refine the rest noisy queries according to the previously accepted predictions. Experiments show that our method can significantly boost the performance of query-based detectors in crowded scenes.
  arXiv  Detail & Related papers  (2022-03-15T06:12:00Z)
• Composing Answer from Multi-spans for Reading Comprehension [77.32873012668783]
  We present a novel method to generate answers for non-extraction machine reading comprehension (MRC) tasks. The proposed method has a better performance on accurately generating long answers, and substantially outperforms two competitive typical one-span and Seq2Seq baseline decoders.
  arXiv  Detail & Related papers  (2020-09-14T01:44:42Z)
• Retrospective Reader for Machine Reading Comprehension [90.6069071495214]
  Machine reading comprehension (MRC) is an AI challenge that requires machine to determine the correct answers to questions based on a given passage. When unanswerable questions are involved in the MRC task, an essential verification module called verifier is especially required in addition to the encoder. This paper devotes itself to exploring better verifier design for the MRC task with unanswerable questions.
  arXiv  Detail & Related papers  (2020-01-27T11:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.