Retrieval from Within: An Intrinsic Capability of Attention-Based Models

• URL: http://arxiv.org/abs/2605.05806v2
• Date: Fri, 08 May 2026 05:21:54 GMT
• Title: Retrieval from Within: An Intrinsic Capability of Attention-Based Models
• Abstract summary: We introduce INTRA (INTrinsic Retrieval via Attention), a framework where decoder attention queries score pre-encoded evidence chunks that are then directly reused as context for generation.<n>On question-answering benchmarks, INTRA outperforms strong engineered retrieval pipelines on both evidence recall and end-to-end answer quality.
• Score: 45.52264380766645
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-augmented generation (RAG) typically treats retrieval and generation as separate systems. We ask whether an attention-based encoder-decoder can instead retrieve directly from its own internal representations. We introduce INTRA (INTrinsic Retrieval via Attention), a framework where decoder attention queries score pre-encoded evidence chunks that are then directly reused as context for generation. By construction, INTRA unifies retrieval and generation, eliminating the retriever-generator mismatch typical of RAG pipelines. This design also amortizes context encoding by reusing precomputed encoder states across queries. On question-answering benchmarks, INTRA outperforms strong engineered retrieval pipelines on both evidence recall and end-to-end answer quality. Our results demonstrate that attention-based models already possess a retrieval mechanism that can be elicited, rather than added as an external module.

Related papers

• Generating Leakage-Free Benchmarks for Robust RAG Evaluation [21.380693848036547]
  Retrieval-augmented generation (RAG) is widely used to augment large language models (LLMs) with external knowledge.<n>Many benchmark datasets, designed to test RAG performance, comprise many questions that can already be answered from an LLM's parametric memory.<n>We introduce SeedRG, a semi-synthetic benchmark generation pipeline that mitigates knowledge leakage and addresses the issue of benchmark aging.
  arXiv  Detail & Related papers  (2026-05-09T09:48:50Z)
• R$^3$AG: Retriever Routing for Retrieval-Augmented Generation [59.47703698994575]
  R$3$AG decomposes retriever capability into two learnable dimensions: retrieval quality and generation utility.<n>R$3$AG consistently outperforms both the best individual retrievers and state-of-the-art static routing methods.
  arXiv  Detail & Related papers  (2026-04-22T06:51:20Z)
• Latent Abstraction for Retrieval-Augmented Generation [4.6096940605642915]
  Existing systems rely on generating natural language queries at each hop.<n>We propose textbfLAnR (Latent Abstraction for RAG), a unified framework in which a single LLM jointly performs encoding, retrieval, and generation entirely within its own latent space.
  arXiv  Detail & Related papers  (2026-04-20T06:26:13Z)
• Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage [89.58253972744531]
  Retrieval-augmented generation (RAG) systems combine document retrieval with a generative model to address complex information seeking tasks like report generation.<n>We investigate whether upstream retrieval metrics can serve as reliable early indicators of the final generated response's information coverage.<n>Our findings demonstrate strong correlations between coverage-based retrieval metrics and nugget coverage in generated responses at both topic and system levels.
  arXiv  Detail & Related papers  (2026-03-09T18:20:20Z)
• Test-time Corpus Feedback: From Retrieval to RAG [21.517949407443453]
  Retrieval-Augmented Generation (RAG) has emerged as a standard framework for knowledge-intensive NLP tasks.<n>Most RAG pipelines treat retrieval and reasoning as isolated components, retrieving documents once and then generating answers without further interaction.<n>Recent work in both the information retrieval (IR) and NLP communities has begun to close this gap by introducing adaptive retrieval and ranking methods that incorporate feedback.
  arXiv  Detail & Related papers  (2025-08-21T10:57:38Z)
• Parallel Key-Value Cache Fusion for Position Invariant RAG [55.9809686190244]
  Large Language Models (LLMs) are sensitive to the position of relevant information within contexts.<n>We introduce a framework that generates consistent outputs for decoder-only models, irrespective of the input context order.
  arXiv  Detail & Related papers  (2025-01-13T17:50:30Z)
• RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation [21.764973680014368]
  RetroLLM is a unified framework that integrates retrieval and generation into a single, cohesive process.<n>To mitigate false pruning in the process of constrained evidence generation, we introduce hierarchical FM-Index constraints.<n>Experiments on five open-domain QA datasets demonstrate RetroLLM's superior performance across both in-domain and out-of-domain tasks.
  arXiv  Detail & Related papers  (2024-12-16T16:03:25Z)
• Corrective Retrieval Augmented Generation [36.04062963574603]
  Retrieval-augmented generation (RAG) relies heavily on relevance of retrieved documents, raising concerns about how the model behaves if retrieval goes wrong. We propose the Corrective Retrieval Augmented Generation (CRAG) to improve the robustness of generation. CRAG is plug-and-play and can be seamlessly coupled with various RAG-based approaches.
  arXiv  Detail & Related papers  (2024-01-29T04:36:39Z)
• ReFIT: Relevance Feedback from a Reranker during Inference [109.33278799999582]
  Retrieve-and-rerank is a prevalent framework in neural information retrieval. We propose to leverage the reranker to improve recall by making it provide relevance feedback to the retriever at inference time.
  arXiv  Detail & Related papers  (2023-05-19T15:30:33Z)
• Generation-Augmented Retrieval for Open-domain Question Answering [134.27768711201202]
  Generation-Augmented Retrieval (GAR) for answering open-domain questions. We show that generating diverse contexts for a query is beneficial as fusing their results consistently yields better retrieval accuracy. GAR achieves state-of-the-art performance on Natural Questions and TriviaQA datasets under the extractive QA setup when equipped with an extractive reader.
  arXiv  Detail & Related papers  (2020-09-17T23:08:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.