Why Retrieval-Augmented Generation Fails: A Graph Perspective

• URL: http://arxiv.org/abs/2605.14192v1
• Date: Wed, 13 May 2026 23:18:07 GMT
• Title: Why Retrieval-Augmented Generation Fails: A Graph Perspective
• Abstract summary: We present a model-internal study of retrieval-augmented generation that examines how retrieved evidence influences answer generation.<n>We develop a graph-based error detection framework that uses attribution-graph topology features.
• Score: 40.34482080762174
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieval-Augmented Generation (RAG) has become a powerful and widely used approach for improving large language models by grounding generation in retrieved evidence. However, RAG systems still produce incorrect answers in many cases. Why RAG fails despite having access to external information remains poorly understood. We present a model-internal study of retrieval-augmented generation that examines how retrieved evidence influences answer generation. Using circuit tracing, we construct attribution graphs that model the flow of information through transformer layers during decoding. These graphs represent interactions among retrieved context, intermediate model activations, and generated tokens, providing a graph, circuit-level view of how external evidence is integrated into the model's reasoning process across multiple question answering benchmarks, we observe consistent structural differences: correct predictions exhibit deeper reasoning paths, more distributed evidence flow, and a more structured pattern of local connectivity, while failed predictions show shallower, fragmented, and overly concentrated evidence flow. Building on these findings, we develop a graph-based error detection framework that uses attribution-graph topology features. Furthermore, we show that attribution graphs enable targeted interventions. By reinforcing question-constrained evidence grounding, we reshape internal routing so that answer generation remains guided by the question, leading to more effective integration of retrieved information and fewer errors.

Related papers

• GraphLoom: Reliability-Calibrated Graph Evidence Routing for Multimodal KG-RAG [4.035350563640088]
  We present GraphLoom, a reliability-calibrated multimodal knowledge-graph RAG framework for compact and faithful evidence routing.<n>Instead of injecting all retrieved evidence into the generator, GraphLoom performs reliability-aware subgraph retrieval.<n>We show consistent gains in answer quality and evidence faithfulness over strong multimodal RAG, graph-retrieval, and open-source vision-language baselines.
  arXiv  Detail & Related papers  (2026-08-15T06:02:23Z)
• EvoGraph-R1: Self-Evolving Multimodal Knowledge Hypergraphs for Agentic Retrieval [23.778354620399316]
  EvoGraph-R1 is a self-evolving GraphRAG framework that reconceptualizes knowledge graphs as dynamic environments shaped through agent interactions.<n>We formulate retrieval as a Markov Decision Process (MDP) where the agent observes the graph state and executes actions to query (GraphRetrieve), expand (WebSearch), refine (GraphEdit), or terminate (Answer) the reasoning.<n> Experiments on multimodal VQA and text QA benchmarks demonstrate substantial improvements over existing RAG baselines in accuracy, coverage, and traceability.
  arXiv  Detail & Related papers  (2026-07-14T13:34:07Z)
• ArbGraph: Conflict-Aware Evidence Arbitration for Reliable Long-Form Retrieval-Augmented Generation [47.25596369834373]
  ArbGraph is a framework for pre-generation evidence arbitration in long-form RAG.<n>ArbGraph decomposes retrieved documents into atomic claims and organizes them into a conflict-aware evidence graph.<n>We show that ArbGraph consistently improves factual recall and information density while reducing hallucinations and sensitivity to retrieval noise.
  arXiv  Detail & Related papers  (2026-04-20T14:51:43Z)
• HugRAG: Hierarchical Causal Knowledge Graph Design for RAG [13.71290541205767]
  HugRAG is a framework that rethinks knowledge organization for graph-based RAG.<n>Our work establishes a principled foundation for structured, scalable, and causally grounded RAG systems.
  arXiv  Detail & Related papers  (2026-02-04T23:59:02Z)
• Graph-Anchored Knowledge Indexing for Retrieval-Augmented Generation [53.42323544075114]
  We propose GraphAnchor, a novel Graph-Anchored Knowledge Indexing approach.<n> Experiments on four multi-hop question answering benchmarks demonstrate the effectiveness of GraphAnchor.
  arXiv  Detail & Related papers  (2026-01-23T05:41:05Z)
• RouteRAG: Efficient Retrieval-Augmented Generation from Text and Graph via Reinforcement Learning [69.87510139069218]
  Retrieval-Augmented Generation (RAG) integrates non-parametric knowledge into Large Language Models (LLMs)<n>Recent progress has advanced text-based RAG to multi-turn reasoning through Reinforcement Learning (RL)<n>We introduce model, an RL-based framework that enables LLMs to perform multi-turn and adaptive graph-text hybrid RAG.
  arXiv  Detail & Related papers  (2025-12-10T10:05:31Z)
• Divide by Question, Conquer by Agent: SPLIT-RAG with Question-Driven Graph Partitioning [62.640169289390535]
  SPLIT-RAG is a multi-agent RAG framework that addresses the limitations with question-driven semantic graph partitioning and collaborative subgraph retrieval.<n>The innovative framework first create Semantic Partitioning of Linked Information, then use the Type-Specialized knowledge base to achieve Multi-Agent RAG.<n>The attribute-aware graph segmentation manages to divide knowledge graphs into semantically coherent subgraphs, ensuring subgraphs align with different query types.<n>A hierarchical merging module resolves inconsistencies across subgraph-derived answers through logical verifications.
  arXiv  Detail & Related papers  (2025-05-20T06:44:34Z)
• Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey [15.60128530639056]
  Large language models (LLMs) struggle with the factual error during inference due to the lack of sufficient training data and the most updated knowledge.<n>Retrieval-Augmented Generation (RAG) has gained attention as a promising solution to address the limitation of LLMs.<n>This survey offers a novel perspective on the functionality of graphs within RAG and their impact on enhancing performance.
  arXiv  Detail & Related papers  (2025-04-08T03:52:05Z)
• Out-of-Distribution Detection on Graphs: A Survey [58.47395497985277]
  Graph out-of-distribution (GOOD) detection focuses on identifying graph data that deviates from the distribution seen during training.<n>We categorize existing methods into four types: enhancement-based, reconstruction-based, information propagation-based, and classification-based approaches.<n>We discuss practical applications and theoretical foundations, highlighting the unique challenges posed by graph data.
  arXiv  Detail & Related papers  (2025-02-12T04:07:12Z)
• Causal Graphs Meet Thoughts: Enhancing Complex Reasoning in Graph-Augmented LLMs [4.701165676405066]
  It is critical not only to retrieve relevant information but also to provide causal reasoning and explainability.<n>This paper proposes a novel pipeline that filters large knowledge graphs to emphasize cause-effect edges.<n> Experiments on medical question-answering tasks show consistent gains, with up to a 10% absolute improvement.
  arXiv  Detail & Related papers  (2025-01-24T19:31:06Z)
• Counterfactual Intervention Feature Transfer for Visible-Infrared Person Re-identification [69.45543438974963]
  We find graph-based methods in the visible-infrared person re-identification task (VI-ReID) suffer from bad generalization because of two issues. The well-trained input features weaken the learning of graph topology, making it not generalized enough during the inference process. We propose a Counterfactual Intervention Feature Transfer (CIFT) method to tackle these problems.
  arXiv  Detail & Related papers  (2022-08-01T16:15:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.