The Reasoning Bottleneck in Graph-RAG: Structured Prompting and Context Compression for Multi-Hop QA

• URL: http://arxiv.org/abs/2603.14045v2
• Date: Wed, 18 Mar 2026 02:14:55 GMT
• Title: The Reasoning Bottleneck in Graph-RAG: Structured Prompting and Context Compression for Multi-Hop QA
• Abstract summary: Graph-RAG systems achieve strong multi-hop question answering by indexing documents into knowledge graphs, but strong retrieval does not guarantee strong answers.<n>We show that 77% to 91% of questions have the gold answer in the retrieved context, yet accuracy is only 35% to 78%, and 73% to 84% of errors are reasoning failures.<n>We propose two augmentations: (i) SPARQL chain-of-thought prompting, which decomposes questions into triple-pattern queries aligned with the entity-relationship context, and (ii) graph-walk compression.
• Score: 4.190399245544758
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph-RAG systems achieve strong multi-hop question answering by indexing documents into knowledge graphs, but strong retrieval does not guarantee strong answers. Evaluating KET-RAG, a leading Graph-RAG system, on three multi-hop QA benchmarks (HotpotQA, MuSiQue, 2WikiMultiHopQA), we find that 77% to 91% of questions have the gold answer in the retrieved context, yet accuracy is only 35% to 78%, and 73% to 84% of errors are reasoning failures. We propose two augmentations: (i) SPARQL chain-of-thought prompting, which decomposes questions into triple-pattern queries aligned with the entity-relationship context, and (ii) graph-walk compression, which compresses the context by ~60% via knowledge-graph traversal with no LLM calls. SPARQL CoT improves accuracy by +2 to +14 pp; graph-walk compression adds +6 pp on average when paired with structured prompting on smaller models. Surprisingly, we show that, with question-type routing, a fully augmented budget open-weight Llama-8B model matches or exceeds the unaugmented Llama-70B baseline on all three benchmarks at ~12x lower cost. A replication on LightRAG confirms that our augmentations transfer across Graph-RAG systems.

Related papers

• Query-Aware Spreading Activation for Multi-Hop Retrieval over Knowledge Graphs [0.0]
  In most existing systems, the question only sets the seed nodes.<n>This architecture requires loading the full graph into Python memory and an iterative solver with a variable number of iterations.<n>We propose a spreading-activation method that achieves the same query-aware.<n>retrieval with a single per-step semantic gate.
  arXiv  Detail & Related papers  (2026-06-29T11:10:19Z)
• ContextRAG: Extraction-Free Hierarchical Graph Construction for Retrieval-Augmented Generation [0.9936176521793759]
  Graph-structured retrieval-augmented generation (RAG) systems can improve answer quality on multi-hop questions.<n>Current systems rely on large language models (LLMs) to extract entities, relations, and summaries during indexing.<n>We present ContextRAG, a graph RAG system whose graph topology is constructed without LLM-based entity or relation extraction.
  arXiv  Detail & Related papers  (2026-05-19T12:08:19Z)
• GRASP: Graph Agentic Search over Propositions for Multi-hop Question Answering [9.683512181435901]
  Graph Agentic Search over Propositions (GRASP) is an agentic system that simultaneously optimize for high accuracy and minimal token usage in multi-hop question answering.<n>GRASP actively coordinates its retrieval strategy by decomposing multi-hop queries into dependency-aware plans.<n>We evaluate GRASP on MuSiQue, 2WikiMultihopQA, and HotpotQA under two settings: open-corpus retrieval and extended context reasoning (LongBench)
  arXiv  Detail & Related papers  (2026-05-15T19:59:35Z)
• PAR$^2$-RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering [57.89576196160413]
  Large language models (LLMs) remain brittle on multi-hop question answering (MHQA)<n>We propose textbfPlanned Active Retrieval and Reasoning RAG (PAR$2$-RAG), a two-stage framework that separates emphcoverage from emphcommitment.
  arXiv  Detail & Related papers  (2026-03-30T23:52:54Z)
• TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework [62.66056331998838]
  TeaRAG is a token-efficient agentic RAG framework capable of compressing both retrieval content and reasoning steps.<n>Our reward function evaluates the knowledge sufficiency by a knowledge matching mechanism, while penalizing excessive reasoning steps.
  arXiv  Detail & Related papers  (2025-11-07T16:08:34Z)
• Interpretable Question Answering with Knowledge Graphs [0.19695349076827803]
  This paper presents a question answering system that operates exclusively on a knowledge graph retrieval.<n>A small paraphraser model is used to paraphrase the entity relationship edges retrieved from querying the knowledge graph.<n>This work includes an evaluation using LLM-as-a-judge on the CRAG benchmark, which resulted in accuracies of 71.9% and 54.4%.
  arXiv  Detail & Related papers  (2025-10-22T02:36:35Z)
• SUBQRAG: Sub-Question Driven Dynamic Graph RAG [34.20328335590984]
  SubQRAG is a sub-question-driven framework that enhances reasoning depth.<n>SubQRAG decomposes a complex question into an ordered chain of verifiable sub-questions.<n> Experiments on three multi-hop QA benchmarks demonstrate that SubQRAG achieves consistent and significant improvements.
  arXiv  Detail & Related papers  (2025-10-09T02:55:58Z)
• StepChain GraphRAG: Reasoning Over Knowledge Graphs for Multi-Hop Question Answering [32.34277555394503]
  We introduce StepChain GraphRAG, a framework that unites question decomposition with a Breadth-First Search (BFS) Reasoning Flow.<n>Our approach first builds a global index over the corpus; at inference time, only retrieved passages are parsed on-the-fly into a knowledge graph.<n> Experiments on MuSiQue, 2WikiMultiHopQA, and HotpotQA show that StepChain GraphRAG achieves state-of-the-art Exact Match and F1 scores.
  arXiv  Detail & Related papers  (2025-10-03T09:06:37Z)
• Query-Centric Graph Retrieval Augmented Generation [15.423162448800134]
  QCG-RAG is a query-centric graph RAG framework that enables query-granular indexing and multi-hop chunk retrieval.<n> Experiments on LiHuaWorld and MultiHop-RAG show that QCG-RAG consistently outperforms prior chunk-based and graph-based RAG methods in question answering accuracy.
  arXiv  Detail & Related papers  (2025-09-25T14:35:44Z)
• Enrich-on-Graph: Query-Graph Alignment for Complex Reasoning with LLM Enriching [61.824094419641575]
  Large Language Models (LLMs) struggle with hallucinations and factual errors in knowledge-intensive scenarios like knowledge graph question answering (KGQA)<n>We attribute this to the semantic gap between structured knowledge graphs (KGs) and unstructured queries, caused by inherent differences in their focuses and structures.<n>Existing methods usually employ resource-intensive, non-scalable reasoning on vanilla KGs, but overlook this gap.<n>We propose a flexible framework, Enrich-on-Graph (EoG), which leverages LLMs' prior knowledge to enrich KGs, bridge the semantic gap between graphs and queries.
  arXiv  Detail & Related papers  (2025-09-25T06:48:52Z)
• Divide by Question, Conquer by Agent: SPLIT-RAG with Question-Driven Graph Partitioning [62.640169289390535]
  SPLIT-RAG is a multi-agent RAG framework that addresses the limitations with question-driven semantic graph partitioning and collaborative subgraph retrieval.<n>The innovative framework first create Semantic Partitioning of Linked Information, then use the Type-Specialized knowledge base to achieve Multi-Agent RAG.<n>The attribute-aware graph segmentation manages to divide knowledge graphs into semantically coherent subgraphs, ensuring subgraphs align with different query types.<n>A hierarchical merging module resolves inconsistencies across subgraph-derived answers through logical verifications.
  arXiv  Detail & Related papers  (2025-05-20T06:44:34Z)
• UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph [89.98762327725112]
  Multi-hop Question Answering over Knowledge Graph(KGQA) aims to find the answer entities that are multiple hops away from the topic entities mentioned in a natural language question. We propose UniKGQA, a novel approach for multi-hop KGQA task, by unifying retrieval and reasoning in both model architecture and parameter learning.
  arXiv  Detail & Related papers  (2022-12-02T04:08:09Z)
• QA4QG: Using Question Answering to Constrain Multi-Hop Question Generation [54.136509061542775]
  Multi-hop question generation (MQG) aims to generate complex questions which require reasoning over multiple pieces of information of the input passage. We propose a novel framework, QA4QG, a QA-augmented BART-based framework for MQG. Our results on the HotpotQA dataset show that QA4QG outperforms all state-of-the-art models.
  arXiv  Detail & Related papers  (2022-02-14T08:16:47Z)
• Multi-hop Question Generation with Graph Convolutional Network [58.31752179830959]
  Multi-hop Question Generation (QG) aims to generate answer-related questions by aggregating and reasoning over multiple scattered evidence from different paragraphs. We propose Multi-Hop volution Fusion Network for Question Generation (MulQG), which does context encoding in multiple hops. Our proposed model is able to generate fluent questions with high completeness and outperforms the strongest baseline by 20.8% in the multi-hop evaluation.
  arXiv  Detail & Related papers  (2020-10-19T06:15:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.