Stepwise Reasoning Enhancement for LLMs via External Subgraph Generation

• URL: http://arxiv.org/abs/2606.04454v1
• Date: Wed, 03 Jun 2026 04:58:02 GMT
• Title: Stepwise Reasoning Enhancement for LLMs via External Subgraph Generation
• Abstract summary: Large language models struggle with logical consistency, factual grounding, and interpretability in complex multi-step reasoning.<n>This paper proposes SGR, a framework that integrates large language models with external knowledge graphs.<n> Experiments show that SGR improves reasoning accuracy and Hits@1 performance over standard prompting.
• Score: 10.192419467718475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models have shown strong performance in natural language generation and downstream reasoning tasks, but they still struggle with logical consistency, factual grounding, and interpretability in complex multi-step reasoning. To address these limitations, this paper proposes SGR, a stepwise reasoning enhancement framework that integrates large language models with external knowledge graphs through query-relevant subgraph generation. Given an input question, SGR first extracts key entities, relations, and constraints to construct a structured schema, then retrieves compact subgraphs from a knowledge graph using schema-guided querying. The generated subgraphs provide explicit relational evidence that guides the language model through step-by-step reasoning. In addition, SGR combines direct Cypher-based reasoning with collaborative reasoning integration, allowing candidate answers from multiple reasoning paths to be validated and aggregated according to both model confidence and graph consistency. Experiments on benchmark datasets including CWQ, WebQSP, GrailQA, and KQA Pro demonstrate that SGR improves reasoning accuracy and Hits@1 performance over standard prompting and several knowledge-enhanced baselines. Ablation studies further show that schema guidance and Neo4j-based retrieval are both crucial to the effectiveness of the framework. These results indicate that dynamically generated external subgraphs can improve the accuracy, robustness, and interpretability of LLM-based reasoning.

Related papers

• SGR: A Stepwise Reasoning Framework for LLMs with External Subgraph Generation [10.192419467718475]
  Large Language Models (LLMs) have demonstrated strong capabilities across diverse NLP applications, such as translation, text generation, and question answering.<n>We introduce SGR, a stepwise framework that enhances LLM reasoning through external subgraph generation.<n>SGR builds query-specific subgraphs from external knowledge bases and uses their semantic structure to support multi-step inference.
  arXiv  Detail & Related papers  (2026-05-15T16:02:09Z)
• Multi-Step Semantic Reasoning in Generative Retrieval [50.65947993017158]
  Generative retrieval (GR) models encode a corpus within model parameters and generate relevant document identifiers for a given query.<n>Existing GR models struggle with complex queries in numerical contexts, such as those involving semantic reasoning over financial reports.<n>We propose ReasonGR, a framework designed to enhance multi-step semantic reasoning in numerical contexts within GR.
  arXiv  Detail & Related papers  (2026-03-12T18:38:50Z)
• A Stepwise-Enhanced Reasoning Framework for Large Language Models Based on External Subgraph Generation [12.27505872942231]
  Large Language Models (LLMs) have achieved strong performance across a wide range of natural language processing tasks.<n>We propose a stepwise reasoning enhancement framework for LLMs based on external subgraph generation.<n>By performing reasoning in a step by step manner over structured subgraphs, SGR reduces the influence of noisy information and improves reasoning accuracy.
  arXiv  Detail & Related papers  (2025-12-29T10:35:53Z)
• KBQA-R1: Reinforcing Large Language Models for Knowledge Base Question Answering [64.62317305868264]
  We present textbfKBQA-R1, a framework that shifts the paradigm from text imitation to interaction optimization via Reinforcement Learning.<n>Treating KBQA as a multi-turn decision process, our model learns to navigate the knowledge base using a list of actions.<n>Experiments on WebQSP, GrailQA, and GraphQuestions demonstrate that KBQA-R1 achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-10T17:45:42Z)
• RouteRAG: Efficient Retrieval-Augmented Generation from Text and Graph via Reinforcement Learning [69.87510139069218]
  Retrieval-Augmented Generation (RAG) integrates non-parametric knowledge into Large Language Models (LLMs)<n>Recent progress has advanced text-based RAG to multi-turn reasoning through Reinforcement Learning (RL)<n>We introduce model, an RL-based framework that enables LLMs to perform multi-turn and adaptive graph-text hybrid RAG.
  arXiv  Detail & Related papers  (2025-12-10T10:05:31Z)
• G-reasoner: Foundation Models for Unified Reasoning over Graph-structured Knowledge [88.82814893945077]
  Large language models (LLMs) excel at complex reasoning but remain limited by static and incomplete parametric knowledge.<n>Recent graph-enhanced RAG (GraphRAG) attempts to bridge this gap by constructing tailored graphs and enabling LLMs to reason on them.<n>G-reasoner is a unified framework that integrates graph and language foundation models for reasoning over diverse graph-structured knowledge.
  arXiv  Detail & Related papers  (2025-09-29T04:38:12Z)
• GRIL: Knowledge Graph Retrieval-Integrated Learning with Large Language Models [59.72897499248909]
  We propose a novel graph retriever trained end-to-end with Large Language Models (LLMs)<n>Within the extracted subgraph, structural knowledge and semantic features are encoded via soft tokens and the verbalized graph, respectively, which are infused into the LLM together.<n>Our approach consistently achieves state-of-the-art performance, validating the strength of joint graph-LLM optimization for complex reasoning tasks.
  arXiv  Detail & Related papers  (2025-09-20T02:38:00Z)
• Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering [75.12322966980003]
  Large Language Models (LLMs) have shown strong inductive reasoning ability across various domains.<n>Most existing RAG pipelines rely on unstructured text, limiting interpretability and structured reasoning.<n>Recent studies have explored integrating knowledge graphs with LLMs for knowledge graph question answering.<n>We propose RAPL, a novel framework for efficient and effective graph retrieval in KGQA.
  arXiv  Detail & Related papers  (2025-06-11T12:03:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.