Structure-Internalized Rule Language Model for Faithful Knowledge Graph Reasoning

• URL: http://arxiv.org/abs/2608.17443v1
• Date: Tue, 18 Aug 2026 07:21:20 GMT
• Title: Structure-Internalized Rule Language Model for Faithful Knowledge Graph Reasoning
• Abstract summary: Knowledge Graph Reasoning (KGR) aims to discover latent facts by leveraging the structural evidence available in Knowledge Graphs ( KGs)<n>Recent studies have demonstrated that Large Language Models (LLMs) can achieve remarkable progress on KGR tasks via flexible in-context learning.<n>We propose a Structure-Internalized Rule Language Model (SIRLM) to address the representation inconsistency between KG structural context and LLM parametric knowledge.
• Score: 6.421907131279842
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge Graph Reasoning (KGR) aims to discover latent facts by leveraging the structural evidence available in KGs, posing a challenge to the structural semantic understanding capability of KGR models. Recent studies have demonstrated that Large Language Models (LLMs) can achieve remarkable progress on KGR tasks via flexible in-context learning. However, the inherent representation inconsistency between KG structural context and LLM parametric knowledge remains inadequately addressed. This limitation prevents LLMs from effectively perceiving reasoning evidence that aligns with KG constraints, which undermines both the effectiveness and faithfulness of reasoning. We refer to this problem as reasoning evidence perception drift of LLMs over KGs. To address this problem, we propose a Structure-Internalized Rule Language Model (SIRLM), which centers on structural rule generation to couple the parametric learning of structural knowledge with the faithfulness evaluation of reasoning logic, enabling LLMs to anchor tightly to KG-grounded evidence. Specifically, we first design a Structure-Internalized Rule Generator (SIRG), which incorporates an in-context learning block augmented with a structural relation memory to coordinate structural and parametric knowledge. Furthermore, we equip SIRG with a KG tokenizer based on structural invariance learning and a neuro-symbolic reasoner based on rule-constrained message propagation. These components provide SIRG with learnable structural representations and faithful rule-execution feedback, respectively. Our SIRLM can be seamlessly integrated into standard LLM training paradigms, such as SFT and GRPO. Extensive experiments against 17 state-of-the-art KGR methods on 36 datasets demonstrate the significant superiority of SIRLM.

Related papers

• Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoning [47.967495648005986]
  We propose a Knowledge Reasoning Language Model (KRLM) that achieves unified coordination between LLM knowledge and KG context.<n>Extensive experimental results on 25 real-world inductive KGR datasets demonstrate the significant superiority of the proposed KRLM.
  arXiv  Detail & Related papers  (2025-10-15T02:11:58Z)
• Beyond Textual Context: Structural Graph Encoding with Adaptive Space Alignment to alleviate the hallucination of LLMs [15.260879306368674]
  The SSKG-LLM is an innovative model architecture that efficiently integrate both the Structural information of KGs into the reasoning processes of Large Language Models.<n>We conduct extensive experiments and provide a detailed analysis to explore how incorporating the structural information of KGs can enhance the factual reasoning abilities of LLMs.
  arXiv  Detail & Related papers  (2025-09-26T12:14:01Z)
• Enhancing Large Language Models with Reliable Knowledge Graphs [0.6345523830122166]
  Knowledge Graphs offer a promising solution to ground Large Language Models in verified knowledge.<n>Their potential remains constrained by inherent noise, incompleteness, and the complexity of integrating their rigid structure with the flexible reasoning of LLMs.<n>This thesis addresses these limitations through a cohesive framework that enhances LLMs by refining and leveraging reliable KGs.
  arXiv  Detail & Related papers  (2025-06-16T07:43:18Z)
• LightPROF: A Lightweight Reasoning Framework for Large Language Model on Knowledge Graph [57.382255728234064]
  Large Language Models (LLMs) have impressive capabilities in text understanding and zero-shot reasoning.<n> Knowledge Graphs (KGs) provide rich and reliable contextual information for the reasoning process of LLMs.<n>We propose a novel Lightweight and efficient Prompt learning-ReasOning Framework for KGQA (LightPROF)
  arXiv  Detail & Related papers  (2025-04-04T03:03:47Z)
• Decoding on Graphs: Faithful and Sound Reasoning on Knowledge Graphs through Generation of Well-Formed Chains [66.55612528039894]
  Knowledge Graphs (KGs) can serve as reliable knowledge sources for question answering (QA) We present DoG (Decoding on Graphs), a novel framework that facilitates a deep synergy between LLMs and KGs. Experiments across various KGQA tasks with different background KGs demonstrate that DoG achieves superior and robust performance.
  arXiv  Detail & Related papers  (2024-10-24T04:01:40Z)
• Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models [92.71304585906624]
  Large language models (LLMs) struggle with faithful reasoning due to knowledge gaps and hallucinations.<n>We introduce graph-constrained reasoning (GCR), a novel framework that bridges structured knowledge in KGs with unstructured reasoning in LLMs.<n>GCR achieves state-of-the-art performance and exhibits strong zero-shot generalizability to unseen KGs without additional training.
  arXiv  Detail & Related papers  (2024-10-16T22:55:17Z)
• Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofitting [51.7049140329611]
  This paper proposes Knowledge Graph-based Retrofitting (KGR) to mitigate factual hallucination during the reasoning process. Experiments show that KGR can significantly improve the performance of LLMs on factual QA benchmarks.
  arXiv  Detail & Related papers  (2023-11-22T11:08:38Z)
• Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning [104.92384929827776]
  Large language models (LLMs) have demonstrated impressive reasoning abilities in complex tasks. They lack up-to-date knowledge and experience hallucinations during reasoning. Knowledge graphs (KGs) offer a reliable source of knowledge for reasoning.
  arXiv  Detail & Related papers  (2023-10-02T10:14:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.