The Mechanism Matters: When Knowledge Graphs Help Reinforcement Learning

• URL: http://arxiv.org/abs/2607.19616v1
• Date: Tue, 21 Jul 2026 22:43:16 GMT
• Title: The Mechanism Matters: When Knowledge Graphs Help Reinforcement Learning
• Abstract summary: Knowledge graphs (KGs) are widely used to inject prior knowledge into learning reinforcement (RL)<n>We conduct a controlled study that independently varies the RL task, the injection mechanism, and KG quality.<n>Using a synthetic, fully controllable KG over MiniGrid environments, we report three findings.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge graphs (KGs) are widely used to inject prior knowledge into reinforcement learning (RL), yet the literature is dominated by single-domain, positive-result method papers, so we lack a systematic account of when KG structure helps an agent, when it is neutral, and when it hurts. We conduct a controlled study that independently varies the RL task, the injection mechanism (state features, action masking, or potential-based reward shaping), and KG quality. Using a synthetic, fully controllable KG over MiniGrid environments, we report three findings. First, on compositional sparse-reward tasks structured KG guidance improves sample efficiency and solve reliability (70% to 97% of seeds), and a shuffle control that permutes the KG's edges while preserving their count collapses the benefit toward baseline (masking p=0.0001; shaping p=0.006), so the gain is structural rather than generic regularization. Second, KG value scales with the amount of task-relevant knowledge the graph contains. Third, and most consequential, safety depends on the mechanism: soft, optimality-preserving injection benefits from correct knowledge and harmlessly ignores incorrect knowledge, whereas hard masking is brittle, forbidding essential actions when the KG is incomplete or corrupted and making a wrong KG worse than none. A UMLS-derived clinical case study on sepsis management under offline RL is a careful null, underscoring that benefits require task structure the chosen mechanism can exploit. Our results give practitioners concrete guidance on how, and how much, to trust a KG when using it to guide RL.

Related papers

• KG2Code: Bridging Knowledge Graphs and Large Language Models via Executable Code for Question Answering [64.63418702318029]
  KG2Code is a novel approach that transforms knowledge graphs into a code-based representation.<n> KG2Code-QA is introduced as a KGQA framework that formulates KGQA as a code generation task.
  arXiv  Detail & Related papers  (2026-06-26T11:27:55Z)
• Mixture-of-Experts Knowledge Graph Retrieval-Augmented Generation for Multi-Agent LLM-based Recommendation [56.36851606088493]
  We propose a cooperative multi-agent framework for knowledge graph retrieval-augmented generation (KG-RAG) recommendations.<n>MixRAGRec integrates a Mixture-of-Experts Retrieval Agent that routes each query to a KG retrieval expert with different granularities, a Knowledge Preference Alignment Agent that converts structured knowledge into LLM-friendly natural language, and a Contrastive Learning-reinforced Recommendation Agent trained with contrastive preference feedback.
  arXiv  Detail & Related papers  (2026-05-27T08:56:57Z)
• Efficient and Transferable Agentic Knowledge Graph RAG via Reinforcement Learning [18.9814789695716]
  Knowledge-graph retrieval-augmented generation (KG-RAG) couples large language models (LLMs) with structured, verifiable knowledge graphs (KGs) to reduce hallucinations and expose reasoning traces.<n>We introduce KG-R1, an agentic KG retrieval-augmented generation (KG-RAG) framework through reinforcement learning (RL)<n> KG-R1 utilizes a single agent that interacts with KGs as its environment, learning to retrieve at each step and incorporating the retrieved information into its reasoning and generation.
  arXiv  Detail & Related papers  (2025-09-30T15:14:24Z)
• KG-Infused RAG: Augmenting Corpus-Based RAG with External Knowledge Graphs [58.12674907593879]
  KG-Infused RAG is a framework that incorporates pre-existing large-scale knowledge graphs into RAG.<n> KG-Infused RAG directly performs spreading activation over external KGs to retrieve relevant structured knowledge.<n>Experiments show that KG-Infused RAG consistently outperforms vanilla RAG.
  arXiv  Detail & Related papers  (2025-06-11T09:20:02Z)
• Evaluating Knowledge Graph Based Retrieval Augmented Generation Methods under Knowledge Incompleteness [26.18941757096083]
  Knowledge Graph based Retrieval-Augmented Generation (KG-RAG) is a technique that enhances Large Language Model (LLM) inference in tasks like Question Answering (QA)<n>Existing benchmarks do not adequately capture the impact of KG incompleteness on KG-RAG performance.<n>We demonstrate that KG-RAG methods are sensitive to KG incompleteness, highlighting the need for more robust approaches in realistic settings.
  arXiv  Detail & Related papers  (2025-04-07T15:08:03Z)
• All Your Knowledge Belongs to Us: Stealing Knowledge Graphs via Reasoning APIs [7.685940197285116]
  We present KGX, an attack that extracts confidential sub-KGs with high fidelity under limited query budgets.<n>We validate the efficacy of KGX against both experimental and real-world KGR APIs.<n>Our findings suggest the need for a more principled approach to developing and deploying KGR systems.
  arXiv  Detail & Related papers  (2025-03-12T18:18:44Z)
• Anomaly Detection and Classification in Knowledge Graphs [1.2599533416395767]
  We propose SEKA (SEeking Knowledge graph Anomalies), an unsupervised approach for the detection of abnormal triples and entities in Knowledge Graphs.<n>We also present TAXO, a taxonomy of possible anomaly types that can occur in a KG.<n>We evaluate both approaches using the four real-world KGs YAGO-1, KBpedia, Wikidata, and DSKG.
  arXiv  Detail & Related papers  (2024-12-06T05:03:10Z)
• Decoding on Graphs: Faithful and Sound Reasoning on Knowledge Graphs through Generation of Well-Formed Chains [66.55612528039894]
  Knowledge Graphs (KGs) can serve as reliable knowledge sources for question answering (QA) We present DoG (Decoding on Graphs), a novel framework that facilitates a deep synergy between LLMs and KGs. Experiments across various KGQA tasks with different background KGs demonstrate that DoG achieves superior and robust performance.
  arXiv  Detail & Related papers  (2024-10-24T04:01:40Z)
• Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models [92.71304585906624]
  Large language models (LLMs) struggle with faithful reasoning due to knowledge gaps and hallucinations.<n>We introduce graph-constrained reasoning (GCR), a novel framework that bridges structured knowledge in KGs with unstructured reasoning in LLMs.<n>GCR achieves state-of-the-art performance and exhibits strong zero-shot generalizability to unseen KGs without additional training.
  arXiv  Detail & Related papers  (2024-10-16T22:55:17Z)
• Explainable Sparse Knowledge Graph Completion via High-order Graph Reasoning Network [111.67744771462873]
  This paper proposes a novel explainable model for sparse Knowledge Graphs (KGs) It combines high-order reasoning into a graph convolutional network, namely HoGRN. It can not only improve the generalization ability to mitigate the information insufficiency issue but also provide interpretability.
  arXiv  Detail & Related papers  (2022-07-14T10:16:56Z)
• IterefinE: Iterative KG Refinement Embeddings using Symbolic Knowledge [10.689559910656474]
  Knowledge Graphs (KGs) extracted from text sources are often noisy and lead to poor performance in downstream application tasks such as KG-based question answering. Most successful techniques for KG refinement make use of inference rules and reasoning oversupervised. In this paper, we present a KG refinement framework called IterefinE which iteratively combines the two techniques.
  arXiv  Detail & Related papers  (2020-06-03T14:05:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.