Are Large Language Models Effective Knowledge Graph Constructors?

• URL: http://arxiv.org/abs/2510.11297v1
• Date: Mon, 13 Oct 2025 11:37:48 GMT
• Title: Are Large Language Models Effective Knowledge Graph Constructors?
• Authors: Ruirui Chen, Weifeng Jiang, Chengwei Qin, Bo Xiong, Fiona Liausvia, Dongkyu Choi, Boon Kiat Quek,
• Abstract summary: Knowledge graphs (KGs) are vital for knowledge-intensive tasks and have shown promise in reducing hallucinations in large language models (LLMs)<n>We propose a hierarchical extraction framework that organizes information at multiple levels, enabling the creation of semantically rich and well-structured KGs.<n>Using state-of-the-art LLMs, we extract and construct knowledge graphs and evaluate them comprehensively from both structural and semantic perspectives.
• Score: 26.60279256406507
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge graphs (KGs) are vital for knowledge-intensive tasks and have shown promise in reducing hallucinations in large language models (LLMs). However, constructing high-quality KGs remains difficult, requiring accurate information extraction and structured representations that support interpretability and downstream utility. Existing LLM-based approaches often focus narrowly on entity and relation extraction, limiting coverage to sentence-level contexts or relying on predefined schemas. We propose a hierarchical extraction framework that organizes information at multiple levels, enabling the creation of semantically rich and well-structured KGs. Using state-of-the-art LLMs, we extract and construct knowledge graphs and evaluate them comprehensively from both structural and semantic perspectives. Our results highlight the strengths and shortcomings of current LLMs in KG construction and identify key challenges for future work. To advance research in this area, we also release a curated dataset of LLM-generated KGs derived from research papers on children's mental well-being. This resource aims to foster more transparent, reliable, and impactful applications in high-stakes domains such as healthcare.

Related papers

• A Human-in-the-Loop, LLM-Centered Architecture for Knowledge-Graph Question Answering [41.99844472131922]
  Large Language Models excel at language understanding but are limited in knowledge-intensive domains.<n>This work introduces an interactive framework in which LLMs generate and explain Cypher graph queries.
  arXiv  Detail & Related papers  (2026-02-05T10:10:19Z)
• Beyond Textual Context: Structural Graph Encoding with Adaptive Space Alignment to alleviate the hallucination of LLMs [15.260879306368674]
  The SSKG-LLM is an innovative model architecture that efficiently integrate both the Structural information of KGs into the reasoning processes of Large Language Models.<n>We conduct extensive experiments and provide a detailed analysis to explore how incorporating the structural information of KGs can enhance the factual reasoning abilities of LLMs.
  arXiv  Detail & Related papers  (2025-09-26T12:14:01Z)
• GRIL: Knowledge Graph Retrieval-Integrated Learning with Large Language Models [59.72897499248909]
  We propose a novel graph retriever trained end-to-end with Large Language Models (LLMs)<n>Within the extracted subgraph, structural knowledge and semantic features are encoded via soft tokens and the verbalized graph, respectively, which are infused into the LLM together.<n>Our approach consistently achieves state-of-the-art performance, validating the strength of joint graph-LLM optimization for complex reasoning tasks.
  arXiv  Detail & Related papers  (2025-09-20T02:38:00Z)
• GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion [52.026016846945424]
  We propose a new method called GLTW, which encodes the structural information of KGs and merges it with Large Language Models.<n>Specifically, we introduce an improved Graph Transformer (iGT) that effectively encodes subgraphs with both local and global structural information.<n>Also, we develop a subgraph-based multi-classification training objective, using all entities within KG as classification objects, to boost learning efficiency.
  arXiv  Detail & Related papers  (2025-02-17T06:02:59Z)
• Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data [0.9284740716447338]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation.<n>Recent research has shown promising results in leveraging knowledge graphs (KGs) to enhance LLM performance.<n>We have developed different techniques that tightly integrate KG structures and semantics into LLM representations.
  arXiv  Detail & Related papers  (2024-12-14T02:51:47Z)
• KaLM: Knowledge-aligned Autoregressive Language Modeling via Dual-view Knowledge Graph Contrastive Learning [74.21524111840652]
  This paper proposes textbfKaLM, a textitKnowledge-aligned Language Modeling approach.<n>It fine-tunes autoregressive large language models to align with KG knowledge via the joint objective of explicit knowledge alignment and implicit knowledge alignment.<n> Notably, our method achieves a significant performance boost in evaluations of knowledge-driven tasks.
  arXiv  Detail & Related papers  (2024-12-06T11:08:24Z)
• Ontology Population using LLMs [0.9894420655516563]
  Knowledge graphs (KGs) are increasingly utilized for data integration, representation, and visualization. LLMs offer promising capabilities for such tasks, excelling in natural language understanding and content generation. This study investigates LLM effectiveness for the KG population, focusing on the Enslaved.org Hub Ontology.
  arXiv  Detail & Related papers  (2024-11-03T15:39:20Z)
• GIVE: Structured Reasoning of Large Language Models with Knowledge Graph Inspired Veracity Extrapolation [108.2008975785364]
  Graph Inspired Veracity Extrapolation (GIVE) is a novel reasoning method that merges parametric and non-parametric memories to improve accurate reasoning with minimal external input.<n>GIVE guides the LLM agent to select the most pertinent expert data (observe), engage in query-specific divergent thinking (reflect), and then synthesize this information to produce the final output (speak)
  arXiv  Detail & Related papers  (2024-10-11T03:05:06Z)
• Dual Reasoning: A GNN-LLM Collaborative Framework for Knowledge Graph Question Answering [38.31983923708175]
  We propose Dual-Reasoning, a novel framework that integrates an external system based on Graph Neural Network (GNN) for explicit reasoning on Knowledge Graphs (KGs)<n>We show that DualR achieves state-of-the-art performance while maintaining high efficiency and interpretability.
  arXiv  Detail & Related papers  (2024-06-03T09:38:28Z)
• KG-RAG: Bridging the Gap Between Knowledge and Creativity [0.0]
  Large Language Model Agents (LMAs) face issues such as information hallucinations, catastrophic forgetting, and limitations in processing long contexts. This paper introduces a KG-RAG (Knowledge Graph-Retrieval Augmented Generation) pipeline to enhance the knowledge capabilities of LMAs. Preliminary experiments on the ComplexWebQuestions dataset demonstrate notable improvements in the reduction of hallucinated content.
  arXiv  Detail & Related papers  (2024-05-20T14:03:05Z)
• Large Language Models Can Better Understand Knowledge Graphs Than We Thought [13.336418752729987]
  We study how large language models (LLMs) process and interpret knowledge graphs (KGs)<n>At the literal level, we reveal LLMs' preferences for various input formats.<n>At the attention distribution level, we discuss the underlying mechanisms driving these preferences.
  arXiv  Detail & Related papers  (2024-02-18T10:44:03Z)
• Towards Verifiable Generation: A Benchmark for Knowledge-aware Language Model Attribution [48.86322922826514]
  This paper defines a new task of Knowledge-aware Language Model Attribution (KaLMA) First, we extend attribution source from unstructured texts to Knowledge Graph (KG), whose rich structures benefit both the attribution performance and working scenarios. Second, we propose a new Conscious Incompetence" setting considering the incomplete knowledge repository. Third, we propose a comprehensive automatic evaluation metric encompassing text quality, citation quality, and text citation alignment.
  arXiv  Detail & Related papers  (2023-10-09T11:45:59Z)
• Unifying Large Language Models and Knowledge Graphs: A Roadmap [61.824618473293725]
  Large language models (LLMs) are making new waves in the field of natural language processing and artificial intelligence. Knowledge Graphs (KGs), Wikipedia and Huapu for example, are structured knowledge models that explicitly store rich factual knowledge.
  arXiv  Detail & Related papers  (2023-06-14T07:15:26Z)
• LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities [66.36633042421387]
  Large Language Models (LLMs) for Knowledge Graph (KG) construction and reasoning evaluated.<n>We propose AutoKG, a multi-agent-based approach employing LLMs and external sources for KG construction and reasoning.
  arXiv  Detail & Related papers  (2023-05-22T15:56:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.