Knowledge Graph Large Language Model (KG-LLM) for Link Prediction

• URL: http://arxiv.org/abs/2403.07311v8
• Date: Fri, 9 Aug 2024 15:39:05 GMT
• Title: Knowledge Graph Large Language Model (KG-LLM) for Link Prediction
• Authors: Dong Shu, Tianle Chen, Mingyu Jin, Chong Zhang, Mengnan Du, Yongfeng Zhang,
• Abstract summary: We introduce the Knowledge Graph Large Language Model (KG-LLM), a novel framework that leverages large language models (LLMs) for knowledge graph tasks. We first convert structured knowledge graph data into natural language and then use these natural language prompts to fine-tune LLMs. To show the efficacy of the KG-LLM Framework, we fine-tune three leading LLMs within this framework, including Flan-T5, LLaMa2 and Gemma.
• Score: 43.55117421485917
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The task of multi-hop link prediction within knowledge graphs (KGs) stands as a challenge in the field of knowledge graph analysis, as it requires the model to reason through and understand all intermediate connections before making a prediction. In this paper, we introduce the Knowledge Graph Large Language Model (KG-LLM), a novel framework that leverages large language models (LLMs) for knowledge graph tasks. We first convert structured knowledge graph data into natural language and then use these natural language prompts to fine-tune LLMs to enhance multi-hop link prediction in KGs. By converting the KG to natural language prompts, our framework is designed to learn the latent representations of entities and their interrelations. To show the efficacy of the KG-LLM Framework, we fine-tune three leading LLMs within this framework, including Flan-T5, LLaMa2 and Gemma. Further, we explore the framework's potential to provide LLMs with zero-shot capabilities for handling previously unseen prompts. Experimental results show that KG-LLM significantly improves the models' generalization capabilities, leading to more accurate predictions in unfamiliar scenarios.

Related papers

• GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion [52.026016846945424]
  We propose a new method called GLTW, which encodes the structural information of KGs and merges it with Large Language Models. Specifically, we introduce an improved Graph Transformer (iGT) that effectively encodes subgraphs with both local and global structural information. Also, we develop a subgraph-based multi-classification training objective, using all entities within KG as classification objects, to boost learning efficiency.
  arXiv  Detail & Related papers  (2025-02-17T06:02:59Z)
• Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Models [17.88134311726175]
  We propose a framework to learn and apply quantized codes for each entity, aiming for the seamless integration of Knowledge Graphs with Large Language Models. Experiment results demonstrate that SSQR outperforms existing unsupervised quantized methods, producing more distinguishable codes. The fine-tuned LLaMA2 and LLaMA3.1 also have superior performance on KG link prediction and triple classification tasks.
  arXiv  Detail & Related papers  (2025-01-30T03:40:20Z)
• Narrative Analysis of True Crime Podcasts With Knowledge Graph-Augmented Large Language Models [8.78598447041169]
  Large language models (LLMs) still struggle with complex narrative arcs as well as narratives containing conflicting information. Recent work indicates LLMs augmented with external knowledge bases can improve the accuracy and interpretability of the resulting models. In this work, we analyze the effectiveness of applying knowledge graphs (KGs) in understanding true-crime podcast data.
  arXiv  Detail & Related papers  (2024-11-01T21:49:00Z)
• Tree-of-Traversals: A Zero-Shot Reasoning Algorithm for Augmenting Black-box Language Models with Knowledge Graphs [72.89652710634051]
  Knowledge graphs (KGs) complement Large Language Models (LLMs) by providing reliable, structured, domain-specific, and up-to-date external knowledge. We introduce Tree-of-Traversals, a novel zero-shot reasoning algorithm that enables augmentation of black-box LLMs with one or more KGs.
  arXiv  Detail & Related papers  (2024-07-31T06:01:24Z)
• Unifying Large Language Models and Knowledge Graphs: A Roadmap [61.824618473293725]
  Large language models (LLMs) are making new waves in the field of natural language processing and artificial intelligence. Knowledge Graphs (KGs), Wikipedia and Huapu for example, are structured knowledge models that explicitly store rich factual knowledge.
  arXiv  Detail & Related papers  (2023-06-14T07:15:26Z)
• Deep Bidirectional Language-Knowledge Graph Pretraining [159.9645181522436]
  DRAGON is a self-supervised approach to pretraining a deeply joint language-knowledge foundation model from text and KG at scale. Our model takes pairs of text segments and relevant KG subgraphs as input and bidirectionally fuses information from both modalities.
  arXiv  Detail & Related papers  (2022-10-17T18:02:52Z)
• KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs [26.557447199727758]
  We propose a novel knowledge-aware language model framework based on fine-tuning process. Our model can efficiently incorporate world knowledge from KGs into existing language models such as BERT.
  arXiv  Detail & Related papers  (2021-09-09T12:39:17Z)
• Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models [42.38563175680914]
  This paper studies how to automatically generate a natural language text that describes the facts in knowledge graph (KG) Considering the few-shot setting, we leverage the excellent capacities of pretrained language models (PLMs) in language understanding and generation.
  arXiv  Detail & Related papers  (2021-06-03T06:48:00Z)
• JAKET: Joint Pre-training of Knowledge Graph and Language Understanding [73.43768772121985]
  We propose a novel joint pre-training framework, JAKET, to model both the knowledge graph and language. The knowledge module and language module provide essential information to mutually assist each other. Our design enables the pre-trained model to easily adapt to unseen knowledge graphs in new domains.
  arXiv  Detail & Related papers  (2020-10-02T05:53:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.