Enhancing Text-based Knowledge Graph Completion with Zero-Shot Large Language Models: A Focus on Semantic Enhancement

• URL: http://arxiv.org/abs/2310.08279v3
• Date: Thu, 27 Jun 2024 04:55:28 GMT
• Title: Enhancing Text-based Knowledge Graph Completion with Zero-Shot Large Language Models: A Focus on Semantic Enhancement
• Authors: Rui Yang, Jiahao Zhu, Jianping Man, Li Fang, Yi Zhou,
• Abstract summary: We introduce a framework termed constrained prompts for KGC (CP-KGC) This framework designs prompts that adapt to different datasets to enhance semantic richness. This study extends the performance limits of existing models and promotes further integration of KGC with large language models.
• Score: 8.472388165833292
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The design and development of text-based knowledge graph completion (KGC) methods leveraging textual entity descriptions are at the forefront of research. These methods involve advanced optimization techniques such as soft prompts and contrastive learning to enhance KGC models. The effectiveness of text-based methods largely hinges on the quality and richness of the training data. Large language models (LLMs) can utilize straightforward prompts to alter text data, thereby enabling data augmentation for KGC. Nevertheless, LLMs typically demand substantial computational resources. To address these issues, we introduce a framework termed constrained prompts for KGC (CP-KGC). This CP-KGC framework designs prompts that adapt to different datasets to enhance semantic richness. Additionally, CP-KGC employs a context constraint strategy to effectively identify polysemous entities within KGC datasets. Through extensive experimentation, we have verified the effectiveness of this framework. Even after quantization, the LLM (Qwen-7B-Chat-int4) still enhances the performance of text-based KGC methods \footnote{Code and datasets are available at \href{https://github.com/sjlmg/CP-KGC}{https://github.com/sjlmg/CP-KGC}}. This study extends the performance limits of existing models and promotes further integration of KGC with LLMs.

Related papers

• All Against Some: Efficient Integration of Large Language Models for Message Passing in Graph Neural Networks [51.19110891434727]
  Large Language Models (LLMs) with pretrained knowledge and powerful semantic comprehension abilities have recently shown a remarkable ability to benefit applications using vision and text data. E-LLaGNN is a framework with an on-demand LLM service that enriches message passing procedure of graph learning by enhancing a limited fraction of nodes from the graph.
  arXiv  Detail & Related papers  (2024-07-20T22:09:42Z)
• Subgraph-Aware Training of Text-based Methods for Knowledge Graph Completion [4.741342276627672]
  Fine-tuning pre-trained language models (PLMs) has recently shown a potential to improve knowledge graph completion (KGC) We propose a Subgraph-Aware Training framework for KGC (SATKGC) that combines (i) subgraph-aware mini-batching to encourage hard negative sampling, and (ii) a new contrastive learning method to focus more on harder entities and harder negative triples.
  arXiv  Detail & Related papers  (2024-07-17T16:25:37Z)
• Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs [59.76268575344119]
  We introduce a novel framework for enhancing large language models' (LLMs) planning capabilities by using planning data derived from knowledge graphs (KGs) LLMs fine-tuned with KG data have improved planning capabilities, better equipping them to handle complex QA tasks that involve retrieval.
  arXiv  Detail & Related papers  (2024-06-20T13:07:38Z)
• Hierarchical Compression of Text-Rich Graphs via Large Language Models [63.75293588479027]
  Text-rich graphs are prevalent in data mining contexts like e-commerce and academic graphs. This paper introduces Hierarchical Compression'' (HiCom), a novel method to align the capabilities of LLMs with the structure of text-rich graphs. HiCom can outperform both GNNs and LLM backbones for node classification on e-commerce and citation graphs.
  arXiv  Detail & Related papers  (2024-06-13T07:24:46Z)
• Text clustering with LLM embeddings [0.0]
  We investigate how different textual embeddings and clustering algorithms affect how text datasets are clustered. Findings reveal that LLM embeddings excel at capturing subtleties in structured language, while BERT leads the lightweight options in performance.
  arXiv  Detail & Related papers  (2024-03-22T11:08:48Z)
• Multi-perspective Improvement of Knowledge Graph Completion with Large Language Models [95.31941227776711]
  We propose MPIKGC to compensate for the deficiency of contextualized knowledge and improve KGC by querying large language models (LLMs) We conducted extensive evaluation of our framework based on four description-based KGC models and four datasets, for both link prediction and triplet classification tasks.
  arXiv  Detail & Related papers  (2024-03-04T12:16:15Z)
• KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion [27.405080941584533]
  We propose KICGPT, a framework that integrates a large language model and a triple-based KGC retriever. It alleviates the long-tail problem without incurring additional training overhead. Empirical results on benchmark datasets demonstrate the effectiveness of KICGPT with smaller training overhead and no finetuning.
  arXiv  Detail & Related papers  (2024-02-04T08:01:07Z)
• Contextualization Distillation from Large Language Model for Knowledge Graph Completion [51.126166442122546]
  We introduce the Contextualization Distillation strategy, a plug-in-and-play approach compatible with both discriminative and generative KGC frameworks. Our method begins by instructing large language models to transform compact, structural triplets into context-rich segments. Comprehensive evaluations across diverse datasets and KGC techniques highlight the efficacy and adaptability of our approach.
  arXiv  Detail & Related papers  (2024-01-28T08:56:49Z)
• Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model [38.00241874974804]
  Both graph structures and textual information play a critical role in Knowledge Graph Completion (KGC) We propose a new KGC method named PDKGC with two prompts -- a hard task prompt and a disentangled structure prompt. With the two prompts, PDKGC builds a textual predictor and a structural predictor, respectively, and their combination leads to more comprehensive entity prediction.
  arXiv  Detail & Related papers  (2023-12-04T12:20:25Z)
• Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning [51.90524745663737]
  A key innovation is our use of explanations as features, which can be used to boost GNN performance on downstream tasks. Our method achieves state-of-the-art results on well-established TAG datasets. Our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv.
  arXiv  Detail & Related papers  (2023-05-31T03:18:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.