KG-FIT: Knowledge Graph Fine-Tuning Upon Open-World Knowledge

• URL: http://arxiv.org/abs/2405.16412v3
• Date: Sun, 27 Oct 2024 23:31:49 GMT
• Title: KG-FIT: Knowledge Graph Fine-Tuning Upon Open-World Knowledge
• Authors: Pengcheng Jiang, Lang Cao, Cao Xiao, Parminder Bhatia, Jimeng Sun, Jiawei Han,
• Abstract summary: Knowledge Graph Embedding (KGE) techniques are crucial in learning compact representations of entities and relations within a knowledge graph. This study introduces KG-FIT, which builds a semantically coherent hierarchical structure of entity clusters. Experiments on the benchmark datasets FB15K-237, YAGO3-10, and PrimeKG demonstrate the superiority of KG-FIT over state-of-the-art pre-trained language model-based methods.
• Score: 63.19837262782962
• License:
• Abstract: Knowledge Graph Embedding (KGE) techniques are crucial in learning compact representations of entities and relations within a knowledge graph, facilitating efficient reasoning and knowledge discovery. While existing methods typically focus either on training KGE models solely based on graph structure or fine-tuning pre-trained language models with classification data in KG, KG-FIT leverages LLM-guided refinement to construct a semantically coherent hierarchical structure of entity clusters. By incorporating this hierarchical knowledge along with textual information during the fine-tuning process, KG-FIT effectively captures both global semantics from the LLM and local semantics from the KG. Extensive experiments on the benchmark datasets FB15K-237, YAGO3-10, and PrimeKG demonstrate the superiority of KG-FIT over state-of-the-art pre-trained language model-based methods, achieving improvements of 14.4%, 13.5%, and 11.9% in the Hits@10 metric for the link prediction task, respectively. Furthermore, KG-FIT yields substantial performance gains of 12.6%, 6.7%, and 17.7% compared to the structure-based base models upon which it is built. These results highlight the effectiveness of KG-FIT in incorporating open-world knowledge from LLMs to significantly enhance the expressiveness and informativeness of KG embeddings.

Related papers

• Distill-SynthKG: Distilling Knowledge Graph Synthesis Workflow for Improved Coverage and Efficiency [59.6772484292295]
  Knowledge graphs (KGs) generated by large language models (LLMs) are increasingly valuable for Retrieval-Augmented Generation (RAG) applications. Existing KG extraction methods rely on prompt-based approaches, which are inefficient for processing large-scale corpora. We propose SynthKG, a multi-step, document-level synthesis KG workflow based on LLMs. We also design a novel graph-based retrieval framework for RAG.
  arXiv  Detail & Related papers  (2024-10-22T00:47:54Z)
• Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering [87.67177556994525]
  We propose a training-free method called Generate-on-Graph (GoG) to generate new factual triples while exploring Knowledge Graphs (KGs) GoG performs reasoning through a Thinking-Searching-Generating framework, which treats LLM as both Agent and KG in IKGQA.
  arXiv  Detail & Related papers  (2024-04-23T04:47:22Z)
• Multi-perspective Improvement of Knowledge Graph Completion with Large Language Models [95.31941227776711]
  We propose MPIKGC to compensate for the deficiency of contextualized knowledge and improve KGC by querying large language models (LLMs) We conducted extensive evaluation of our framework based on four description-based KGC models and four datasets, for both link prediction and triplet classification tasks.
  arXiv  Detail & Related papers  (2024-03-04T12:16:15Z)
• FedMKGC: Privacy-Preserving Federated Multilingual Knowledge Graph Completion [21.4302940596294]
  Knowledge graph completion (KGC) aims to predict missing facts in knowledge graphs (KGs) Previous methods that rely on transferring raw data among KGs raise privacy concerns. We propose a new federated learning framework that implicitly aggregates knowledge from multiple KGs without demanding raw data exchange and entity alignment.
  arXiv  Detail & Related papers  (2023-12-17T08:09:27Z)
• KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models [18.20425100517317]
  We propose KG-GPT, a framework leveraging large language models for tasks employing knowledge graphs. KG-GPT comprises three steps: Sentence, Graph Retrieval, and Inference, each aimed at partitioning sentences, retrieving relevant graph components, and deriving logical conclusions. We evaluate KG-GPT using KG-based fact verification and KGQA benchmarks, with the model showing competitive and robust performance, even outperforming several fully-supervised models.
  arXiv  Detail & Related papers  (2023-10-17T12:51:35Z)
• Collective Knowledge Graph Completion with Mutual Knowledge Distillation [11.922522192224145]
  We study the problem of multi-KG completion, where we focus on maximizing the collective knowledge from different KGs. We propose a novel method called CKGC-CKD that uses relation-aware graph convolutional network encoder models on both individual KGs and a large fused KG. Experimental results on multilingual datasets have shown that our method outperforms all state-of-the-art models in the KGC task.
  arXiv  Detail & Related papers  (2023-05-25T09:49:40Z)
• Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models [53.09723678623779]
  We propose TAGREAL to automatically generate quality query prompts and retrieve support information from large text corpora. The results show that TAGREAL achieves state-of-the-art performance on two benchmark datasets. We find that TAGREAL has superb performance even with limited training data, outperforming existing embedding-based, graph-based, and PLM-based methods.
  arXiv  Detail & Related papers  (2023-05-24T22:09:35Z)
• KGNN: Distributed Framework for Graph Neural Knowledge Representation [38.080926752998586]
  We develop a novel framework called KGNN to take full advantage of knowledge data for representation learning in the distributed learning system. KGNN is equipped with GNN based encoder and knowledge aware decoder, which aim to jointly explore high-order structure and attribute information together.
  arXiv  Detail & Related papers  (2022-05-17T12:32:02Z)
• Meta-Learning Based Knowledge Extrapolation for Knowledge Graphs in the Federated Setting [43.85991094675398]
  We study the knowledge extrapolation problem to embed new components (i.e., entities and relations) that come with emerging knowledge graphs (KGs) in the federated setting. In this problem, a model trained on an existing KG needs to embed an emerging KG with unseen entities and relations. We introduce the meta-learning setting, where a set of tasks are sampled on the existing KG to mimic the link prediction task on the emerging KG. Based on sampled tasks, we meta-train a graph neural network framework that can construct features for unseen components based on structural information and output embeddings for them.
  arXiv  Detail & Related papers  (2022-05-10T06:27:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.