A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning

• URL: http://arxiv.org/abs/2410.12288v1
• Date: Wed, 16 Oct 2024 06:47:18 GMT
• Title: A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning
• Authors: Yuanning Cui, Zequn Sun, Wei Hu,
• Abstract summary: We propose a prompt-based KG foundation model via in-context learning, namely KG-ICL, to achieve a universal reasoning ability. To encode prompt graphs with the generalization ability to unseen entities and relations in queries, we first propose a unified tokenizer. Then, we propose two message passing neural networks to perform prompt encoding and KG reasoning, respectively.
• Score: 17.676185326247946
• License:
• Abstract: Extensive knowledge graphs (KGs) have been constructed to facilitate knowledge-driven tasks across various scenarios. However, existing work usually develops separate reasoning models for different KGs, lacking the ability to generalize and transfer knowledge across diverse KGs and reasoning settings. In this paper, we propose a prompt-based KG foundation model via in-context learning, namely KG-ICL, to achieve a universal reasoning ability. Specifically, we introduce a prompt graph centered with a query-related example fact as context to understand the query relation. To encode prompt graphs with the generalization ability to unseen entities and relations in queries, we first propose a unified tokenizer that maps entities and relations in prompt graphs to predefined tokens. Then, we propose two message passing neural networks to perform prompt encoding and KG reasoning, respectively. We conduct evaluation on 43 different KGs in both transductive and inductive settings. Results indicate that the proposed KG-ICL outperforms baselines on most datasets, showcasing its outstanding generalization and universal reasoning capabilities. The source code is accessible on GitHub: https://github.com/nju-websoft/KG-ICL.

Related papers

• Decoding on Graphs: Faithful and Sound Reasoning on Knowledge Graphs through Generation of Well-Formed Chains [66.55612528039894]
  Knowledge Graphs (KGs) can serve as reliable knowledge sources for question answering (QA) We present DoG (Decoding on Graphs), a novel framework that facilitates a deep synergy between LLMs and KGs. Experiments across various KGQA tasks with different background KGs demonstrate that DoG achieves superior and robust performance.
  arXiv  Detail & Related papers  (2024-10-24T04:01:40Z)
• Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models [83.28737898989694]
  Large language models (LLMs) struggle with faithful reasoning due to knowledge gaps and hallucinations. We introduce graph-constrained reasoning (GCR), a novel framework that bridges structured knowledge in KGs with unstructured reasoning in LLMs. GCR achieves state-of-the-art performance and exhibits strong zero-shot generalizability to unseen KGs without additional training.
  arXiv  Detail & Related papers  (2024-10-16T22:55:17Z)
• Context Graph [8.02985792541121]
  We present a context graph reasoning textbfCGR$3$ paradigm that leverages large language models (LLMs) to retrieve candidate entities and related contexts. Our experimental results demonstrate that CGR$3$ significantly improves performance on KG completion (KGC) and KG question answering (KGQA) tasks.
  arXiv  Detail & Related papers  (2024-06-17T02:59:19Z)
• Multi-hop Question Answering over Knowledge Graphs using Large Language Models [1.8130068086063336]
  We evaluate the capability of (LLMs) to answer questions over Knowledge graphs that involve multiple hops. We show that depending upon the size and nature of the KG we need different approaches to extract and feed the relevant information to an LLM.
  arXiv  Detail & Related papers  (2024-04-30T03:31:03Z)
• Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering [87.67177556994525]
  We propose a training-free method called Generate-on-Graph (GoG) to generate new factual triples while exploring Knowledge Graphs (KGs) GoG performs reasoning through a Thinking-Searching-Generating framework, which treats LLM as both Agent and KG in IKGQA.
  arXiv  Detail & Related papers  (2024-04-23T04:47:22Z)
• Knowledge Graphs Querying [4.548471481431569]
  We aim at uniting different interdisciplinary topics and concepts that have been developed for KG querying. Recent advances on KG and query embedding, multimodal KG, and KG-QA come from deep learning, IR, NLP, and computer vision domains.
  arXiv  Detail & Related papers  (2023-05-23T19:32:42Z)
• A Retrieve-and-Read Framework for Knowledge Graph Link Prediction [13.91545690758128]
  Knowledge graph (KG) link prediction aims to infer new facts based on existing facts in the KG. Recent studies have shown that using the graph neighborhood of a node via graph neural networks (GNNs) provides more useful information compared to just using the query information. We propose a novel retrieve-and-read framework, which first retrieves a relevant subgraph context for the query and then jointly reasons over the context and the query with a high-capacity reader.
  arXiv  Detail & Related papers  (2022-12-19T18:50:54Z)
• Explainable Sparse Knowledge Graph Completion via High-order Graph Reasoning Network [111.67744771462873]
  This paper proposes a novel explainable model for sparse Knowledge Graphs (KGs) It combines high-order reasoning into a graph convolutional network, namely HoGRN. It can not only improve the generalization ability to mitigate the information insufficiency issue but also provide interpretability.
  arXiv  Detail & Related papers  (2022-07-14T10:16:56Z)
• GreaseLM: Graph REASoning Enhanced Language Models for Question Answering [159.9645181522436]
  GreaseLM is a new model that fuses encoded representations from pretrained LMs and graph neural networks over multiple layers of modality interaction operations. We show that GreaseLM can more reliably answer questions that require reasoning over both situational constraints and structured knowledge, even outperforming models 8x larger.
  arXiv  Detail & Related papers  (2022-01-21T19:00:05Z)
• Toward Subgraph-Guided Knowledge Graph Question Generation with Graph Neural Networks [53.58077686470096]
  Knowledge graph (KG) question generation (QG) aims to generate natural language questions from KGs and target answers. In this work, we focus on a more realistic setting where we aim to generate questions from a KG subgraph and target answers.
  arXiv  Detail & Related papers  (2020-04-13T15:43:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.