In-Context Learning with Topological Information for Knowledge Graph Completion

• URL: http://arxiv.org/abs/2412.08742v1
• Date: Wed, 11 Dec 2024 19:29:36 GMT
• Title: In-Context Learning with Topological Information for Knowledge Graph Completion
• Authors: Udari Madhushani Sehwag, Kassiani Papasotiriou, Jared Vann, Sumitra Ganesh,
• Abstract summary: We develop a novel method that incorporates topological information through in-context learning to enhance knowledge graph performance. Our approach achieves strong performance in the transductive setting i.e., nodes in the test graph dataset are present in the training graph dataset. Our method demonstrates superior performance compared to baselines on the ILPC-small and ILPC-large datasets.
• Score: 3.035601871864059
• License:
• Abstract: Knowledge graphs (KGs) are crucial for representing and reasoning over structured information, supporting a wide range of applications such as information retrieval, question answering, and decision-making. However, their effectiveness is often hindered by incompleteness, limiting their potential for real-world impact. While knowledge graph completion (KGC) has been extensively studied in the literature, recent advances in generative AI models, particularly large language models (LLMs), have introduced new opportunities for innovation. In-context learning has recently emerged as a promising approach for leveraging pretrained knowledge of LLMs across a range of natural language processing tasks and has been widely adopted in both academia and industry. However, how to utilize in-context learning for effective KGC remains relatively underexplored. We develop a novel method that incorporates topological information through in-context learning to enhance KGC performance. By integrating ontological knowledge and graph structure into the context of LLMs, our approach achieves strong performance in the transductive setting i.e., nodes in the test graph dataset are present in the training graph dataset. Furthermore, we apply our approach to KGC in the more challenging inductive setting, i.e., nodes in the training graph dataset and test graph dataset are disjoint, leveraging the ontology to infer useful information about missing nodes which serve as contextual cues for the LLM during inference. Our method demonstrates superior performance compared to baselines on the ILPC-small and ILPC-large datasets.

Related papers

• Graph Foundation Models for Recommendation: A Comprehensive Survey [55.70529188101446]
  Large language models (LLMs) are designed to process and comprehend natural language, making both approaches highly effective and widely adopted. Recent research has focused on graph foundation models (GFMs) GFMs integrate the strengths of GNNs and LLMs to model complex RS problems more efficiently by leveraging the graph-based structure of user-item relationships alongside textual understanding.
  arXiv  Detail & Related papers  (2025-02-12T12:13:51Z)
• All Against Some: Efficient Integration of Large Language Models for Message Passing in Graph Neural Networks [51.19110891434727]
  Large Language Models (LLMs) with pretrained knowledge and powerful semantic comprehension abilities have recently shown a remarkable ability to benefit applications using vision and text data. E-LLaGNN is a framework with an on-demand LLM service that enriches message passing procedure of graph learning by enhancing a limited fraction of nodes from the graph.
  arXiv  Detail & Related papers  (2024-07-20T22:09:42Z)
• Enhancing Graph Neural Networks with Limited Labeled Data by Actively Distilling Knowledge from Large Language Models [30.867447814409623]
  Graph neural networks (GNNs) have great ability in node classification, a fundamental task on graphs. We propose a novel approach that integrates Large Language Models (LLMs) and GNNs. Our model in improving node classification accuracy with considerably limited labeled data, surpassing state-of-the-art baselines by significant margins.
  arXiv  Detail & Related papers  (2024-07-19T02:34:10Z)
• G-SAP: Graph-based Structure-Aware Prompt Learning over Heterogeneous Knowledge for Commonsense Reasoning [8.02547453169677]
  We propose a novel Graph-based Structure-Aware Prompt Learning Model for commonsense reasoning, named G-SAP. In particular, an evidence graph is constructed by integrating multiple knowledge sources, i.e. ConceptNet, Wikipedia, and Cambridge Dictionary. The results reveal a significant advancement over the existing models, especially, with 6.12% improvement over the SoTA LM+GNNs model on the OpenbookQA dataset.
  arXiv  Detail & Related papers  (2024-05-09T08:28:12Z)
• LLM-Enhanced User-Item Interactions: Leveraging Edge Information for Optimized Recommendations [28.77605585519833]
  Graph neural networks, as a popular research area in recent years, have numerous studies on relationship mining. Current cutting-edge research in graph neural networks has not been effectively integrated with large language models. We propose an innovative framework that combines the strong contextual representation capabilities of LLMs with the relationship extraction and analysis functions of GNNs.
  arXiv  Detail & Related papers  (2024-02-14T23:12:09Z)
• Contextualization Distillation from Large Language Model for Knowledge Graph Completion [51.126166442122546]
  We introduce the Contextualization Distillation strategy, a plug-in-and-play approach compatible with both discriminative and generative KGC frameworks. Our method begins by instructing large language models to transform compact, structural triplets into context-rich segments. Comprehensive evaluations across diverse datasets and KGC techniques highlight the efficacy and adaptability of our approach.
  arXiv  Detail & Related papers  (2024-01-28T08:56:49Z)
• Efficient End-to-end Language Model Fine-tuning on Graphs [21.23522552579571]
  Learning from Text-Attributed Graphs (TAGs) has attracted significant attention due to its wide range of real-world applications. We introduce LEADING, a novel and efficient approach for end-to-end fine-tuning of language models on TAGs. Our proposed approach demonstrates superior performance, achieving state-of-the-art (SOTA) results on the ogbn-arxiv leaderboard.
  arXiv  Detail & Related papers  (2023-12-07T22:35:16Z)
• Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations [63.19448893196642]
  We present a novel framework that harnesses the rich contextual information and semantic representations provided by large language models to analyze behavior graphs. By leveraging this capability, our framework enables personalized and accurate job recommendations for individual users.
  arXiv  Detail & Related papers  (2023-07-10T11:29:41Z)
• Iterative Zero-Shot LLM Prompting for Knowledge Graph Construction [104.29108668347727]
  This paper proposes an innovative knowledge graph generation approach that leverages the potential of the latest generative large language models. The approach is conveyed in a pipeline that comprises novel iterative zero-shot and external knowledge-agnostic strategies. We claim that our proposal is a suitable solution for scalable and versatile knowledge graph construction and may be applied to different and novel contexts.
  arXiv  Detail & Related papers  (2023-07-03T16:01:45Z)
• Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction [57.854498238624366]
  We propose a retrieval-augmented approach, which retrieves schema-aware Reference As Prompt (RAP) for data-efficient knowledge graph construction. RAP can dynamically leverage schema and knowledge inherited from human-annotated and weak-supervised data as a prompt for each sample.
  arXiv  Detail & Related papers  (2022-10-19T16:40:28Z)
• Quantifying Challenges in the Application of Graph Representation Learning [0.0]
  We provide an application oriented perspective to a set of popular embedding approaches. We evaluate their representational power with respect to real-world graph properties. Our results suggest that "one-to-fit-all" GRL approaches are hard to define in real-world scenarios.
  arXiv  Detail & Related papers  (2020-06-18T03:19:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.