Predictive Multiplicity of Knowledge Graph Embeddings in Link Prediction

• URL: http://arxiv.org/abs/2408.08226v2
• Date: Fri, 4 Oct 2024 14:03:41 GMT
• Title: Predictive Multiplicity of Knowledge Graph Embeddings in Link Prediction
• Authors: Yuqicheng Zhu, Nico Potyka, Mojtaba Nayyeri, Bo Xiong, Yunjie He, Evgeny Kharlamov, Steffen Staab,
• Abstract summary: Knowledge graph embedding (KGE) models are often used to predict missing links for knowledge graphs (KGs) Multiple KG embeddings can perform almost equally well for link prediction yet give conflicting predictions for unseen queries. It poses substantial risks for KGE-based applications in high-stake domains but has been overlooked in KGE research.
• Score: 31.79167587393948
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Knowledge graph embedding (KGE) models are often used to predict missing links for knowledge graphs (KGs). However, multiple KG embeddings can perform almost equally well for link prediction yet give conflicting predictions for unseen queries. This phenomenon is termed \textit{predictive multiplicity} in the literature. It poses substantial risks for KGE-based applications in high-stake domains but has been overlooked in KGE research. We define predictive multiplicity in link prediction, introduce evaluation metrics and measure predictive multiplicity for representative KGE methods on commonly used benchmark datasets. Our empirical study reveals significant predictive multiplicity in link prediction, with $8\%$ to $39\%$ testing queries exhibiting conflicting predictions. We address this issue by leveraging voting methods from social choice theory, significantly mitigating conflicts by $66\%$ to $78\%$ in our experiments.

Related papers

• Uncertainty-Aware Graph Neural Networks: A Multi-Hop Evidence Fusion Approach [55.43914153271912]
  Graph neural networks (GNNs) excel in graph representation learning by integrating graph structure and node features.<n>Existing GNNs fail to account for the uncertainty of class probabilities that vary with the depth of the model, leading to unreliable and risky predictions in real-world scenarios.<n>We propose a novel Evidence Fusing Graph Neural Network (EFGNN for short) to achieve trustworthy prediction, enhance node classification accuracy, and make explicit the risk of wrong predictions.
  arXiv  Detail & Related papers  (2025-06-16T03:59:38Z)
• PageRank Bandits for Link Prediction [72.61386754332776]
  Link prediction is a critical problem in graph learning with broad applications such as recommender systems and knowledge graph completion. This paper reformulates link prediction as a sequential decision-making process, where each link prediction interaction occurs sequentially. We propose a novel fusion algorithm, PRB (PageRank Bandits), which is the first to combine contextual bandits with PageRank for collaborative exploitation and exploration.
  arXiv  Detail & Related papers  (2024-11-03T02:39:28Z)
• Correct after Answer: Enhancing Multi-Span Question Answering with Post-Processing Method [11.794628063040108]
  Multi-Span Question Answering (MSQA) requires models to extract one or multiple answer spans from a given context to answer a question. We propose Answering-Classifying-Correcting (ACC) framework, which employs a post-processing strategy to handle incorrect predictions.
  arXiv  Detail & Related papers  (2024-10-22T08:04:32Z)
• Conformalized Answer Set Prediction for Knowledge Graph Embedding [26.803481853670064]
  We show how conformal prediction can be used to generate correct answer sets for link prediction tasks. We also demonstrate that the generated answer sets often have a sensible size and that the size adapts well with respect to the difficulty of the query.
  arXiv  Detail & Related papers  (2024-08-15T16:36:59Z)
• Conformalized Link Prediction on Graph Neural Networks [8.807684750444626]
  Graph Neural Networks (GNNs) excel in diverse tasks, yet their applications in high-stakes domains are often hampered by unreliable predictions. We introduce a distribution-free and model-agnostic uncertainty quantification approach to construct a predictive interval with a statistical guarantee for GNN-based link prediction.
  arXiv  Detail & Related papers  (2024-06-26T21:17:37Z)
• Less is More: One-shot Subgraph Reasoning on Large-scale Knowledge Graphs [49.547988001231424]
  We propose the one-shot-subgraph link prediction to achieve efficient and adaptive prediction. Design principle is that, instead of directly acting on the whole KG, the prediction procedure is decoupled into two steps. We achieve promoted efficiency and leading performances on five large-scale benchmarks.
  arXiv  Detail & Related papers  (2024-03-15T12:00:12Z)
• Enhancing Hyperedge Prediction with Context-Aware Self-Supervised Learning [64.46188414653204]
  We propose a novel hyperedge prediction framework (CASH) CASH employs context-aware node aggregation to capture complex relations among nodes in each hyperedge for (C1) and (2) self-supervised contrastive learning in the context of hyperedge prediction to enhance hypergraph representations for (C2) Experiments on six real-world hypergraphs reveal that CASH consistently outperforms all competing methods in terms of the accuracy in hyperedge prediction.
  arXiv  Detail & Related papers  (2023-09-11T20:06:00Z)
• NETpred: Network-based modeling and prediction of multiple connected market indices [8.122270502556372]
  We introduce a framework called NETpred that generates a novel graph representing multiple related indices and their stocks. It then thoroughly selects a diverse set of representative nodes that cover different parts of the state space and whose price movements are accurately predictable. The resulting model is then used to predict the stock labels which are finally aggregated to infer the labels for all the index nodes in the graph.
  arXiv  Detail & Related papers  (2022-12-02T17:23:09Z)
• Probable Domain Generalization via Quantile Risk Minimization [90.15831047587302]
  Domain generalization seeks predictors which perform well on unseen test distributions. We propose a new probabilistic framework for DG where the goal is to learn predictors that perform well with high probability.
  arXiv  Detail & Related papers  (2022-07-20T14:41:09Z)
• RelWalk A Latent Variable Model Approach to Knowledge Graph Embedding [50.010601631982425]
  This paper extends the random walk model (Arora et al., 2016a) of word embeddings to Knowledge Graph Embeddings (KGEs) We derive a scoring function that evaluates the strength of a relation R between two entities h (head) and t (tail) We propose a learning objective motivated by the theoretical analysis to learn KGEs from a given knowledge graph.
  arXiv  Detail & Related papers  (2021-01-25T13:31:29Z)
• Adversarial Learning for Debiasing Knowledge Graph Embeddings [9.53284633479507]
  Social and cultural biases can have detrimental consequences on different population and minority groups. This paper aims at identifying and mitigating such biases in Knowledge Graph (KG) embeddings. We introduce a novel framework to filter out the sensitive attribute information from the KG embeddings, which we call FAN (Filtering Adversarial Network)
  arXiv  Detail & Related papers  (2020-06-29T18:36:15Z)
• Ambiguity in Sequential Data: Predicting Uncertain Futures with Recurrent Models [110.82452096672182]
  We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data. We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
  arXiv  Detail & Related papers  (2020-03-10T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.