Towards Trustworthy Knowledge Graph Reasoning: An Uncertainty Aware Perspective

• URL: http://arxiv.org/abs/2410.08985v2
• Date: Sun, 20 Oct 2024 19:35:09 GMT
• Title: Towards Trustworthy Knowledge Graph Reasoning: An Uncertainty Aware Perspective
• Abstract summary: We propose a new trustworthy KG-LLM framework, Uncertainty Aware Knowledge-Graph Reasoning (UAG) We design an uncertainty-aware multi-step reasoning framework that leverages conformal prediction to provide a theoretical guarantee on the prediction set. Our proposed UAG can achieve any pre-defined coverage rate while reducing the prediction set/interval size by 40% on average.
• Score: 17.39302677990171
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, Knowledge Graphs (KGs) have been successfully coupled with Large Language Models (LLMs) to mitigate their hallucinations and enhance their reasoning capability, such as in KG-based retrieval-augmented frameworks. However, current KG-LLM frameworks lack rigorous uncertainty estimation, limiting their reliable deployment in high-stakes applications. Directly incorporating uncertainty quantification into KG-LLM frameworks presents challenges due to their complex architectures and the intricate interactions between the knowledge graph and language model components. To address this gap, we propose a new trustworthy KG-LLM framework, Uncertainty Aware Knowledge-Graph Reasoning (UAG), which incorporates uncertainty quantification into the KG-LLM framework. We design an uncertainty-aware multi-step reasoning framework that leverages conformal prediction to provide a theoretical guarantee on the prediction set. To manage the error rate of the multi-step process, we additionally introduce an error rate control module to adjust the error rate within the individual components. Extensive experiments show that our proposed UAG can achieve any pre-defined coverage rate while reducing the prediction set/interval size by 40% on average over the baselines.

Related papers

• Conformal Changepoint Localization and Root Cause Analysis with Corrupted Observations [55.76952683833966]
  This paper proposes weighted CONCH (W-CONCH) and weighted CROC (W-CROC) to reduce confidence set size when data may be corrupted.<n>Experiments on image-based and real-world changepoint and root-cause benchmarks show that uncertainty-based weighting substantially reduces confidence set size.
  arXiv  Detail & Related papers  (2026-07-29T05:16:59Z)
• Quantifying Faithful Confidence Expression in Large Reasoning Models [40.863410553607984]
  We introduce a novel framework to quantify faithful calibration of large reasoning models.<n>Our framework analyzes linguistic decisiveness relative to three sources of internal uncertainty.<n>We find that faithful confidence expression is a significant challenge for LRMs.
  arXiv  Detail & Related papers  (2026-06-02T17:53:45Z)
• FERA: Uncertainty-Aware Federated Reasoning for Large Language Models [60.52562148874846]
  We study federated reasoning, where a server improves multi-step reasoning by coordinating with heterogeneous clients holding private demonstrations.<n>The key challenge is that client reliability is query-dependent, while the server cannot inspect client data to determine which contributions are trustworthy.<n>We propose Uncertainty-Aware Federated Reasoning (FERA), a training-free framework based on iterative server-client co-refinement.
  arXiv  Detail & Related papers  (2026-05-11T07:04:51Z)
• Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learning [49.28548464288051]
  Composed Image Retrieval (CIR) enables image search by combining a reference image with modification text.<n>In intrinsic noise in CIR triplets incurs intrinsic uncertainty and threatens the model's robustness.<n>This paper introduces a Heterogeneous Uncertainty-Guided (HUG) paradigm to overcome these limitations.
  arXiv  Detail & Related papers  (2026-01-16T16:05:49Z)
• CoG: Controllable Graph Reasoning via Relational Blueprints and Failure-Aware Refinement over Knowledge Graphs [53.199517625701475]
  CoG is a training-free framework inspired by Dual-Process Theory that mimics the interplay between intuition and deliberation.<n>CoG significantly outperforms state-of-the-art approaches in both accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-01-16T07:27:40Z)
• When to Trust: A Causality-Aware Calibration Framework for Accurate Knowledge Graph Retrieval-Augmented Generation [9.05691591939794]
  Knowledge Graph Retrieval-Augmented Generation (KG-RAG) extends the RAG paradigm by incorporating structured knowledge from knowledge graphs.<n>Existing KG-RAG models are often severely overconfident, producing high-confidence predictions even when retrieved sub-graphs are incomplete or unreliable.<n>We propose Ca2KG, a Causality-aware framework for KG-RAG, which integrates counterfactual prompting, with a panel-based re-scoring mechanism.
  arXiv  Detail & Related papers  (2026-01-14T07:22:59Z)
• Certainty in Uncertainty: Reasoning over Uncertain Knowledge Graphs with Statistical Guarantees [24.48143253497661]
  textscUnKGCP generates prediction intervals guaranteed to contain the true score with a user-specified level of confidence.<n>We provide theoretical guarantees for the intervals and empirically verify these guarantees.<n>Experiments on standard benchmarks across diverse UnKGE methods further demonstrate that the intervals are sharp and effectively capture predictive uncertainty.
  arXiv  Detail & Related papers  (2025-10-18T17:58:17Z)
• GENUINE: Graph Enhanced Multi-level Uncertainty Estimation for Large Language Models [15.035039054096188]
  Uncertainty estimation is essential for enhancing the reliability of Large Language Models.<n>We propose GENUINE: Graph ENhanced mUlti-level uncertaINty Estimation for Large Language Models.
  arXiv  Detail & Related papers  (2025-09-09T17:07:44Z)
• An Uncertainty-aware DETR Enhancement Framework for Object Detection [10.102900613370817]
  We propose an uncertainty-aware enhancement framework for DETR-based object detectors.<n>We derive a Bayes Risk formulation to filter high-risk information and improve detection reliability.<n> Experiments on the COCO benchmark show that our method can be effectively integrated into existing DETR variants.
  arXiv  Detail & Related papers  (2025-07-20T07:53:04Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Hierarchical Uncertainty-Aware Graph Neural Network [1.7495213911983414]
  We introduce the Hierarchical Uncertainty-Aware Graph Neural Network (HU-GNN), which unifies multi-scale representation learning, principled uncertainty estimation, and self-supervised embedding diversity within a single end-to-end framework. Specifically, HU-GNN adaptively forms node clusters and estimates uncertainty at multiple structural scales from individual nodes to higher levels.
  arXiv  Detail & Related papers  (2025-04-28T14:22:18Z)
• Enhancing LLM Reliability via Explicit Knowledge Boundary Modeling [48.15636223774418]
  Large language models (LLMs) frequently hallucinate due to misaligned self-awareness. Existing approaches mitigate hallucinations via uncertainty estimation or query rejection. We propose the Explicit Knowledge Boundary Modeling framework to integrate fast and slow reasoning systems.
  arXiv  Detail & Related papers  (2025-03-04T03:16:02Z)
• Quantifying calibration error in modern neural networks through evidence based theory [0.0]
  This paper introduces a novel framework for quantifying the trustworthiness of neural networks by incorporating subjective logic into the evaluation of Expected Error (ECE) We demonstrate the effectiveness of this approach through experiments on MNIST and CIFAR-10 datasets where post-calibration results indicate improved trustworthiness. The proposed framework offers a more interpretable and nuanced assessment of AI models, with potential applications in sensitive domains such as healthcare and autonomous systems.
  arXiv  Detail & Related papers  (2024-10-31T23:54:21Z)
• Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study over Open-ended Question Answering [35.2451096137883]
  We introduce OKGQA, a new benchmark specifically designed to assess Large Language Models (LLMs) enhanced with Knowledge Graphs (KGs) OKGQA is designed to closely reflect the complexities of practical applications using questions from different types, and incorporates specific metrics to measure both the reduction in hallucinations and the enhancement in reasoning capabilities. We also propose OKGQA-P to assess model performance when the semantics and structure of KGs are deliberately perturbed and contaminated.
  arXiv  Detail & Related papers  (2024-10-10T16:29:21Z)
• ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees [68.33498595506941]
  We introduce a novel uncertainty measure based on self-consistency theory. We then develop a conformal uncertainty criterion by integrating the uncertainty condition aligned with correctness into the CP algorithm. Empirical evaluations indicate that our uncertainty measure outperforms prior state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-06-29T17:33:07Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• Uncertainty-Aware Relational Graph Neural Network for Few-Shot Knowledge Graph Completion [12.887073684904147]
  Few-shot knowledge graph completion (FKGC) aims to query the unseen facts of a relation given its few-shot reference entity pairs. Existing FKGC works neglect such uncertainty, which leads them more susceptible to limited reference samples with noises. We propose a novel uncertainty-aware few-shot KG completion framework (UFKGC) to model uncertainty for a better understanding of the limited data.
  arXiv  Detail & Related papers  (2024-03-07T14:23:25Z)
• Federated Knowledge Graph Completion via Latent Embedding Sharing and Tensor Factorization [51.286715478399515]
  Federated Latent Embedding factorization (FLEST) is a novel approach using federated factorization for KG completion. FLEST decomposes the embedding matrix and enables sharing of latent dictionary embeddings to lower privacy risks. Empirical results demonstrate FLEST's effectiveness and efficiency, offering a balanced solution between performance and privacy.
  arXiv  Detail & Related papers  (2023-11-17T06:03:56Z)
• Normalizing Flow-based Neural Process for Few-Shot Knowledge Graph Completion [69.55700751102376]
  Few-shot knowledge graph completion (FKGC) aims to predict missing facts for unseen relations with few-shot associated facts. Existing FKGC methods are based on metric learning or meta-learning, which often suffer from the out-of-distribution and overfitting problems. In this paper, we propose a normalizing flow-based neural process for few-shot knowledge graph completion (NP-FKGC)
  arXiv  Detail & Related papers  (2023-04-17T11:42:28Z)
• Lightweight, Uncertainty-Aware Conformalized Visual Odometry [2.429910016019183]
  Data-driven visual odometry (VO) is a critical subroutine for autonomous edge robotics. Emerging edge robotics devices like insect-scale drones and surgical robots lack a computationally efficient framework to estimate VO's predictive uncertainties. This paper presents a novel, lightweight, and statistically robust framework that leverages conformal inference (CI) to extract VO's uncertainty bands.
  arXiv  Detail & Related papers  (2023-03-03T20:37:55Z)
• Uncertainty in Extreme Multi-label Classification [81.14232824864787]
  eXtreme Multi-label Classification (XMC) is an essential task in the era of big data for web-scale machine learning applications. In this paper, we aim to investigate general uncertainty quantification approaches for tree-based XMC models with a probabilistic ensemble-based framework. In particular, we analyze label-level and instance-level uncertainty in XMC, and propose a general approximation framework based on beam search to efficiently estimate the uncertainty with a theoretical guarantee under long-tail XMC predictions.
  arXiv  Detail & Related papers  (2022-10-18T20:54:33Z)
• Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization [69.07420650261649]
  We introduce a novel, simple, and powerful contrastive MI estimator named as FLO. Empirically, our FLO estimator overcomes the limitations of its predecessors and learns more efficiently. The utility of FLO is verified using an extensive set of benchmarks, which also reveals the trade-offs in practical MI estimation.
  arXiv  Detail & Related papers  (2021-07-02T15:20:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.