Enhancing Trustworthiness of Graph Neural Networks with Rank-Based Conformal Training

• URL: http://arxiv.org/abs/2501.02767v1
• Date: Mon, 06 Jan 2025 05:19:24 GMT
• Title: Enhancing Trustworthiness of Graph Neural Networks with Rank-Based Conformal Training
• Authors: Ting Wang, Zhixin Zhou, Rui Luo,
• Abstract summary: Conformal Prediction can produce statistically guaranteed uncertainty estimates. We propose a Rank-based CP during training framework to GNNs (RCP-GNN) for reliable uncertainty estimates.
• Score: 17.120502204791407
• License:
• Abstract: Graph Neural Networks (GNNs) has been widely used in a variety of fields because of their great potential in representing graph-structured data. However, lacking of rigorous uncertainty estimations limits their application in high-stakes. Conformal Prediction (CP) can produce statistically guaranteed uncertainty estimates by using the classifier's probability estimates to obtain prediction sets, which contains the true class with a user-specified probability. In this paper, we propose a Rank-based CP during training framework to GNNs (RCP-GNN) for reliable uncertainty estimates to enhance the trustworthiness of GNNs in the node classification scenario. By exploiting rank information of the classifier's outcome, prediction sets with desired coverage rate can be efficiently constructed. The strategy of CP during training with differentiable rank-based conformity loss function is further explored to adapt prediction sets according to network topology information. In this way, the composition of prediction sets can be guided by the goal of jointly reducing inefficiency and probability estimation errors. Extensive experiments on several real-world datasets show that our model achieves any pre-defined target marginal coverage while significantly reducing the inefficiency compared with state-of-the-art methods.

Related papers

• RoCP-GNN: Robust Conformal Prediction for Graph Neural Networks in Node-Classification [0.0]
  Graph Neural Networks (GNNs) have emerged as powerful tools for predicting outcomes in graph-structured data. One way to address this issue is by providing prediction sets that contain the true label with a predefined probability margin. We propose a novel approach termed Robust Conformal Prediction for GNNs (RoCP-GNN) Our approach robustly predicts outcomes with any predictive GNN model while quantifying the uncertainty in predictions within the realm of graph-based semi-supervised learning (SSL)
  arXiv  Detail & Related papers  (2024-08-25T12:51:19Z)
• Conditional Shift-Robust Conformal Prediction for Graph Neural Network [0.0]
  Graph Neural Networks (GNNs) have emerged as potent tools for predicting outcomes in graph-structured data. Despite their efficacy, GNNs have limited ability to provide robust uncertainty estimates. We propose Conditional Shift Robust (CondSR) conformal prediction for GNNs.
  arXiv  Detail & Related papers  (2024-05-20T11:47:31Z)
• Uncertainty in Graph Neural Networks: A Survey [50.63474656037679]
  Graph Neural Networks (GNNs) have been extensively used in various real-world applications. However, the predictive uncertainty of GNNs stemming from diverse sources can lead to unstable and erroneous predictions. This survey aims to provide a comprehensive overview of the GNNs from the perspective of uncertainty.
  arXiv  Detail & Related papers  (2024-03-11T21:54:52Z)
• Uncertainty Quantification over Graph with Conformalized Graph Neural Networks [52.20904874696597]
  Graph Neural Networks (GNNs) are powerful machine learning prediction models on graph-structured data. GNNs lack rigorous uncertainty estimates, limiting their reliable deployment in settings where the cost of errors is significant. We propose conformalized GNN (CF-GNN), extending conformal prediction (CP) to graph-based models for guaranteed uncertainty estimates.
  arXiv  Detail & Related papers  (2023-05-23T21:38:23Z)
• Distribution Free Prediction Sets for Node Classification [0.0]
  We leverage recent advances in conformal prediction to construct prediction sets for node classification in inductive learning scenarios. We show through experiments on standard benchmark datasets using popular GNN models that our approach provides tighter and better prediction sets than a naive application of conformal prediction.
  arXiv  Detail & Related papers  (2022-11-26T12:54:45Z)
• Predicting Deep Neural Network Generalization with Perturbation Response Curves [58.8755389068888]
  We propose a new framework for evaluating the generalization capabilities of trained networks. Specifically, we introduce two new measures for accurately predicting generalization gaps. We attain better predictive scores than the current state-of-the-art measures on a majority of tasks in the Predicting Generalization in Deep Learning (PGDL) NeurIPS 2020 competition.
  arXiv  Detail & Related papers  (2021-06-09T01:37:36Z)
• Failure Prediction by Confidence Estimation of Uncertainty-Aware Dirichlet Networks [6.700873164609009]
  It is shown that uncertainty-aware deep Dirichlet neural networks provide an improved separation between the confidence of correct and incorrect predictions in the true class probability metric. A new criterion is proposed for learning the true class probability by matching prediction confidence scores while taking imbalance and TCP constraints into account.
  arXiv  Detail & Related papers  (2020-10-19T21:06:45Z)
• Evaluating probabilistic classifiers: Reliability diagrams and score decompositions revisited [68.8204255655161]
  We introduce the CORP approach, which generates provably statistically Consistent, Optimally binned, and Reproducible reliability diagrams in an automated way. Corpor is based on non-parametric isotonic regression and implemented via the Pool-adjacent-violators (PAV) algorithm.
  arXiv  Detail & Related papers  (2020-08-07T08:22:26Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Frequentist Uncertainty in Recurrent Neural Networks via Blockwise Influence Functions [121.10450359856242]
  Recurrent neural networks (RNNs) are instrumental in modelling sequential and time-series data. Existing approaches for uncertainty quantification in RNNs are based predominantly on Bayesian methods. We develop a frequentist alternative that: (a) does not interfere with model training or compromise its accuracy, (b) applies to any RNN architecture, and (c) provides theoretical coverage guarantees on the estimated uncertainty intervals.
  arXiv  Detail & Related papers  (2020-06-20T22:45:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.