Accurate and Scalable Estimation of Epistemic Uncertainty for Graph
Neural Networks
- URL: http://arxiv.org/abs/2401.03350v1
- Date: Sun, 7 Jan 2024 00:58:33 GMT
- Title: Accurate and Scalable Estimation of Epistemic Uncertainty for Graph
Neural Networks
- Authors: Puja Trivedi, Mark Heimann, Rushil Anirudh, Danai Koutra, Jayaraman J.
Thiagarajan
- Abstract summary: We propose a novel training framework designed to improve intrinsic GNN uncertainty estimates.
Our framework adapts the principle of centering data to graph data through novel graph anchoring strategies.
Our work provides insights into uncertainty estimation for GNNs, and demonstrates the utility of G-$Delta$UQ in obtaining reliable estimates.
- Score: 40.95782849532316
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: While graph neural networks (GNNs) are widely used for node and graph
representation learning tasks, the reliability of GNN uncertainty estimates
under distribution shifts remains relatively under-explored. Indeed, while
post-hoc calibration strategies can be used to improve in-distribution
calibration, they need not also improve calibration under distribution shift.
However, techniques which produce GNNs with better intrinsic uncertainty
estimates are particularly valuable, as they can always be combined with
post-hoc strategies later. Therefore, in this work, we propose G-$\Delta$UQ, a
novel training framework designed to improve intrinsic GNN uncertainty
estimates. Our framework adapts the principle of stochastic data centering to
graph data through novel graph anchoring strategies, and is able to support
partially stochastic GNNs. While, the prevalent wisdom is that fully stochastic
networks are necessary to obtain reliable estimates, we find that the
functional diversity induced by our anchoring strategies when sampling
hypotheses renders this unnecessary and allows us to support G-$\Delta$UQ on
pretrained models. Indeed, through extensive evaluation under covariate,
concept and graph size shifts, we show that G-$\Delta$UQ leads to better
calibrated GNNs for node and graph classification. Further, it also improves
performance on the uncertainty-based tasks of out-of-distribution detection and
generalization gap estimation. Overall, our work provides insights into
uncertainty estimation for GNNs, and demonstrates the utility of G-$\Delta$UQ
in obtaining reliable estimates.
Related papers
- Positional Encoder Graph Quantile Neural Networks for Geographic Data [4.277516034244117]
We introduce the Positional Graph Quantile Neural Network (PE-GQNN), a novel method that integrates PE-GNNs, Quantile Neural Networks, and recalibration techniques in a fully nonparametric framework.
Experiments on benchmark datasets demonstrate that PE-GQNN significantly outperforms existing state-of-the-art methods in both predictive accuracy and uncertainty quantification.
arXiv Detail & Related papers (2024-09-27T16:02:12Z) - Conditional Shift-Robust Conformal Prediction for Graph Neural Network [0.0]
Graph Neural Networks (GNNs) have emerged as potent tools for predicting outcomes in graph-structured data.
Despite their efficacy, GNNs have limited ability to provide robust uncertainty estimates.
We propose Conditional Shift Robust (CondSR) conformal prediction for GNNs.
arXiv Detail & Related papers (2024-05-20T11:47:31Z) - Online GNN Evaluation Under Test-time Graph Distribution Shifts [92.4376834462224]
A new research problem, online GNN evaluation, aims to provide valuable insights into the well-trained GNNs's ability to generalize to real-world unlabeled graphs.
We develop an effective learning behavior discrepancy score, dubbed LeBeD, to estimate the test-time generalization errors of well-trained GNN models.
arXiv Detail & Related papers (2024-03-15T01:28:08Z) - Uncertainty in Graph Neural Networks: A Survey [50.63474656037679]
Graph Neural Networks (GNNs) have been extensively used in various real-world applications.
However, the predictive uncertainty of GNNs stemming from diverse sources can lead to unstable and erroneous predictions.
This survey aims to provide a comprehensive overview of the GNNs from the perspective of uncertainty.
arXiv Detail & Related papers (2024-03-11T21:54:52Z) - GNNEvaluator: Evaluating GNN Performance On Unseen Graphs Without Labels [81.93520935479984]
We study a new problem, GNN model evaluation, that aims to assess the performance of a specific GNN model trained on labeled and observed graphs.
We propose a two-stage GNN model evaluation framework, including (1) DiscGraph set construction and (2) GNNEvaluator training and inference.
Under the effective training supervision from the DiscGraph set, GNNEvaluator learns to precisely estimate node classification accuracy of the to-be-evaluated GNN model.
arXiv Detail & Related papers (2023-10-23T05:51:59Z) - Accurate and Scalable Estimation of Epistemic Uncertainty for Graph
Neural Networks [40.95782849532316]
Confidence indicators (CIs) are crucial for safe deployment of graph neural networks (GNNs) under distribution shift.
We show that increased expressivity or model size do not always lead to improved CI performance.
We propose G-$$UQ, a new single model UQ method that extends the recently proposed framework.
Overall, our work not only introduces a new, flexible GNN UQ method, but also provides novel insights into GNN CIs on safety-critical tasks.
arXiv Detail & Related papers (2023-09-20T00:35:27Z) - A Biased Graph Neural Network Sampler with Near-Optimal Regret [57.70126763759996]
Graph neural networks (GNN) have emerged as a vehicle for applying deep network architectures to graph and relational data.
In this paper, we build upon existing work and treat GNN neighbor sampling as a multi-armed bandit problem.
We introduce a newly-designed reward function that introduces some degree of bias designed to reduce variance and avoid unstable, possibly-unbounded payouts.
arXiv Detail & Related papers (2021-03-01T15:55:58Z) - Bayesian Graph Neural Networks with Adaptive Connection Sampling [62.51689735630133]
We propose a unified framework for adaptive connection sampling in graph neural networks (GNNs)
The proposed framework not only alleviates over-smoothing and over-fitting tendencies of deep GNNs, but also enables learning with uncertainty in graph analytic tasks with GNNs.
arXiv Detail & Related papers (2020-06-07T07:06:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.