Do Deep Ensembles Actually Capture Uncertainty in Graph Neural Networks?

• URL: http://arxiv.org/abs/2605.22593v1
• Date: Thu, 21 May 2026 15:07:47 GMT
• Title: Do Deep Ensembles Actually Capture Uncertainty in Graph Neural Networks?
• Abstract summary: We investigate standard deep ensembles specifically for message-passing graph neural networks.<n>We find that ensembles provide surprisingly little improvement over a single model.<n>Our results suggest that deep ensemble success does not seamlessly transfer to graph machine learning.
• Score: 4.034187803388112
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While deep ensembles are widely considered to be the default method for uncertainty quantification in deep learning, their effectiveness for graph-structured data is often simply assumed based on successes in domains like computer vision. We investigate standard deep ensembles specifically for message-passing graph neural networks. Benchmarking across seven datasets representing varied tasks and complexities, we reveal that ensembles provide surprisingly little improvement over a single model. Instead, the observed marginal gains stem primarily from stabilizing optimization noise in point predictions rather than yielding meaningfully better uncertainty estimates. Through an aleatoric-epistemic decomposition, we identify epistemic collapse: independently trained networks consistently converge to overly similar predictions. Because disagreement is the fundamental mechanism through which ensembles capture epistemic uncertainty, this lack of diversity neutralizes their key advantage. Analyzing this phenomenon further, we suggest this collapse is driven by functional rather than weight-space convexity, where distinct parameter solutions induce almost identical behavior. Our results suggest that deep ensemble success does not seamlessly transfer to graph machine learning.

Related papers

• Variational Graph Neural Networks for Uncertainty Quantification in Inverse Problems [0.5599792629509229]
  We present a variational graph neural network (VGNN) architecture that integrates variational layers into its architecture to model the probability distribution of weights.<n>Unlike computationally expensive full Bayesian networks, our approach strategically introduces variational layers exclusively in the decoder, allowing us to estimate cognitive uncertainty and statistical uncertainty at a relatively lower cost.<n>The results show that the model not only recovers the physical parameters with high precision, but also provides confidence intervals consistent with the physics of the problem, as well as being able to locate the position of the applied load and estimate its value.
  arXiv  Detail & Related papers  (2026-03-31T09:57:52Z)
• Elimination-compensation pruning for fully-connected neural networks [2.785400823678854]
  Pruning techniques are used to extract sparse representations of neural networks parameters.<n>We introduce a novel method in which the importance measure of each weight is computed considering the output behavior.<n>Our findings are discussed and the theoretical implications of our results are presented.
  arXiv  Detail & Related papers  (2026-02-24T01:56:12Z)
• Quantification of Uncertainties in Probabilistic Deep Neural Network by Implementing Boosting of Variational Inference [0.38366697175402226]
  Boosted Bayesian Neural Networks (BBNN) is a novel approach that enhances neural network weight distribution approximations.<n>BBNN achieves 5% higher accuracy compared to conventional neural networks.
  arXiv  Detail & Related papers  (2025-03-18T05:11:21Z)
• Contextual Similarity Distillation: Ensemble Uncertainties with a Single Model [5.624791703748109]
  Uncertainty quantification is a critical aspect of reinforcement learning and deep learning.<n>We propose contextual similarity distillation, a novel approach that explicitly estimates the variance of an ensemble of deep neural networks with a single model.<n>We empirically validate our method across a variety of out-of-distribution detection benchmarks and sparse-reward reinforcement learning environments.
  arXiv  Detail & Related papers  (2025-03-14T12:09:58Z)
• Structured Radial Basis Function Network: Modelling Diversity for Multiple Hypotheses Prediction [51.82628081279621]
  Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions. A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems. It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
  arXiv  Detail & Related papers  (2023-09-02T01:27:53Z)
• Learning Linear Causal Representations from Interventions under General Nonlinear Mixing [52.66151568785088]
  We prove strong identifiability results given unknown single-node interventions without access to the intervention targets. This is the first instance of causal identifiability from non-paired interventions for deep neural network embeddings.
  arXiv  Detail & Related papers  (2023-06-04T02:32:12Z)
• Toward Robust Uncertainty Estimation with Random Activation Functions [3.0586855806896045]
  We propose a novel approach for uncertainty quantification via ensembles, called Random Activation Functions (RAFs) Ensemble. RAFs Ensemble outperforms state-of-the-art ensemble uncertainty quantification methods on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-02-28T13:17:56Z)
• Pathologies of Predictive Diversity in Deep Ensembles [29.893614175153235]
  Classic results establish that encouraging predictive diversity improves performance in ensembles of low-capacity models. Here we demonstrate that these intuitions do not apply to high-capacity neural network ensembles (deep ensembles)
  arXiv  Detail & Related papers  (2023-02-01T19:01:18Z)
• Generalization of Neural Combinatorial Solvers Through the Lens of Adversarial Robustness [68.97830259849086]
  Most datasets only capture a simpler subproblem and likely suffer from spurious features. We study adversarial robustness - a local generalization property - to reveal hard, model-specific instances and spurious features. Unlike in other applications, where perturbation models are designed around subjective notions of imperceptibility, our perturbation models are efficient and sound. Surprisingly, with such perturbations, a sufficiently expressive neural solver does not suffer from the limitations of the accuracy-robustness trade-off common in supervised learning.
  arXiv  Detail & Related papers  (2021-10-21T07:28:11Z)
• Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
  Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions. We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
  arXiv  Detail & Related papers  (2021-02-22T07:02:37Z)
• Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
  In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
  arXiv  Detail & Related papers  (2020-03-10T03:10:41Z)
• Bayesian Deep Learning and a Probabilistic Perspective of Generalization [56.69671152009899]
  We show that deep ensembles provide an effective mechanism for approximate Bayesian marginalization. We also propose a related approach that further improves the predictive distribution by marginalizing within basins of attraction.
  arXiv  Detail & Related papers  (2020-02-20T15:13:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.