When does distribution shift break graph neural networks calibration?

• URL: http://arxiv.org/abs/2607.10804v1
• Date: Sun, 12 Jul 2026 15:23:38 GMT
• Title: When does distribution shift break graph neural networks calibration?
• Abstract summary: Graph neural networks (GNNs) are increasingly deployed in real-world applications where distribution shift is un-avoidable.<n>I present the first closed-form theoretical characterization of GNN calibration under distribution shift.<n>I propose STAC, a source-free, label-free calibration method.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph neural networks (GNNs) are increasingly deployed in real-world applications where distribution shift is un-avoidable. However, how such shifts affect model calibration, defined as the agreement between predictive confidence and actual accuracy, remains poorly understood, and existing graph calibration methods typically rely on labeled validation data from the deployment distribution. In this work, I present the first closed-form theoretical characterization of GNN calibration under distribution shift. I show that calibration is governed by a single scalar quantity that explicitly depends on structural changes between the source and target graphs, as well as feature quality. This characterization precisely identifies when a model becomes over-confident, under-confident, or remains calibrated, and directly yields the optimal temperature scaling strategy. I further extend the analysis to graph convolutional networks with symmetric normalization, multi-class classification, and covariate shift, and derive a theoretical upper bound on the expected calibration error. My analysis also reveals that, under homogeneous distribution shift, a single global temperature is theoretically optimal, providing a principled explanation for why more complex node-wise recalibration methods offer no additional benefit. Building on these theoretical insights, I propose STAC, a source-free, label-free calibration method. Experiments on synthetic benchmarks demonstrate substantial calibration improvements, while evaluations on five real-world graph datasets show that reliable calibration without target labels remains challenging despite the strong predictive power of the theory.

Related papers

• The Confidence Trap: Calibration Attacks for Graph Neural Networks [30.116997325760945]
  We present a framework for studying calibration attacks on graphs.<n>We show that models with higher accuracy or trained on datasets with more classes are more susceptible under this threat model.<n>Our code is publicly available at https://github.com/CaptainCuong/Graph-Calibration-Attack.git.
  arXiv  Detail & Related papers  (2026-06-07T06:16:58Z)
• WATS: Calibrating Graph Neural Networks with Wavelet-Aware Temperature Scaling [43.23012871829196]
  We propose Wavelet-Aware Temperature Scaling (WATS), a post-hoc calibration framework that assigns node-specific temperatures based on graph wavelet features.<n>WATS harnesses the scalability and topology sensitivity of graph wavelets to refine confidence estimates, all without retraining or access to neighboring logits or predictions.
  arXiv  Detail & Related papers  (2025-06-30T12:23:57Z)
• The Final Layer Holds the Key: A Unified and Efficient GNN Calibration Framework [31.107286245419044]
  Graph Neural Networks (GNNs) have demonstrated remarkable effectiveness on graph-based tasks.<n>However, their predictive confidence is often miscalibrated, typically exhibiting under-confidence.<n>We propose a simple yet efficient graph calibration method to address this issue.
  arXiv  Detail & Related papers  (2025-05-16T15:02:17Z)
• Graph Out-of-Distribution Generalization via Causal Intervention [69.70137479660113]
  We introduce a conceptually simple yet principled approach for training robust graph neural networks (GNNs) under node-level distribution shifts. Our method resorts to a new learning objective derived from causal inference that coordinates an environment estimator and a mixture-of-expert GNN predictor. Our model can effectively enhance generalization with various types of distribution shifts and yield up to 27.4% accuracy improvement over state-of-the-arts on graph OOD generalization benchmarks.
  arXiv  Detail & Related papers  (2024-02-18T07:49:22Z)
• Graph Out-of-Distribution Generalization with Controllable Data Augmentation [51.17476258673232]
  Graph Neural Network (GNN) has demonstrated extraordinary performance in classifying graph properties. Due to the selection bias of training and testing data, distribution deviation is widespread. We propose OOD calibration to measure the distribution deviation of virtual samples.
  arXiv  Detail & Related papers  (2023-08-16T13:10:27Z)
• The Calibration Generalization Gap [15.583540869583484]
  Modern neural networks provide no strong guarantees on their calibration. It is currently unclear which factors contribute to good calibration. We propose a systematic way to study the calibration error.
  arXiv  Detail & Related papers  (2022-10-05T00:04:56Z)
• On Calibration of Graph Neural Networks for Node Classification [29.738179864433445]
  Graph neural networks learn entity and edge embeddings for tasks such as node classification and link prediction. These models achieve good performance with respect to accuracy, but the confidence scores associated with the predictions might not be calibrated. We propose a topology-aware calibration method that takes the neighboring nodes into account and yields improved calibration.
  arXiv  Detail & Related papers  (2022-06-03T13:48:10Z)
• On the Dark Side of Calibration for Modern Neural Networks [65.83956184145477]
  We show the breakdown of expected calibration error (ECE) into predicted confidence and refinement. We highlight that regularisation based calibration only focuses on naively reducing a model's confidence. We find that many calibration approaches with the likes of label smoothing, mixup etc. lower the utility of a DNN by degrading its refinement.
  arXiv  Detail & Related papers  (2021-06-17T11:04:14Z)
• Unsupervised Calibration under Covariate Shift [92.02278658443166]
  We introduce the problem of calibration under domain shift and propose an importance sampling based approach to address it. We evaluate and discuss the efficacy of our method on both real-world datasets and synthetic datasets.
  arXiv  Detail & Related papers  (2020-06-29T21:50:07Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Post-hoc Calibration of Neural Networks by g-Layers [51.42640515410253]
  In recent years, there is a surge of research on neural network calibration. It is known that minimizing Negative Log-Likelihood (NLL) will lead to a calibrated network on the training set if the global optimum is attained. We prove that even though the base network ($f$) does not lead to the global optimum of NLL, by adding additional layers ($g$) and minimizing NLL by optimizing the parameters of $g$ one can obtain a calibrated network.
  arXiv  Detail & Related papers  (2020-06-23T07:55:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.