Uncertainty Modeling in Graph Neural Networks via Stochastic Differential Equations

• URL: http://arxiv.org/abs/2408.16115v4
• Date: Tue, 25 Feb 2025 16:34:08 GMT
• Title: Uncertainty Modeling in Graph Neural Networks via Stochastic Differential Equations
• Authors: Richard Bergna, Sergio Calvo-Ordoñez, Felix L. Opolka, Pietro Liò, Jose Miguel Hernandez-Lobato,
• Abstract summary: We introduce Latent Graph Neural Differential Equations (LGNSDE) to address the problem of learning uncertainty-aware representations for graph-structured data.<n>We prove that the variance of the latent space bounds the variance of model outputs, thereby providing theoretically sensible guarantees for the uncertainty estimates.<n>Our framework is competitive in out-of-distribution detection, robustness to noise, and active learning, underscoring the ability of LGNSDEs to quantify uncertainty reliably.
• Score: 14.422150854883453
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel Stochastic Differential Equation (SDE) framework to address the problem of learning uncertainty-aware representations for graph-structured data. While Graph Neural Ordinary Differential Equations (GNODEs) have shown promise in learning node representations, they lack the ability to quantify uncertainty. To address this, we introduce Latent Graph Neural Stochastic Differential Equations (LGNSDE), which enhance GNODE by embedding randomness through a Bayesian prior-posterior mechanism for epistemic uncertainty and Brownian motion for aleatoric uncertainty. By leveraging the existence and uniqueness of solutions to graph-based SDEs, we prove that the variance of the latent space bounds the variance of model outputs, thereby providing theoretically sensible guarantees for the uncertainty estimates. Furthermore, we show mathematically that LGNSDEs are robust to small perturbations in the input, maintaining stability over time. Empirical results across several benchmarks demonstrate that our framework is competitive in out-of-distribution detection, robustness to noise, and active learning, underscoring the ability of LGNSDEs to quantify uncertainty reliably.

Related papers

• Evidential Physics-Informed Neural Networks [0.0]
  We present a novel class of Physics-Informed Neural Networks that is formulated based on the principles of Evidential Deep Learning. We show how to apply our model to inverse problems involving 1D and 2D nonlinear differential equations.
  arXiv  Detail & Related papers  (2025-01-27T10:01:10Z)
• Learning Latent Graph Structures and their Uncertainty [63.95971478893842]
  Graph Neural Networks (GNNs) use relational information as an inductive bias to enhance the model's accuracy. As task-relevant relations might be unknown, graph structure learning approaches have been proposed to learn them while solving the downstream prediction task.
  arXiv  Detail & Related papers  (2024-05-30T10:49:22Z)
• Uncertainty in Graph Neural Networks: A Survey [50.63474656037679]
  Graph Neural Networks (GNNs) have been extensively used in various real-world applications. However, the predictive uncertainty of GNNs stemming from diverse sources can lead to unstable and erroneous predictions. This survey aims to provide a comprehensive overview of the GNNs from the perspective of uncertainty.
  arXiv  Detail & Related papers  (2024-03-11T21:54:52Z)
• Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data [3.686808512438363]
  Irregular sampling intervals and missing values in real-world time series data present challenges for conventional methods. We propose three stable classes of Neural SDEs: Langevin-type SDE, Linear Noise SDE, and Geometric SDE. Our results demonstrate the efficacy of the proposed method in handling real-world irregular time series data.
  arXiv  Detail & Related papers  (2024-02-22T22:00:03Z)
• Towards stable real-world equation discovery with assessing differentiating quality influence [52.2980614912553]
  We propose alternatives to the commonly used finite differences-based method. We evaluate these methods in terms of applicability to problems, similar to the real ones, and their ability to ensure the convergence of equation discovery algorithms.
  arXiv  Detail & Related papers  (2023-11-09T23:32:06Z)
• Graph Neural Stochastic Differential Equations [3.568455515949288]
  We present a novel model Graph Neural Differential Equations (Graph Neural SDEs) This technique enhances the Graph Neural Ordinary Differential Equations (Graph Neural ODEs) by embedding randomness into data representation using Brownian motion. We find that Latent Graph Neural SDEs surpass conventional models like Graph Convolutional Networks and Graph Neural ODEs, especially in confidence prediction.
  arXiv  Detail & Related papers  (2023-08-23T09:20:38Z)
• The Unreasonable Effectiveness of Deep Evidential Regression [72.30888739450343]
  A new approach with uncertainty-aware regression-based neural networks (NNs) shows promise over traditional deterministic methods and typical Bayesian NNs. We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a quantification rather than an exact uncertainty.
  arXiv  Detail & Related papers  (2022-05-20T10:10:32Z)
• A General Framework for quantifying Aleatoric and Epistemic uncertainty in Graph Neural Networks [0.29494468099506893]
  Graph Neural Networks (GNN) provide a powerful framework that elegantly integrates Graph theory with Machine learning. We consider the problem of quantifying the uncertainty in predictions of GNN stemming from modeling errors and measurement uncertainty. We propose a unified approach to treat both sources of uncertainty in a Bayesian framework.
  arXiv  Detail & Related papers  (2022-05-20T05:25:40Z)
• NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural Networks [151.03112356092575]
  We show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution. We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets.
  arXiv  Detail & Related papers  (2022-02-07T12:30:45Z)
• Distributionally Robust Semi-Supervised Learning Over Graphs [68.29280230284712]
  Semi-supervised learning (SSL) over graph-structured data emerges in many network science applications. To efficiently manage learning over graphs, variants of graph neural networks (GNNs) have been developed recently. Despite their success in practice, most of existing methods are unable to handle graphs with uncertain nodal attributes. Challenges also arise due to distributional uncertainties associated with data acquired by noisy measurements. A distributionally robust learning framework is developed, where the objective is to train models that exhibit quantifiable robustness against perturbations.
  arXiv  Detail & Related papers  (2021-10-20T14:23:54Z)
• Stability of Graph Convolutional Neural Networks to Stochastic Perturbations [122.12962842842349]
  Graph convolutional neural networks (GCNNs) are nonlinear processing tools to learn representations from network data. Current analysis considers deterministic perturbations but fails to provide relevant insights when topological changes are random. This paper investigates the stability of GCNNs to perturbed graph perturbations induced by link losses.
  arXiv  Detail & Related papers  (2021-06-19T16:25:28Z)
• Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Features [0.0]
  We present a novel, domain-agnostic approach to tackling this problem. We use compositions of physics-informed random features, derived from ordinary differential equations. We find that our approach achieves comparable performance to a number of other probabilistic models on benchmark regression tasks.
  arXiv  Detail & Related papers  (2021-06-10T17:55:13Z)
• BayesIMP: Uncertainty Quantification for Causal Data Fusion [52.184885680729224]
  We study the causal data fusion problem, where datasets pertaining to multiple causal graphs are combined to estimate the average treatment effect of a target variable. We introduce a framework which combines ideas from probabilistic integration and kernel mean embeddings to represent interventional distributions in the reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2021-06-07T10:14:18Z)
• Accurate and Reliable Forecasting using Stochastic Differential Equations [48.21369419647511]
  It is critical yet challenging for deep learning models to properly characterize uncertainty that is pervasive in real-world environments. This paper develops SDE-HNN to characterize the interaction between the predictive mean and variance of HNNs for accurate and reliable regression. Experiments on the challenging datasets show that our method significantly outperforms the state-of-the-art baselines in terms of both predictive performance and uncertainty quantification.
  arXiv  Detail & Related papers  (2021-03-28T04:18:11Z)
• Implicit Graph Neural Networks [46.0589136729616]
  We propose a graph learning framework called Implicit Graph Neural Networks (IGNN) IGNNs consistently capture long-range dependencies and outperform state-of-the-art GNN models.
  arXiv  Detail & Related papers  (2020-09-14T06:04:55Z)
• Stochastic Graph Recurrent Neural Network [6.656993023468793]
  We propose SGRNN, a novel neural architecture that applies latent variables to simultaneously capture evolution in node attributes and topology. Specifically, deterministic states are separated from states in the iterative process to suppress mutual interference. Experiments on real-world datasets demonstrate the effectiveness of the proposed model.
  arXiv  Detail & Related papers  (2020-09-01T16:14:30Z)
• SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates [45.43024126674237]
  Uncertainty quantification is a fundamental yet unsolved problem for deep learning. Bayesian framework provides principled way of uncertainty estimation but is often not scalable to modern deep neural nets (DNNs) We propose a new method for quantifying uncertainties of DNNs from a dynamical system perspective.
  arXiv  Detail & Related papers  (2020-08-24T16:33:54Z)
• Stochastic Graph Neural Networks [123.39024384275054]
  Graph neural networks (GNNs) model nonlinear representations in graph data with applications in distributed agent coordination, control, and planning. Current GNN architectures assume ideal scenarios and ignore link fluctuations that occur due to environment, human factors, or external attacks. In these situations, the GNN fails to address its distributed task if the topological randomness is not considered accordingly.
  arXiv  Detail & Related papers  (2020-06-04T08:00:00Z)
• Stochasticity in Neural ODEs: An Empirical Study [68.8204255655161]
  Regularization of neural networks (e.g. dropout) is a widespread technique in deep learning that allows for better generalization. We show that data augmentation during the training improves the performance of both deterministic and versions of the same model. However, the improvements obtained by the data augmentation completely eliminate the empirical regularization gains, making the performance of neural ODE and neural SDE negligible.
  arXiv  Detail & Related papers  (2020-02-22T22:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.