Variational Bayes Deep Operator Network: A data-driven Bayesian solver for parametric differential equations

• URL: http://arxiv.org/abs/2206.05655v1
• Date: Sun, 12 Jun 2022 04:20:11 GMT
• Title: Variational Bayes Deep Operator Network: A data-driven Bayesian solver for parametric differential equations
• Authors: Shailesh Garg and Souvik Chakraborty
• Abstract summary: We propose Variational Bayes DeepONet (VB-DeepONet) for operator learning. VB-DeepONet uses variational inference to take into account high dimensional posterior distributions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Neural network based data-driven operator learning schemes have shown tremendous potential in computational mechanics. DeepONet is one such neural network architecture which has gained widespread appreciation owing to its excellent prediction capabilities. Having said that, being set in a deterministic framework exposes DeepONet architecture to the risk of overfitting, poor generalization and in its unaltered form, it is incapable of quantifying the uncertainties associated with its predictions. We propose in this paper, a Variational Bayes DeepONet (VB-DeepONet) for operator learning, which can alleviate these limitations of DeepONet architecture to a great extent and give user additional information regarding the associated uncertainty at the prediction stage. The key idea behind neural networks set in Bayesian framework is that, the weights and bias of the neural network are treated as probability distributions instead of point estimates and, Bayesian inference is used to update their prior distribution. Now, to manage the computational cost associated with approximating the posterior distribution, the proposed VB-DeepONet uses \textit{variational inference}. Unlike Markov Chain Monte Carlo schemes, variational inference has the capacity to take into account high dimensional posterior distributions while keeping the associated computational cost low. Different examples covering mechanics problems like diffusion reaction, gravity pendulum, advection diffusion have been shown to illustrate the performance of the proposed VB-DeepONet and comparisons have also been drawn against DeepONet set in deterministic framework.

Related papers

• Variational Bayesian Bow tie Neural Networks with Shrinkage [0.276240219662896]
  We build a relaxed version of the standard feed-forward rectified neural network. We employ Polya-Gamma data augmentation tricks to render a conditionally linear and Gaussian model. We derive a variational inference algorithm that avoids distributional assumptions and independence across layers.
  arXiv  Detail & Related papers  (2024-11-17T17:36:30Z)
• Alpha-VI DeepONet: A prior-robust variational Bayesian approach for enhancing DeepONets with uncertainty quantification [0.0]
  We introduce a novel deep operator network (DeepONet) framework that incorporates generalised variational inference (GVI) By incorporating Bayesian neural networks as the building blocks for the branch and trunk networks, our framework endows DeepONet with uncertainty quantification. We demonstrate that modifying the variational objective function yields superior results in terms of minimising the mean squared error.
  arXiv  Detail & Related papers  (2024-08-01T16:22:03Z)
• Tractable Function-Space Variational Inference in Bayesian Neural Networks [72.97620734290139]
  A popular approach for estimating the predictive uncertainty of neural networks is to define a prior distribution over the network parameters. We propose a scalable function-space variational inference method that allows incorporating prior information. We show that the proposed method leads to state-of-the-art uncertainty estimation and predictive performance on a range of prediction tasks.
  arXiv  Detail & Related papers  (2023-12-28T18:33:26Z)
• Probabilistic electric load forecasting through Bayesian Mixture Density Networks [70.50488907591463]
  Probabilistic load forecasting (PLF) is a key component in the extended tool-chain required for efficient management of smart energy grids. We propose a novel PLF approach, framed on Bayesian Mixture Density Networks. To achieve reliable and computationally scalable estimators of the posterior distributions, both Mean Field variational inference and deep ensembles are integrated.
  arXiv  Detail & Related papers  (2020-12-23T16:21:34Z)
• Efficient Variational Inference for Sparse Deep Learning with Theoretical Guarantee [20.294908538266867]
  Sparse deep learning aims to address the challenge of huge storage consumption by deep neural networks. In this paper, we train sparse deep neural networks with a fully Bayesian treatment under spike-and-slab priors. We develop a set of computationally efficient variational inferences via continuous relaxation of Bernoulli distribution.
  arXiv  Detail & Related papers  (2020-11-15T03:27:54Z)
• Ramifications of Approximate Posterior Inference for Bayesian Deep Learning in Adversarial and Out-of-Distribution Settings [7.476901945542385]
  We show that Bayesian deep learning models on certain occasions marginally outperform conventional neural networks. Preliminary investigations indicate the potential inherent role of bias due to choices of initialisation, architecture or activation functions.
  arXiv  Detail & Related papers  (2020-09-03T16:58:15Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Depth Uncertainty in Neural Networks [2.6763498831034043]
  Existing methods for estimating uncertainty in deep learning tend to require multiple forward passes. By exploiting the sequential structure of feed-forward networks, we are able to both evaluate our training objective and make predictions with a single forward pass. We validate our approach on real-world regression and image classification tasks.
  arXiv  Detail & Related papers  (2020-06-15T14:33:40Z)
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network [66.26231423824089]
  We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass. We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models.
  arXiv  Detail & Related papers  (2020-03-04T12:27:36Z)
• Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks [65.24701908364383]
  We show that a sufficient condition for a uncertainty on a ReLU network is "to be a bit Bayesian calibrated" We further validate these findings empirically via various standard experiments using common deep ReLU networks and Laplace approximations.
  arXiv  Detail & Related papers  (2020-02-24T08:52:06Z)
• Bayesian Deep Learning and a Probabilistic Perspective of Generalization [56.69671152009899]
  We show that deep ensembles provide an effective mechanism for approximate Bayesian marginalization. We also propose a related approach that further improves the predictive distribution by marginalizing within basins of attraction.
  arXiv  Detail & Related papers  (2020-02-20T15:13:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.