Efficient Bayesian Physics Informed Neural Networks for Inverse Problems via Ensemble Kalman Inversion

• URL: http://arxiv.org/abs/2303.07392v1
• Date: Mon, 13 Mar 2023 18:15:26 GMT
• Title: Efficient Bayesian Physics Informed Neural Networks for Inverse Problems via Ensemble Kalman Inversion
• Authors: Andrew Pensoneault and Xueyu Zhu
• Abstract summary: We present a new efficient inference algorithm for B-PINNs that uses Ensemble Kalman Inversion (EKI) for high-dimensional inference tasks. We find that our proposed method can achieve inference results with informative uncertainty estimates comparable to Hamiltonian Monte Carlo (HMC)-based B-PINNs with a much reduced computational cost.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Bayesian Physics Informed Neural Networks (B-PINNs) have gained significant attention for inferring physical parameters and learning the forward solutions for problems based on partial differential equations. However, the overparameterized nature of neural networks poses a computational challenge for high-dimensional posterior inference. Existing inference approaches, such as particle-based or variance inference methods, are either computationally expensive for high-dimensional posterior inference or provide unsatisfactory uncertainty estimates. In this paper, we present a new efficient inference algorithm for B-PINNs that uses Ensemble Kalman Inversion (EKI) for high-dimensional inference tasks. We find that our proposed method can achieve inference results with informative uncertainty estimates comparable to Hamiltonian Monte Carlo (HMC)-based B-PINNs with a much reduced computational cost. These findings suggest that our proposed approach has great potential for uncertainty quantification in physics-informed machine learning for practical applications.

Related papers

• Improvement of Bayesian PINN Training Convergence in Solving Multi-scale PDEs with Noise [34.11898314129823]
  In practice, Hamiltonian Monte Carlo (HMC) used to estimate the internal parameters of BPINN often encounters troubles. We develop a robust multi-scale Bayesian PINN (dubbed MBPINN) method by integrating multi-scale neural networks (MscaleDNN) and Bayesian inference. Our findings indicate that the proposed method can avoid HMC failures and provide valid results.
  arXiv  Detail & Related papers  (2024-08-18T03:20:16Z)
• Physics-Informed Neural Networks for an optimal counterdiabatic quantum computation [32.73124984242397]
  We introduce a novel methodology that leverages the strength of Physics-Informed Neural Networks (PINNs) to address the counterdiabatic (CD) protocol in the optimization of quantum circuits comprised of systems with $N_Q$ qubits. The main applications of this methodology have been the $mathrmH_2$ and $mathrmLiH$ molecules, represented by a 2-qubit and 4-qubit systems employing the STO-3G basis.
  arXiv  Detail & Related papers  (2023-09-08T16:55:39Z)
• Learning Active Subspaces for Effective and Scalable Uncertainty Quantification in Deep Neural Networks [13.388835540131508]
  We propose a novel scheme for constructing a low-dimensional subspace of the neural network parameters. We demonstrate that the significantly reduced active subspace enables effective and scalable Bayesian inference. Our approach provides reliable predictions with robust uncertainty estimates for various regression tasks.
  arXiv  Detail & Related papers  (2023-09-06T15:00:36Z)
• Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression [37.69303106863453]
  The Information Bottleneck (IB) principle offers an information-theoretic framework for analyzing the training process of deep neural networks (DNNs) In this paper, we introduce a framework for conducting IB analysis of general NNs. We also perform IB analysis on a close-to-real-scale, which reveals new features of the MI dynamics.
  arXiv  Detail & Related papers  (2023-05-13T21:44:32Z)
• Efficient Bayesian inference using physics-informed invertible neural networks for inverse problems [6.97393424359704]
  We introduce an innovative approach for addressing Bayesian inverse problems through the utilization of physics-informed invertible neural networks (PI-INN) The PI-INN offers a precise and efficient generative model for Bayesian inverse problems, yielding tractable posterior density estimates. As a particular physics-informed deep learning model, the primary training challenge for PI-INN centers on enforcing the independence constraint.
  arXiv  Detail & Related papers  (2023-04-25T03:17:54Z)
• Bayesian deep learning framework for uncertainty quantification in high dimensions [6.282068591820945]
  We develop a novel deep learning method for uncertainty quantification in partial differential equations based on Bayesian neural network (BNN) and Hamiltonian Monte Carlo (HMC) A BNN efficiently learns the posterior distribution of the parameters in deep neural networks by performing Bayesian inference on the network parameters. The posterior distribution is efficiently sampled using HMC to quantify uncertainties in the system.
  arXiv  Detail & Related papers  (2022-10-21T05:20:06Z)
• Single Model Uncertainty Estimation via Stochastic Data Centering [39.71621297447397]
  We are interested in estimating the uncertainties of deep neural networks. We present a striking new finding that an ensemble of neural networks with the same weight initialization, trained on datasets that are shifted by a constant bias gives rise to slightly inconsistent trained models. We show that $Delta-$UQ's uncertainty estimates are superior to many of the current methods on a variety of benchmarks.
  arXiv  Detail & Related papers  (2022-07-14T23:54:54Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• Robust Learning of Physics Informed Neural Networks [2.86989372262348]
  Physics-informed Neural Networks (PINNs) have been shown to be effective in solving partial differential equations. This paper shows that a PINN can be sensitive to errors in training data and overfit itself in dynamically propagating these errors over the domain of the solution of the PDE.
  arXiv  Detail & Related papers  (2021-10-26T00:10:57Z)
• NTopo: Mesh-free Topology Optimization using Implicit Neural Representations [35.07884509198916]
  We present a novel machine learning approach to tackle topology optimization problems. We use multilayer perceptrons (MLPs) to parameterize both density and displacement fields. As we show through our experiments, a major benefit of our approach is that it enables self-supervised learning of continuous solution spaces.
  arXiv  Detail & Related papers  (2021-02-22T05:25:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.