Reliable amortized variational inference with physics-based latent distribution correction

• URL: http://arxiv.org/abs/2207.11640v1
• Date: Sun, 24 Jul 2022 02:38:54 GMT
• Title: Reliable amortized variational inference with physics-based latent distribution correction
• Authors: Ali Siahkoohi and Gabrio Rizzuti and Rafael Orozco and Felix J. Herrmann
• Abstract summary: A neural network is trained to approximate the posterior distribution over existing pairs of model and data. The accuracy of this approach relies on the availability of high-fidelity training data. We show that our correction step improves the robustness of amortized variational inference with respect to changes in number of source experiments, noise variance, and shifts in the prior distribution.
• Score: 0.4588028371034407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bayesian inference for high-dimensional inverse problems is challenged by the computational costs of the forward operator and the selection of an appropriate prior distribution. Amortized variational inference addresses these challenges where a neural network is trained to approximate the posterior distribution over existing pairs of model and data. When fed previously unseen data and normally distributed latent samples as input, the pretrained deep neural network -- in our case a conditional normalizing flow -- provides posterior samples with virtually no cost. However, the accuracy of this approach relies on the availability of high-fidelity training data, which seldom exists in geophysical inverse problems due to the heterogeneous structure of the Earth. In addition, accurate amortized variational inference requires the observed data to be drawn from the training data distribution. As such, we propose to increase the resilience of amortized variational inference when faced with data distribution shift via a physics-based correction to the conditional normalizing flow latent distribution. To accomplish this, instead of a standard Gaussian latent distribution, we parameterize the latent distribution by a Gaussian distribution with an unknown mean and diagonal covariance. These unknown quantities are then estimated by minimizing the Kullback-Leibler divergence between the corrected and true posterior distributions. While generic and applicable to other inverse problems, by means of a seismic imaging example, we show that our correction step improves the robustness of amortized variational inference with respect to changes in number of source experiments, noise variance, and shifts in the prior distribution. This approach provides a seismic image with limited artifacts and an assessment of its uncertainty with approximately the same cost as five reverse-time migrations.

Related papers

• Stochastic full waveform inversion with deep generative prior for uncertainty quantification [0.0]
  Full Waveform Inversion (FWI) involves solving a nonlinear and often non-unique inverse problem. FWI presents challenges such as local minima trapping and inadequate handling of inherent uncertainties. We propose leveraging deep generative models as the prior distribution of geophysical parameters for Bayesian inversion.
  arXiv  Detail & Related papers  (2024-06-07T11:44:50Z)
• Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
  We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data. Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
  arXiv  Detail & Related papers  (2024-04-24T09:04:36Z)
• Uncertainty Quantification via Stable Distribution Propagation [60.065272548502]
  We propose a new approach for propagating stable probability distributions through neural networks. Our method is based on local linearization, which we show to be an optimal approximation in terms of total variation distance for the ReLU non-linearity.
  arXiv  Detail & Related papers  (2024-02-13T09:40:19Z)
• Distributed Variational Inference for Online Supervised Learning [15.038649101409804]
  This paper develops a scalable distributed probabilistic inference algorithm. It applies to continuous variables, intractable posteriors and large-scale real-time data in sensor networks.
  arXiv  Detail & Related papers  (2023-09-05T22:33:02Z)
• On the detrimental effect of invariances in the likelihood for variational inference [21.912271882110986]
  Variational Bayesian posterior inference often requires simplifying approximations such as mean-field parametrisation to ensure tractability. Prior work has associated the variational mean-field approximation for Bayesian neural networks with underfitting in the case of small datasets or large model sizes.
  arXiv  Detail & Related papers  (2022-09-15T09:13:30Z)
• Certifying Model Accuracy under Distribution Shifts [151.67113334248464]
  We present provable robustness guarantees on the accuracy of a model under bounded Wasserstein shifts of the data distribution. We show that a simple procedure that randomizes the input of the model within a transformation space is provably robust to distributional shifts under the transformation.
  arXiv  Detail & Related papers  (2022-01-28T22:03:50Z)
• Training on Test Data with Bayesian Adaptation for Covariate Shift [96.3250517412545]
  Deep neural networks often make inaccurate predictions with unreliable uncertainty estimates. We derive a Bayesian model that provides for a well-defined relationship between unlabeled inputs under distributional shift and model parameters. We show that our method improves both accuracy and uncertainty estimation.
  arXiv  Detail & Related papers  (2021-09-27T01:09:08Z)
• Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise [51.080620762639434]
  We propose a more efficient parameterization of the posterior approximation for sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks.
  arXiv  Detail & Related papers  (2021-03-15T16:16:18Z)
• Empirical Strategy for Stretching Probability Distribution in Neural-network-based Regression [5.35308390309106]
  In regression analysis under artificial neural networks, the prediction performance depends on determining the appropriate weights between layers. We proposed weighted empirical stretching (WES) as a novel loss function to increase the overlap area of the two distributions. The improved results in RMSE for the extreme domain are expected to be utilized for prediction of abnormal events in non-linear complex systems.
  arXiv  Detail & Related papers  (2020-09-08T06:08:14Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.