Uncertainty Quantification via Neural Posterior Principal Components
- URL: http://arxiv.org/abs/2309.15533v2
- Date: Sat, 4 Nov 2023 16:26:34 GMT
- Title: Uncertainty Quantification via Neural Posterior Principal Components
- Authors: Elias Nehme, Omer Yair, Tomer Michaeli
- Abstract summary: Uncertainty quantification is crucial for the deployment of image restoration models in safety-critical domains.
We present a method for predicting the PCs of the posterior distribution for any input image, in a single forward pass of a neural network.
Our method reliably conveys instance-adaptive uncertainty directions, achieving uncertainty quantification comparable with posterior samplers.
- Score: 26.26693707762823
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Uncertainty quantification is crucial for the deployment of image restoration
models in safety-critical domains, like autonomous driving and biological
imaging. To date, methods for uncertainty visualization have mainly focused on
per-pixel estimates. Yet, a heatmap of per-pixel variances is typically of
little practical use, as it does not capture the strong correlations between
pixels. A more natural measure of uncertainty corresponds to the variances
along the principal components (PCs) of the posterior distribution.
Theoretically, the PCs can be computed by applying PCA on samples generated
from a conditional generative model for the input image. However, this requires
generating a very large number of samples at test time, which is painfully slow
with the current state-of-the-art (diffusion) models. In this work, we present
a method for predicting the PCs of the posterior distribution for any input
image, in a single forward pass of a neural network. Our method can either wrap
around a pre-trained model that was trained to minimize the mean square error
(MSE), or can be trained from scratch to output both a predicted image and the
posterior PCs. We showcase our method on multiple inverse problems in imaging,
including denoising, inpainting, super-resolution, and biological
image-to-image translation. Our method reliably conveys instance-adaptive
uncertainty directions, achieving uncertainty quantification comparable with
posterior samplers while being orders of magnitude faster. Code and examples
are available at https://eliasnehme.github.io/NPPC/
Related papers
- Variational Bayes image restoration with compressive autoencoders [5.514022022280169]
Regularization of inverse problems is of paramount importance in computational imaging.
State-of-the-art plug-and-play methods rely on an implicit regularization provided by neural denoisers.
We introduce the Variational Bayes Latent Estimation (VBLE) algorithm, which performs latent estimation.
arXiv Detail & Related papers (2023-11-29T15:49:31Z) - Single Image Depth Prediction Made Better: A Multivariate Gaussian Take [163.14849753700682]
We introduce an approach that performs continuous modeling of per-pixel depth.
Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.
arXiv Detail & Related papers (2023-03-31T16:01:03Z) - Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness.
We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
arXiv Detail & Related papers (2023-03-06T11:51:28Z) - Image-to-Image Regression with Distribution-Free Uncertainty
Quantification and Applications in Imaging [88.20869695803631]
We show how to derive uncertainty intervals around each pixel that are guaranteed to contain the true value.
We evaluate our procedure on three image-to-image regression tasks.
arXiv Detail & Related papers (2022-02-10T18:59:56Z) - Deblurring via Stochastic Refinement [85.42730934561101]
We present an alternative framework for blind deblurring based on conditional diffusion models.
Our method is competitive in terms of distortion metrics such as PSNR.
arXiv Detail & Related papers (2021-12-05T04:36:09Z) - Robustness via Uncertainty-aware Cycle Consistency [44.34422859532988]
Unpaired image-to-image translation refers to learning inter-image-domain mapping without corresponding image pairs.
Existing methods learn deterministic mappings without explicitly modelling the robustness to outliers or predictive uncertainty.
We propose a novel probabilistic method based on Uncertainty-aware Generalized Adaptive Cycle Consistency (UGAC)
arXiv Detail & Related papers (2021-10-24T15:33:21Z) - Training on Test Data with Bayesian Adaptation for Covariate Shift [96.3250517412545]
Deep neural networks often make inaccurate predictions with unreliable uncertainty estimates.
We derive a Bayesian model that provides for a well-defined relationship between unlabeled inputs under distributional shift and model parameters.
We show that our method improves both accuracy and uncertainty estimation.
arXiv Detail & Related papers (2021-09-27T01:09:08Z) - Uncertainty-aware Generalized Adaptive CycleGAN [44.34422859532988]
Unpaired image-to-image translation refers to learning inter-image-domain mapping in an unsupervised manner.
Existing methods often learn deterministic mappings without explicitly modelling the robustness to outliers or predictive uncertainty.
We propose a novel probabilistic method called Uncertainty-aware Generalized Adaptive Cycle Consistency (UGAC)
arXiv Detail & Related papers (2021-02-23T15:22:35Z) - Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal
Solution Characterization for Computational Imaging [11.677576854233394]
We propose a variational deep probabilistic imaging approach to quantify reconstruction uncertainty.
Deep Probabilistic Imaging employs an untrained deep generative model to estimate a posterior distribution of an unobserved image.
arXiv Detail & Related papers (2020-10-27T17:23:09Z) - RAIN: A Simple Approach for Robust and Accurate Image Classification
Networks [156.09526491791772]
It has been shown that the majority of existing adversarial defense methods achieve robustness at the cost of sacrificing prediction accuracy.
This paper proposes a novel preprocessing framework, which we term Robust and Accurate Image classificatioN(RAIN)
RAIN applies randomization over inputs to break the ties between the model forward prediction path and the backward gradient path, thus improving the model robustness.
We conduct extensive experiments on the STL10 and ImageNet datasets to verify the effectiveness of RAIN against various types of adversarial attacks.
arXiv Detail & Related papers (2020-04-24T02:03:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.