Score-Based Diffusion Models as Principled Priors for Inverse Imaging
- URL: http://arxiv.org/abs/2304.11751v2
- Date: Mon, 28 Aug 2023 22:38:31 GMT
- Title: Score-Based Diffusion Models as Principled Priors for Inverse Imaging
- Authors: Berthy T. Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang,
Katherine L. Bouman, William T. Freeman
- Abstract summary: We propose turning score-based diffusion models into principled image priors.
We show how to sample from resulting posteriors by using this probability function for variational inference.
- Score: 46.19536250098105
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Priors are essential for reconstructing images from noisy and/or incomplete
measurements. The choice of the prior determines both the quality and
uncertainty of recovered images. We propose turning score-based diffusion
models into principled image priors ("score-based priors") for analyzing a
posterior of images given measurements. Previously, probabilistic priors were
limited to handcrafted regularizers and simple distributions. In this work, we
empirically validate the theoretically-proven probability function of a
score-based diffusion model. We show how to sample from resulting posteriors by
using this probability function for variational inference. Our results,
including experiments on denoising, deblurring, and interferometric imaging,
suggest that score-based priors enable principled inference with a
sophisticated, data-driven image prior.
Related papers
- Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problems [12.5216516851131]
We study out of distribution (OOD) problems where a known training distribution is first provided.
We use a patch-based diffusion prior that learns the image distribution solely from patches.
In both settings, the patch-based method can obtain high quality image reconstructions that can outperform whole-image models.
arXiv Detail & Related papers (2024-10-15T16:02:08Z) - Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstruction [31.503662384666274]
In science and engineering, the goal is to infer an unknown image from a small number of measurements collected from a known forward model describing certain imaging modality.
Motivated Score-based diffusion models, due to its empirical success, have emerged as an impressive candidate of an exemplary prior in image reconstruction.
arXiv Detail & Related papers (2024-03-25T15:58:26Z) - Projection Regret: Reducing Background Bias for Novelty Detection via
Diffusion Models [72.07462371883501]
We propose emphProjection Regret (PR), an efficient novelty detection method that mitigates the bias of non-semantic information.
PR computes the perceptual distance between the test image and its diffusion-based projection to detect abnormality.
Extensive experiments demonstrate that PR outperforms the prior art of generative-model-based novelty detection methods by a significant margin.
arXiv Detail & Related papers (2023-12-05T09:44:47Z) - Exploiting Diffusion Prior for Generalizable Dense Prediction [85.4563592053464]
Recent advanced Text-to-Image (T2I) diffusion models are sometimes too imaginative for existing off-the-shelf dense predictors to estimate.
We introduce DMP, a pipeline utilizing pre-trained T2I models as a prior for dense prediction tasks.
Despite limited-domain training data, the approach yields faithful estimations for arbitrary images, surpassing existing state-of-the-art algorithms.
arXiv Detail & Related papers (2023-11-30T18:59:44Z) - Calibrating Neural Simulation-Based Inference with Differentiable
Coverage Probability [50.44439018155837]
We propose to include a calibration term directly into the training objective of the neural model.
By introducing a relaxation of the classical formulation of calibration error we enable end-to-end backpropagation.
It is directly applicable to existing computational pipelines allowing reliable black-box posterior inference.
arXiv Detail & Related papers (2023-10-20T10:20:45Z) - Variational Bayesian Imaging with an Efficient Surrogate Score-based Prior [7.155937118886449]
We consider ill-posed inverse imaging problems in which one aims for a clean image posterior given incomplete or noisy measurements.
Recent work turned score-based diffusion models into principled priors for solving ill-posed imaging problems.
Our proposed surrogate prior is based on the evidence lower bound of a score-based diffusion model.
arXiv Detail & Related papers (2023-09-05T04:55:10Z) - Posterior samples of source galaxies in strong gravitational lenses with
score-based priors [107.52670032376555]
We use a score-based model to encode the prior for the inference of undistorted images of background galaxies.
We show how the balance between the likelihood and the prior meet our expectations in an experiment with out-of-distribution data.
arXiv Detail & Related papers (2022-11-07T19:00:42Z) - PriorGrad: Improving Conditional Denoising Diffusion Models with
Data-Driven Adaptive Prior [103.00403682863427]
We propose PriorGrad to improve the efficiency of the conditional diffusion model.
We show that PriorGrad achieves a faster convergence leading to data and parameter efficiency and improved quality.
arXiv Detail & Related papers (2021-06-11T14:04:03Z) - A deep-learning based Bayesian approach to seismic imaging and
uncertainty quantification [0.4588028371034407]
Uncertainty is essential when dealing with ill-conditioned inverse problems.
It is often not possible to formulate a prior distribution that precisely encodes our prior knowledge about the unknown.
We propose to use the functional form of a randomly convolutional neural network as an implicit structured prior.
arXiv Detail & Related papers (2020-01-13T23:46:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.