BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inference

• URL: http://arxiv.org/abs/2310.11142v2
• Date: Mon, 4 Mar 2024 09:07:44 GMT
• Title: BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inference
• Authors: Siqi Kou, Lei Gan, Dequan Wang, Chongxuan Li, Zhijie Deng
• Abstract summary: BayesDiff is a pixel-wise uncertainty estimator for generations from diffusion models based on Bayesian inference. The estimated pixel-wise uncertainty can not only be aggregated into a sample-wise metric to filter out low-fidelity images but also aids in augmenting successful generations and rectifying artifacts in failed generations in text-to-image tasks.
• Score: 29.682407300058394
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models have impressive image generation capability, but low-quality generations still exist, and their identification remains challenging due to the lack of a proper sample-wise metric. To address this, we propose BayesDiff, a pixel-wise uncertainty estimator for generations from diffusion models based on Bayesian inference. In particular, we derive a novel uncertainty iteration principle to characterize the uncertainty dynamics in diffusion, and leverage the last-layer Laplace approximation for efficient Bayesian inference. The estimated pixel-wise uncertainty can not only be aggregated into a sample-wise metric to filter out low-fidelity images but also aids in augmenting successful generations and rectifying artifacts in failed generations in text-to-image tasks. Extensive experiments demonstrate the efficacy of BayesDiff and its promise for practical applications.

Related papers

• Diffusion Models for Inverse Problems in the Exponential Family [45.560812800359685]
  We extend diffusion models to handle inverse problems where the observations follow a distribution from the exponential family. We introduce the evidence trick, a method that provides a tractable approximation to the likelihood score. We demonstrate the real-world impact of our methodology by showing that it performs competitively with the current state-of-the-art in predicting malaria prevalence estimates in Sub-Saharan Africa.
  arXiv  Detail & Related papers  (2025-02-09T18:56:57Z)
• Diffusion Model Guided Sampling with Pixel-Wise Aleatoric Uncertainty Estimation [10.269485943949332]
  We propose to estimate the pixel-wise aleatoric uncertainty during the sampling phase of diffusion models. The uncertainty is computed as the variance of the denoising scores with a perturbation scheme specifically designed for diffusion models. We show that our guided approach leads to better sample generation in terms of FID scores.
  arXiv  Detail & Related papers  (2024-11-29T19:02:08Z)
• VIPaint: Image Inpainting with Pre-Trained Diffusion Models via Variational Inference [5.852077003870417]
  We show that our VIPaint method significantly outperforms previous approaches in both the plausibility and diversity of imputations. We show that our VIPaint method significantly outperforms previous approaches in both the plausibility and diversity of imputations.
  arXiv  Detail & Related papers  (2024-11-28T05:35:36Z)
• Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model [61.53213964333474]
  We propose a unified framework Adv-Diffusion that can generate imperceptible adversarial identity perturbations in the latent space but not the raw pixel space. Specifically, we propose the identity-sensitive conditioned diffusion generative model to generate semantic perturbations in the surroundings. The designed adaptive strength-based adversarial perturbation algorithm can ensure both attack transferability and stealthiness.
  arXiv  Detail & Related papers  (2023-12-18T15:25:23Z)
• Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
  Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
  arXiv  Detail & Related papers  (2023-06-01T03:08:28Z)
• DensePure: Understanding Diffusion Models towards Adversarial Robustness [110.84015494617528]
  We analyze the properties of diffusion models and establish the conditions under which they can enhance certified robustness. We propose a new method DensePure, designed to improve the certified robustness of a pretrained model (i.e. a classifier) We show that this robust region is a union of multiple convex sets, and is potentially much larger than the robust regions identified in previous works.
  arXiv  Detail & Related papers  (2022-11-01T08:18:07Z)
• Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval [51.83967175585896]
  UAL aims at providing reliability-aware predictions by considering data uncertainty and model uncertainty simultaneously. Data uncertainty captures the noise" inherent in the sample, while model uncertainty depicts the model's confidence in the sample's prediction.
  arXiv  Detail & Related papers  (2022-10-24T17:53:20Z)
• Deblurring via Stochastic Refinement [85.42730934561101]
  We present an alternative framework for blind deblurring based on conditional diffusion models. Our method is competitive in terms of distortion metrics such as PSNR.
  arXiv  Detail & Related papers  (2021-12-05T04:36:09Z)
• Deep Bayesian inference for seismic imaging with tasks [0.6445605125467573]
  We propose to use techniques from Bayesian inference and deep neural networks to translate uncertainty in seismic imaging to uncertainty in tasks performed on the image. A systematic approach is proposed to translate uncertainty due to noise in the data to confidence intervals of automatically tracked horizons in the image.
  arXiv  Detail & Related papers  (2021-10-10T15:25:44Z)
• The Hidden Uncertainty in a Neural Networks Activations [105.4223982696279]
  The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data. This work investigates whether this distribution correlates with a model's epistemic uncertainty, thus indicating its ability to generalise to novel inputs.
  arXiv  Detail & Related papers  (2020-12-05T17:30:35Z)
• A deep-learning based Bayesian approach to seismic imaging and uncertainty quantification [0.4588028371034407]
  Uncertainty is essential when dealing with ill-conditioned inverse problems. It is often not possible to formulate a prior distribution that precisely encodes our prior knowledge about the unknown. We propose to use the functional form of a randomly convolutional neural network as an implicit structured prior.
  arXiv  Detail & Related papers  (2020-01-13T23:46:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.