Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Imaging Inverse Problems

• URL: http://arxiv.org/abs/2308.14409v1
• Date: Mon, 28 Aug 2023 08:47:06 GMT
• Title: Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Imaging Inverse Problems
• Authors: Riccardo Barbano, Alexander Denker, Hyungjin Chung, Tae Hoon Roh, Simon Arrdige, Peter Maass, Bangti Jin, Jong Chul Ye
• Abstract summary: We introduce a novel sampling framework called Steerable Conditional Diffusion. This framework adapts the denoising network specifically to the available measured data. We achieve substantial enhancements in OOD performance across diverse imaging modalities.
• Score: 78.76955228709241
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Denoising diffusion models have emerged as the go-to framework for solving inverse problems in imaging. A critical concern regarding these models is their performance on out-of-distribution (OOD) tasks, which remains an under-explored challenge. Realistic reconstructions inconsistent with the measured data can be generated, hallucinating image features that are uniquely present in the training dataset. To simultaneously enforce data-consistency and leverage data-driven priors, we introduce a novel sampling framework called Steerable Conditional Diffusion. This framework adapts the denoising network specifically to the available measured data. Utilising our proposed method, we achieve substantial enhancements in OOD performance across diverse imaging modalities, advancing the robust deployment of denoising diffusion models in real-world applications.

Related papers

• Bayesian Conditioned Diffusion Models for Inverse Problems [11.67269909384503]
  Diffusion models excel in many image reconstruction tasks that involve inverse problems based on a forward measurement operator. We propose a novel Bayesian conditioning technique for diffusion models, BCDM, based on score-functions associated with the conditional distribution of desired images. We show state-of-the-art performance in image dealiasing, deblurring, super-resolution, and inpainting with the proposed technique.
  arXiv  Detail & Related papers  (2024-06-14T07:13:03Z)
• SeNM-VAE: Semi-Supervised Noise Modeling with Hierarchical Variational Autoencoder [13.453138169497903]
  SeNM-VAE is a semi-supervised noise modeling method that leverages both paired and unpaired datasets to generate realistic degraded data. We employ our method to generate paired training samples for real-world image denoising and super-resolution tasks. Our approach excels in the quality of synthetic degraded images compared to other unpaired and paired noise modeling methods.
  arXiv  Detail & Related papers  (2024-03-26T09:03:40Z)
• DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
  Current perceptive models heavily depend on resource-intensive datasets. We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability. Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
  arXiv  Detail & Related papers  (2024-03-20T04:58:03Z)
• Spatial-and-Frequency-aware Restoration method for Images based on Diffusion Models [7.947387272047602]
  We propose SaFaRI, a spatial-and-frequency-aware diffusion model for Image Restoration (IR) Our model encourages images to preserve data-fidelity in both the spatial and frequency domains, resulting in enhanced reconstruction quality. Our thorough evaluation demonstrates that SaFaRI achieves state-of-the-art performance on both the ImageNet datasets and FFHQ datasets.
  arXiv  Detail & Related papers  (2024-01-31T07:11:01Z)
• The Journey, Not the Destination: How Data Guides Diffusion Models [75.19694584942623]
  Diffusion models trained on large datasets can synthesize photo-realistic images of remarkable quality and diversity. We propose a framework that: (i) provides a formal notion of data attribution in the context of diffusion models, and (ii) allows us to counterfactually validate such attributions.
  arXiv  Detail & Related papers  (2023-12-11T08:39:43Z)
• Mitigating Exposure Bias in Discriminator Guided Diffusion Models [4.5349436061325425]
  We propose SEDM-G++, which incorporates a modified sampling approach, combining Discriminator Guidance and Epsilon Scaling. Our proposed approach outperforms the current state-of-the-art, by achieving an FID score of 1.73 on the unconditional CIFAR-10 dataset.
  arXiv  Detail & Related papers  (2023-11-18T20:49:50Z)
• ExposureDiffusion: Learning to Expose for Low-light Image Enhancement [87.08496758469835]
  This work addresses the issue by seamlessly integrating a diffusion model with a physics-based exposure model. Our method obtains significantly improved performance and reduced inference time compared with vanilla diffusion models. The proposed framework can work with both real-paired datasets, SOTA noise models, and different backbone networks.
  arXiv  Detail & Related papers  (2023-07-15T04:48:35Z)
• Stimulating the Diffusion Model for Image Denoising via Adaptive Embedding and Ensembling [56.506240377714754]
  We present a novel strategy called the Diffusion Model for Image Denoising (DMID) Our strategy includes an adaptive embedding method that embeds the noisy image into a pre-trained unconditional diffusion model. Our DMID strategy achieves state-of-the-art performance on both distortion-based and perception-based metrics.
  arXiv  Detail & Related papers  (2023-07-08T14:59:41Z)
• Realistic Noise Synthesis with Diffusion Models [68.48859665320828]
  Deep image denoising models often rely on large amount of training data for the high quality performance. We propose a novel method that synthesizes realistic noise using diffusion models, namely Realistic Noise Synthesize Diffusor (RNSD) RNSD can incorporate guided multiscale content, such as more realistic noise with spatial correlations can be generated at multiple frequencies.
  arXiv  Detail & Related papers  (2023-05-23T12:56:01Z)
• Empowering Diffusion Models on the Embedding Space for Text Generation [38.664533078347304]
  We study the optimization challenges encountered with both the embedding space and the denoising model. Data distribution is learnable for embeddings, which may lead to the collapse of the embedding space and unstable training. Based on the above analysis, we propose Difformer, an embedding diffusion model based on Transformer.
  arXiv  Detail & Related papers  (2022-12-19T12:44:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.