Consistent Diffusion: Denoising Diffusion Model with Data-Consistent Training for Image Restoration

• URL: http://arxiv.org/abs/2412.12550v1
• Date: Tue, 17 Dec 2024 05:21:16 GMT
• Title: Consistent Diffusion: Denoising Diffusion Model with Data-Consistent Training for Image Restoration
• Authors: Xinlong Cheng, Tiantian Cao, Guoan Cheng, Bangxuan Huang, Xinghan Tian, Ye Wang, Xiaoyu He, Weixin Li, Tianfan Xue, Xuan Dong,
• Abstract summary: We address the limitations of denoising diffusion models (DDMs) in image restoration tasks. We propose a novel training method, named data-consistent training, which allows the DDMs to access images with accumulated errors during training. Our method has significant improvements over state-of-the-art methods while effectively minimizing distortions and preserving image fidelity.
• Score: 9.745025353708865
• License:
• Abstract: In this work, we address the limitations of denoising diffusion models (DDMs) in image restoration tasks, particularly the shape and color distortions that can compromise image quality. While DDMs have demonstrated a promising performance in many applications such as text-to-image synthesis, their effectiveness in image restoration is often hindered by shape and color distortions. We observe that these issues arise from inconsistencies between the training and testing data used by DDMs. Based on our observation, we propose a novel training method, named data-consistent training, which allows the DDMs to access images with accumulated errors during training, thereby ensuring the model to learn to correct these errors. Experimental results show that, across five image restoration tasks, our method has significant improvements over state-of-the-art methods while effectively minimizing distortions and preserving image fidelity.

Related papers

• Frequency-Aware Guidance for Blind Image Restoration via Diffusion Models [20.898262207229873]
  Blind image restoration remains a significant challenge in low-level vision tasks. Guided diffusion models have achieved promising results in blind image restoration. We propose a novel frequency-aware guidance loss that can be integrated into various diffusion models in a plug-and-play manner.
  arXiv  Detail & Related papers  (2024-11-19T12:18:16Z)
• Learning Diffusion Model from Noisy Measurement using Principled Expectation-Maximization Method [9.173055778539641]
  We propose a principled expectation-maximization (EM) framework that iteratively learns diffusion models from noisy data with arbitrary corruption types. Our framework employs a plug-and-play Monte Carlo method to accurately estimate clean images from noisy measurements, followed by training the diffusion model using the reconstructed images.
  arXiv  Detail & Related papers  (2024-10-15T03:54:59Z)
• PromptFix: You Prompt and We Fix the Photo [84.69812824355269]
  Diffusion models equipped with language models demonstrate excellent controllability in image generation tasks. The lack of diverse instruction-following data hampers the development of models. We propose PromptFix, a framework that enables diffusion models to follow human instructions.
  arXiv  Detail & Related papers  (2024-05-27T03:13:28Z)
• SeNM-VAE: Semi-Supervised Noise Modeling with Hierarchical Variational Autoencoder [13.453138169497903]
  SeNM-VAE is a semi-supervised noise modeling method that leverages both paired and unpaired datasets to generate realistic degraded data. We employ our method to generate paired training samples for real-world image denoising and super-resolution tasks. Our approach excels in the quality of synthetic degraded images compared to other unpaired and paired noise modeling methods.
  arXiv  Detail & Related papers  (2024-03-26T09:03:40Z)
• Gradpaint: Gradient-Guided Inpainting with Diffusion Models [71.47496445507862]
  Denoising Diffusion Probabilistic Models (DDPMs) have recently achieved remarkable results in conditional and unconditional image generation. We present GradPaint, which steers the generation towards a globally coherent image. We generalizes well to diffusion models trained on various datasets, improving upon current state-of-the-art supervised and unsupervised methods.
  arXiv  Detail & Related papers  (2023-09-18T09:36:24Z)
• Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction [75.91471250967703]
  We introduce a novel sampling framework called Steerable Conditional Diffusion. This framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement. We achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities.
  arXiv  Detail & Related papers  (2023-08-28T08:47:06Z)
• Diffusion Models for Image Restoration and Enhancement -- A Comprehensive Survey [96.99328714941657]
  We present a comprehensive review of recent diffusion model-based methods on image restoration. We classify and emphasize the innovative designs using diffusion models for both IR and blind/real-world IR. We propose five potential and challenging directions for the future research of diffusion model-based IR.
  arXiv  Detail & Related papers  (2023-08-18T08:40:38Z)
• SinDDM: A Single Image Denoising Diffusion Model [28.51951207066209]
  We introduce a framework for training a Denoising diffusion model on a single image. Our method, which we coin SinDDM, learns the internal statistics of the training image by using a multi-scale diffusion process. It is applicable in a wide array of tasks, including style transfer and harmonization.
  arXiv  Detail & Related papers  (2022-11-29T20:44:25Z)
• Denoising Diffusion Restoration Models [110.1244240726802]
  Denoising Diffusion Restoration Models (DDRM) is an efficient, unsupervised posterior sampling method. We demonstrate DDRM's versatility on several image datasets for super-resolution, deblurring, inpainting, and colorization.
  arXiv  Detail & Related papers  (2022-01-27T20:19:07Z)
• Data augmentation for deep learning based accelerated MRI reconstruction with limited data [46.44703053411933]
  Deep neural networks have emerged as very successful tools for image restoration and reconstruction tasks. To achieve state-of-the-art performance, training on large and diverse sets of images is considered critical. We propose a pipeline for data augmentation for accelerated MRI reconstruction and study its effectiveness at reducing the required training data.
  arXiv  Detail & Related papers  (2021-06-28T19:08:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.