NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation

• URL: http://arxiv.org/abs/2403.08840v1
• Date: Wed, 13 Mar 2024 12:32:25 GMT
• Title: NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation
• Authors: PengFei Zheng, Yonggang Zhang, Zhen Fang, Tongliang Liu, Defu Lian, Bo Han,
• Abstract summary: We propose a novel approach to correct noise for image validity, NoiseDiffusion. NoiseDiffusion performs within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss.
• Score: 86.7260950382448
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image interpolation based on diffusion models is promising in creating fresh and interesting images. Advanced interpolation methods mainly focus on spherical linear interpolation, where images are encoded into the noise space and then interpolated for denoising to images. However, existing methods face challenges in effectively interpolating natural images (not generated by diffusion models), thereby restricting their practical applicability. Our experimental investigations reveal that these challenges stem from the invalidity of the encoding noise, which may no longer obey the expected noise distribution, e.g., a normal distribution. To address these challenges, we propose a novel approach to correct noise for image interpolation, NoiseDiffusion. Specifically, NoiseDiffusion approaches the invalid noise to the expected distribution by introducing subtle Gaussian noise and introduces a constraint to suppress noise with extreme values. In this context, promoting noise validity contributes to mitigating image artifacts, but the constraint and introduced exogenous noise typically lead to a reduction in signal-to-noise ratio, i.e., loss of original image information. Hence, NoiseDiffusion performs interpolation within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss. Consequently, NoiseDiffusion enables us to interpolate natural images without causing artifacts or information loss, thus achieving the best interpolation results.

Related papers

• There and Back Again: On the relation between noises, images, and their inversions in diffusion models [3.5707423185282665]
  Diffusion Probabilistic Models (DDPMs) achieve state-of-the-art performance in synthesizing new images from random noise. Recent DDPM-based editing techniques try to mitigate this issue by inverting images back to their approximated staring noise. We study the relation between the initial Gaussian noise, the samples generated from it, and their corresponding latent encodings obtained through the inversion procedure.
  arXiv  Detail & Related papers  (2024-10-31T00:30:35Z)
• Diffusion-Based Image-to-Image Translation by Noise Correction via Prompt Interpolation [43.48099716183503]
  We propose a training-free approach tailored to diffusion-based image-to-image translation. Our approach can be easily incorporated into existing image-to-image translation methods.
  arXiv  Detail & Related papers  (2024-09-12T14:30:45Z)
• Stimulating the Diffusion Model for Image Denoising via Adaptive Embedding and Ensembling [56.506240377714754]
  We present a novel strategy called the Diffusion Model for Image Denoising (DMID) Our strategy includes an adaptive embedding method that embeds the noisy image into a pre-trained unconditional diffusion model. Our DMID strategy achieves state-of-the-art performance on both distortion-based and perception-based metrics.
  arXiv  Detail & Related papers  (2023-07-08T14:59:41Z)
• SVNR: Spatially-variant Noise Removal with Denoising Diffusion [43.2405873681083]
  We present a novel formulation of denoising diffusion that assumes a more realistic, spatially-variant noise model. In experiments we demonstrate the advantages of our approach over a strong diffusion model baseline, as well as over a state-of-the-art single image denoising method.
  arXiv  Detail & Related papers  (2023-06-28T09:32:00Z)
• Advancing Unsupervised Low-light Image Enhancement: Noise Estimation, Illumination Interpolation, and Self-Regulation [55.07472635587852]
  Low-Light Image Enhancement (LLIE) techniques have made notable advancements in preserving image details and enhancing contrast. These approaches encounter persistent challenges in efficiently mitigating dynamic noise and accommodating diverse low-light scenarios. We first propose a method for estimating the noise level in low light images in a quick and accurate way. We then devise a Learnable Illumination Interpolator (LII) to satisfy general constraints between illumination and input.
  arXiv  Detail & Related papers  (2023-05-17T13:56:48Z)
• Real-World Denoising via Diffusion Model [14.722529440511446]
  Real-world image denoising aims to recover clean images from noisy images captured in natural environments. diffusion models have achieved very promising results in the field of image generation, outperforming previous generation models. This paper proposes a novel general denoising diffusion model that can be used for real-world image denoising.
  arXiv  Detail & Related papers  (2023-05-08T04:48:03Z)
• Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training [50.018580462619425]
  We propose a novel framework, namely Pixel-level Noise-aware Generative Adrial Network (PNGAN) PNGAN employs a pre-trained real denoiser to map the fake and real noisy images into a nearly noise-free solution space. For better noise fitting, we present an efficient architecture Simple Multi-versa-scale Network (SMNet) as the generator.
  arXiv  Detail & Related papers  (2022-04-06T14:09:02Z)
• FINO: Flow-based Joint Image and Noise Model [23.9749061109964]
  Flow-based joint Image and NOise model (FINO) We propose a novel Flow-based joint Image and NOise model (FINO) that distinctly decouples the image and noise in the latent space and losslessly reconstructs them via a series of invertible transformations.
  arXiv  Detail & Related papers  (2021-11-11T02:51:54Z)
• Adaptive noise imitation for image denoising [58.21456707617451]
  We develop a new textbfadaptive noise imitation (ADANI) algorithm that can synthesize noisy data from naturally noisy images. To produce realistic noise, a noise generator takes unpaired noisy/clean images as input, where the noisy image is a guide for noise generation. Coupling the noisy data output from ADANI with the corresponding ground-truth, a denoising CNN is then trained in a fully-supervised manner.
  arXiv  Detail & Related papers  (2020-11-30T02:49:36Z)
• Variational Denoising Network: Toward Blind Noise Modeling and Removal [59.36166491196973]
  Blind image denoising is an important yet very challenging problem in computer vision. We propose a new variational inference method, which integrates both noise estimation and image denoising.
  arXiv  Detail & Related papers  (2019-08-29T15:54:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.