Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise

• URL: http://arxiv.org/abs/2208.09392v1
• Date: Fri, 19 Aug 2022 15:18:39 GMT
• Title: Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise
• Authors: Arpit Bansal, Eitan Borgnia, Hong-Min Chu, Jie S. Li, Hamid Kazemi, Furong Huang, Micah Goldblum, Jonas Geiping, Tom Goldstein
• Abstract summary: We show that an entire family of generative models can be constructed by varying the choice of image degradation. The success of fully deterministic models calls into question the community's understanding of diffusion models.
• Score: 52.59444045853966
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Standard diffusion models involve an image transform -- adding Gaussian noise -- and an image restoration operator that inverts this degradation. We observe that the generative behavior of diffusion models is not strongly dependent on the choice of image degradation, and in fact an entire family of generative models can be constructed by varying this choice. Even when using completely deterministic degradations (e.g., blur, masking, and more), the training and test-time update rules that underlie diffusion models can be easily generalized to create generative models. The success of these fully deterministic models calls into question the community's understanding of diffusion models, which relies on noise in either gradient Langevin dynamics or variational inference, and paves the way for generalized diffusion models that invert arbitrary processes. Our code is available at https://github.com/arpitbansal297/Cold-Diffusion-Models

Related papers

• DeltaDiff: A Residual-Guided Diffusion Model for Enhanced Image Super-Resolution [9.948203187433196]
  We propose a new diffusion model called Deltadiff, which uses only residuals between images for diffusion. Our method surpasses state-of-the-art models and generates results with better fidelity.
  arXiv  Detail & Related papers  (2025-02-18T06:07:14Z)
• Continuous Diffusion Model for Language Modeling [57.396578974401734]
  Existing continuous diffusion models for discrete data have limited performance compared to discrete approaches. We propose a continuous diffusion model for language modeling that incorporates the geometry of the underlying categorical distribution.
  arXiv  Detail & Related papers  (2025-02-17T08:54:29Z)
• Non-Normal Diffusion Models [3.5534933448684134]
  Diffusion models generate samples by incrementally reversing a process that turns data into noise. We show that when the step size goes to zero, the reversed process is invariant to the distribution of these increments. We demonstrate the effectiveness of these models on density estimation and generative modeling tasks on standard image datasets.
  arXiv  Detail & Related papers  (2024-12-10T21:31:12Z)
• Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later [1.8416014644193066]
  We observe that the reverse diffusion process that underlies image generation has the following properties. Individual trajectories tend to be low-dimensional and resemble 2D rotations' We find that this solution accurately describes the initial phase of image generation for pretrained models.
  arXiv  Detail & Related papers  (2023-03-04T20:08:57Z)
• Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models [60.63556257324894]
  A key desired property of image generative models is the ability to disentangle different attributes. We propose a simple, light-weight image editing algorithm where the mixing weights of the two text embeddings are optimized for style matching and content preservation. Experiments show that the proposed method can modify a wide range of attributes, with the performance outperforming diffusion-model-based image-editing algorithms.
  arXiv  Detail & Related papers  (2022-12-16T19:58:52Z)
• Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance [95.12230117950232]
  We show that a common latent space emerges from two diffusion models trained independently on related domains. Applying CycleDiffusion to text-to-image diffusion models, we show that large-scale text-to-image diffusion models can be used as zero-shot image-to-image editors.
  arXiv  Detail & Related papers  (2022-10-11T15:53:52Z)
• OCD: Learning to Overfit with Conditional Diffusion Models [95.1828574518325]
  We present a dynamic model in which the weights are conditioned on an input sample x. We learn to match those weights that would be obtained by finetuning a base model on x and its label y.
  arXiv  Detail & Related papers  (2022-10-02T09:42:47Z)
• Progressive Deblurring of Diffusion Models for Coarse-to-Fine Image Synthesis [39.671396431940224]
  diffusion models have shown remarkable results in image synthesis by gradually removing noise and amplifying signals. We propose a novel generative process that synthesizes images in a coarse-to-fine manner. Experiments show that the proposed model outperforms the previous method in FID on LSUN bedroom and church datasets.
  arXiv  Detail & Related papers  (2022-07-16T15:00:21Z)
• Generative Modelling With Inverse Heat Dissipation [21.738877553160304]
  We propose a new diffusion-like model that generates images through reversing the heat equation, a PDE that erases fine-scale information when run over the 2D plane of the image. Our new model shows emergent properties not seen in standard diffusion models, such as disentanglement of overall colour and shape in images.
  arXiv  Detail & Related papers  (2022-06-21T13:40:38Z)
• Discrete Denoising Flows [87.44537620217673]
  We introduce a new discrete flow-based model for categorical random variables: Discrete Denoising Flows (DDFs) In contrast with other discrete flow-based models, our model can be locally trained without introducing gradient bias. We show that DDFs outperform Discrete Flows on modeling a toy example, binary MNIST and Cityscapes segmentation maps, measured in log-likelihood.
  arXiv  Detail & Related papers  (2021-07-24T14:47:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.