Generative Modelling With Inverse Heat Dissipation

• URL: http://arxiv.org/abs/2206.13397v7
• Date: Wed, 12 Apr 2023 19:50:24 GMT
• Title: Generative Modelling With Inverse Heat Dissipation
• Authors: Severi Rissanen, Markus Heinonen, Arno Solin
• Abstract summary: We propose a new diffusion-like model that generates images through reversing the heat equation, a PDE that erases fine-scale information when run over the 2D plane of the image. Our new model shows emergent properties not seen in standard diffusion models, such as disentanglement of overall colour and shape in images.
• Score: 21.738877553160304
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While diffusion models have shown great success in image generation, their noise-inverting generative process does not explicitly consider the structure of images, such as their inherent multi-scale nature. Inspired by diffusion models and the empirical success of coarse-to-fine modelling, we propose a new diffusion-like model that generates images through stochastically reversing the heat equation, a PDE that locally erases fine-scale information when run over the 2D plane of the image. We interpret the solution of the forward heat equation with constant additive noise as a variational approximation in the diffusion latent variable model. Our new model shows emergent qualitative properties not seen in standard diffusion models, such as disentanglement of overall colour and shape in images. Spectral analysis on natural images highlights connections to diffusion models and reveals an implicit coarse-to-fine inductive bias in them.

Related papers

• G2D2: Gradient-guided Discrete Diffusion for image inverse problem solving [55.185588994883226]
  This paper presents a novel method for addressing linear inverse problems by leveraging image-generation models based on discrete diffusion as priors. To the best of our knowledge, this is the first approach to use discrete diffusion model-based priors for solving image inverse problems.
  arXiv  Detail & Related papers  (2024-10-09T06:18:25Z)
• Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later [1.8416014644193066]
  We observe that the reverse diffusion process that underlies image generation has the following properties. Individual trajectories tend to be low-dimensional and resemble 2D rotations' We find that this solution accurately describes the initial phase of image generation for pretrained models.
  arXiv  Detail & Related papers  (2023-03-04T20:08:57Z)
• Bi-Noising Diffusion: Towards Conditional Diffusion Models with Generative Restoration Priors [64.24948495708337]
  We introduce a new method that brings predicted samples to the training data manifold using a pretrained unconditional diffusion model. We perform comprehensive experiments to demonstrate the effectiveness of our approach on super-resolution, colorization, turbulence removal, and image-deraining tasks.
  arXiv  Detail & Related papers  (2022-12-14T17:26:35Z)
• SinDiffusion: Learning a Diffusion Model from a Single Natural Image [159.4285444680301]
  We present SinDiffusion, leveraging denoising diffusion models to capture internal distribution of patches from a single natural image. It is based on two core designs. First, SinDiffusion is trained with a single model at a single scale instead of multiple models with progressive growing of scales. Second, we identify that a patch-level receptive field of the diffusion network is crucial and effective for capturing the image's patch statistics.
  arXiv  Detail & Related papers  (2022-11-22T18:00:03Z)
• Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance [95.12230117950232]
  We show that a common latent space emerges from two diffusion models trained independently on related domains. Applying CycleDiffusion to text-to-image diffusion models, we show that large-scale text-to-image diffusion models can be used as zero-shot image-to-image editors.
  arXiv  Detail & Related papers  (2022-10-11T15:53:52Z)
• Blurring Diffusion Models [27.339469450737525]
  We show that blurring can equivalently be defined through a Gaussian diffusion process with non-isotropic noise. We propose a class of diffusion models that offers the best of both standard Gaussian denoising diffusion and inverse heat dissipation.
  arXiv  Detail & Related papers  (2022-09-12T19:16:48Z)
• Diffusion Models in Vision: A Survey [80.82832715884597]
  A diffusion model is a deep generative model that is based on two stages, a forward diffusion stage and a reverse diffusion stage. Diffusion models are widely appreciated for the quality and diversity of the generated samples, despite their known computational burdens.
  arXiv  Detail & Related papers  (2022-09-10T22:00:30Z)
• Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise [52.59444045853966]
  We show that an entire family of generative models can be constructed by varying the choice of image degradation. The success of fully deterministic models calls into question the community's understanding of diffusion models.
  arXiv  Detail & Related papers  (2022-08-19T15:18:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.