Generative Modelling With Inverse Heat Dissipation
- URL: http://arxiv.org/abs/2206.13397v7
- Date: Wed, 12 Apr 2023 19:50:24 GMT
- Title: Generative Modelling With Inverse Heat Dissipation
- Authors: Severi Rissanen, Markus Heinonen, Arno Solin
- Abstract summary: We propose a new diffusion-like model that generates images through reversing the heat equation, a PDE that erases fine-scale information when run over the 2D plane of the image.
Our new model shows emergent properties not seen in standard diffusion models, such as disentanglement of overall colour and shape in images.
- Score: 21.738877553160304
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: While diffusion models have shown great success in image generation, their
noise-inverting generative process does not explicitly consider the structure
of images, such as their inherent multi-scale nature. Inspired by diffusion
models and the empirical success of coarse-to-fine modelling, we propose a new
diffusion-like model that generates images through stochastically reversing the
heat equation, a PDE that locally erases fine-scale information when run over
the 2D plane of the image. We interpret the solution of the forward heat
equation with constant additive noise as a variational approximation in the
diffusion latent variable model. Our new model shows emergent qualitative
properties not seen in standard diffusion models, such as disentanglement of
overall colour and shape in images. Spectral analysis on natural images
highlights connections to diffusion models and reveals an implicit
coarse-to-fine inductive bias in them.
Related papers
- Soft Mixture Denoising: Beyond the Expressive Bottleneck of Diffusion
Models [76.46246743508651]
We show that current diffusion models actually have an expressive bottleneck in backward denoising.
We introduce soft mixture denoising (SMD), an expressive and efficient model for backward denoising.
arXiv Detail & Related papers (2023-09-25T12:03:32Z) - Exploring the Optimal Choice for Generative Processes in Diffusion
Models: Ordinary vs Stochastic Differential Equations [6.2284442126065525]
We study the problem mathematically for two limiting scenarios: the zero diffusion (ODE) case and the large diffusion case.
Our findings indicate that when the perturbation occurs at the end of the generative process, the ODE model outperforms the SDE model with a large diffusion coefficient.
arXiv Detail & Related papers (2023-06-03T09:27:15Z) - Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later [1.8416014644193066]
We observe that the reverse diffusion process that underlies image generation has the following properties.
Individual trajectories tend to be low-dimensional and resemble 2D rotations'
We find that this solution accurately describes the initial phase of image generation for pretrained models.
arXiv Detail & Related papers (2023-03-04T20:08:57Z) - Bi-Noising Diffusion: Towards Conditional Diffusion Models with
Generative Restoration Priors [64.24948495708337]
We introduce a new method that brings predicted samples to the training data manifold using a pretrained unconditional diffusion model.
We perform comprehensive experiments to demonstrate the effectiveness of our approach on super-resolution, colorization, turbulence removal, and image-deraining tasks.
arXiv Detail & Related papers (2022-12-14T17:26:35Z) - SinDiffusion: Learning a Diffusion Model from a Single Natural Image [159.4285444680301]
We present SinDiffusion, leveraging denoising diffusion models to capture internal distribution of patches from a single natural image.
It is based on two core designs. First, SinDiffusion is trained with a single model at a single scale instead of multiple models with progressive growing of scales.
Second, we identify that a patch-level receptive field of the diffusion network is crucial and effective for capturing the image's patch statistics.
arXiv Detail & Related papers (2022-11-22T18:00:03Z) - Unifying Diffusion Models' Latent Space, with Applications to
CycleDiffusion and Guidance [95.12230117950232]
We show that a common latent space emerges from two diffusion models trained independently on related domains.
Applying CycleDiffusion to text-to-image diffusion models, we show that large-scale text-to-image diffusion models can be used as zero-shot image-to-image editors.
arXiv Detail & Related papers (2022-10-11T15:53:52Z) - Blurring Diffusion Models [27.339469450737525]
We show that blurring can equivalently be defined through a Gaussian diffusion process with non-isotropic noise.
We propose a class of diffusion models that offers the best of both standard Gaussian denoising diffusion and inverse heat dissipation.
arXiv Detail & Related papers (2022-09-12T19:16:48Z) - Diffusion Models in Vision: A Survey [80.82832715884597]
A diffusion model is a deep generative model that is based on two stages, a forward diffusion stage and a reverse diffusion stage.
Diffusion models are widely appreciated for the quality and diversity of the generated samples, despite their known computational burdens.
arXiv Detail & Related papers (2022-09-10T22:00:30Z) - Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise [52.59444045853966]
We show that an entire family of generative models can be constructed by varying the choice of image degradation.
The success of fully deterministic models calls into question the community's understanding of diffusion models.
arXiv Detail & Related papers (2022-08-19T15:18:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.