Efficient Diffusion Models for Vision: A Survey
- URL: http://arxiv.org/abs/2210.09292v3
- Date: Tue, 12 Mar 2024 02:08:37 GMT
- Title: Efficient Diffusion Models for Vision: A Survey
- Authors: Anwaar Ulhaq and Naveed Akhtar
- Abstract summary: Diffusion Models (DMs) have demonstrated state-of-the-art performance in content generation without requiring adversarial training.
DMs are inspired by non-equilibrium thermodynamics and have inherent high computational complexity.
DMs incur considerable computational overhead during both training and inference stages.
- Score: 34.610299976294904
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Diffusion Models (DMs) have demonstrated state-of-the-art performance in
content generation without requiring adversarial training. These models are
trained using a two-step process. First, a forward - diffusion - process
gradually adds noise to a datum (usually an image). Then, a backward - reverse
diffusion - process gradually removes the noise to turn it into a sample of the
target distribution being modelled. DMs are inspired by non-equilibrium
thermodynamics and have inherent high computational complexity. Due to the
frequent function evaluations and gradient calculations in high-dimensional
spaces, these models incur considerable computational overhead during both
training and inference stages. This can not only preclude the democratization
of diffusion-based modelling, but also hinder the adaption of diffusion models
in real-life applications. Not to mention, the efficiency of computational
models is fast becoming a significant concern due to excessive energy
consumption and environmental scares. These factors have led to multiple
contributions in the literature that focus on devising computationally
efficient DMs. In this review, we present the most recent advances in diffusion
models for vision, specifically focusing on the important design aspects that
affect the computational efficiency of DMs. In particular, we emphasize the
recently proposed design choices that have led to more efficient DMs. Unlike
the other recent reviews, which discuss diffusion models from a broad
perspective, this survey is aimed at pushing this research direction forward by
highlighting the design strategies in the literature that are resulting in
practicable models for the broader research community. We also provide a future
outlook of diffusion models in vision from their computational efficiency
viewpoint.
Related papers
- Diffusion Models in Low-Level Vision: A Survey [82.77962165415153]
diffusion model-based solutions have emerged as widely acclaimed for their ability to produce samples of superior quality and diversity.
We present three generic diffusion modeling frameworks and explore their correlations with other deep generative models.
We summarize extended diffusion models applied in other tasks, including medical, remote sensing, and video scenarios.
arXiv Detail & Related papers (2024-06-17T01:49:27Z) - An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization [59.63880337156392]
Diffusion models have achieved tremendous success in computer vision, audio, reinforcement learning, and computational biology.
Despite the significant empirical success, theory of diffusion models is very limited.
This paper provides a well-rounded theoretical exposure for stimulating forward-looking theories and methods of diffusion models.
arXiv Detail & Related papers (2024-04-11T14:07:25Z) - Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation [59.184980778643464]
Fine-tuning Diffusion Models remains an underexplored frontier in generative artificial intelligence (GenAI)
In this paper, we introduce an innovative technique called self-play fine-tuning for diffusion models (SPIN-Diffusion)
Our approach offers an alternative to conventional supervised fine-tuning and RL strategies, significantly improving both model performance and alignment.
arXiv Detail & Related papers (2024-02-15T18:59:18Z) - Restoration based Generative Models [0.886014926770622]
Denoising diffusion models (DDMs) have attracted increasing attention by showing impressive synthesis quality.
In this paper, we establish the interpretation of DDMs in terms of image restoration (IR)
We propose a multi-scale training, which improves the performance compared to the diffusion process, by taking advantage of the flexibility of the forward process.
We believe that our framework paves the way for designing a new type of flexible general generative model.
arXiv Detail & Related papers (2023-02-20T00:53:33Z) - Diffusion Models in Vision: A Survey [80.82832715884597]
A diffusion model is a deep generative model that is based on two stages, a forward diffusion stage and a reverse diffusion stage.
Diffusion models are widely appreciated for the quality and diversity of the generated samples, despite their known computational burdens.
arXiv Detail & Related papers (2022-09-10T22:00:30Z) - A Survey on Generative Diffusion Model [75.93774014861978]
Diffusion models are an emerging class of deep generative models.
They have certain limitations, including a time-consuming iterative generation process and confinement to high-dimensional Euclidean space.
This survey presents a plethora of advanced techniques aimed at enhancing diffusion models.
arXiv Detail & Related papers (2022-09-06T16:56:21Z) - How Much is Enough? A Study on Diffusion Times in Score-based Generative
Models [76.76860707897413]
Current best practice advocates for a large T to ensure that the forward dynamics brings the diffusion sufficiently close to a known and simple noise distribution.
We show how an auxiliary model can be used to bridge the gap between the ideal and the simulated forward dynamics, followed by a standard reverse diffusion process.
arXiv Detail & Related papers (2022-06-10T15:09:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.