Multilevel Diffusion: Infinite Dimensional Score-Based Diffusion Models
for Image Generation
- URL: http://arxiv.org/abs/2303.04772v3
- Date: Sat, 4 Nov 2023 15:11:05 GMT
- Title: Multilevel Diffusion: Infinite Dimensional Score-Based Diffusion Models
for Image Generation
- Authors: Paul Hagemann, Sophie Mildenberger, Lars Ruthotto, Gabriele Steidl,
Nicole Tianjiao Yang
- Abstract summary: Score-based diffusion models (SBDM) have emerged as state-of-the-art approaches for image generation.
This paper develops SBDMs in the infinite-dimensional setting, that is, we model the training data as functions supported on a rectangular domain.
We demonstrate how to overcome two shortcomings of current SBDM approaches in the infinite-dimensional setting.
- Score: 2.7418627495572134
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Score-based diffusion models (SBDM) have recently emerged as state-of-the-art
approaches for image generation. Existing SBDMs are typically formulated in a
finite-dimensional setting, where images are considered as tensors of finite
size. This paper develops SBDMs in the infinite-dimensional setting, that is,
we model the training data as functions supported on a rectangular domain.
Besides the quest for generating images at ever higher resolution, our primary
motivation is to create a well-posed infinite-dimensional learning problem so
that we can discretize it consistently on multiple resolution levels. We
thereby intend to obtain diffusion models that generalize across different
resolution levels and improve the efficiency of the training process. We
demonstrate how to overcome two shortcomings of current SBDM approaches in the
infinite-dimensional setting. First, we modify the forward process to ensure
that the latent distribution is well-defined in the infinite-dimensional
setting using the notion of trace class operators. We derive the reverse
processes for finite approximations. Second, we illustrate that approximating
the score function with an operator network is beneficial for multilevel
training. After deriving the convergence of the discretization and the
approximation of multilevel training, we implement an infinite-dimensional SBDM
approach and show the first promising results on MNIST and Fashion-MNIST,
underlining our developed theory.
Related papers
- Binarized Diffusion Model for Image Super-Resolution [61.963833405167875]
We introduce a novel binarized diffusion model, BI-DiffSR, for image SR.
For the model structure, we design a UNet architecture optimized for binarization.
We propose the consistent-pixel-downsample (CP-Down) and consistent-pixel-upsample (CP-Up) to maintain dimension consistent.
Comprehensive experiments demonstrate that our BI-DiffSR outperforms existing binarization methods.
arXiv Detail & Related papers (2024-06-09T10:30:25Z) - Fixed Point Diffusion Models [13.035518953879539]
Fixed Point Diffusion Model (FPDM) is a novel approach to image generation that integrates the concept of fixed point solving into the framework of diffusion-based generative modeling.
Our approach embeds an implicit fixed point solving layer into the denoising network of a diffusion model, transforming the diffusion process into a sequence of closely-related fixed point problems.
We conduct experiments with state-of-the-art models on ImageNet, FFHQ, CelebA-HQ, and LSUN-Church, demonstrating substantial improvements in performance and efficiency.
arXiv Detail & Related papers (2024-01-16T18:55:54Z) - Exploiting Diffusion Prior for Generalizable Dense Prediction [85.4563592053464]
Recent advanced Text-to-Image (T2I) diffusion models are sometimes too imaginative for existing off-the-shelf dense predictors to estimate.
We introduce DMP, a pipeline utilizing pre-trained T2I models as a prior for dense prediction tasks.
Despite limited-domain training data, the approach yields faithful estimations for arbitrary images, surpassing existing state-of-the-art algorithms.
arXiv Detail & Related papers (2023-11-30T18:59:44Z) - SDDM: Score-Decomposed Diffusion Models on Manifolds for Unpaired
Image-to-Image Translation [96.11061713135385]
This work presents a new score-decomposed diffusion model to explicitly optimize the tangled distributions during image generation.
We equalize the refinement parts of the score function and energy guidance, which permits multi-objective optimization on the manifold.
SDDM outperforms existing SBDM-based methods with much fewer diffusion steps on several I2I benchmarks.
arXiv Detail & Related papers (2023-08-04T06:21:57Z) - Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance.
We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring.
Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
arXiv Detail & Related papers (2023-05-22T12:18:20Z) - DELAD: Deep Landweber-guided deconvolution with Hessian and sparse prior [0.22940141855172028]
We present a model for non-blind image deconvolution that incorporates the classic iterative method into a deep learning application.
We build our network based on the iterative Landweber deconvolution algorithm, which is integrated with trainable convolutional layers to enhance the recovered image structures and details.
arXiv Detail & Related papers (2022-09-30T11:15:03Z) - Image Generation with Multimodal Priors using Denoising Diffusion
Probabilistic Models [54.1843419649895]
A major challenge in using generative models to accomplish this task is the lack of paired data containing all modalities and corresponding outputs.
We propose a solution based on a denoising diffusion probabilistic synthesis models to generate images under multi-model priors.
arXiv Detail & Related papers (2022-06-10T12:23:05Z) - Learning Deformable Image Registration from Optimization: Perspective,
Modules, Bilevel Training and Beyond [62.730497582218284]
We develop a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation.
We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data.
arXiv Detail & Related papers (2020-04-30T03:23:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.