Image generation with shortest path diffusion
- URL: http://arxiv.org/abs/2306.00501v1
- Date: Thu, 1 Jun 2023 09:53:35 GMT
- Title: Image generation with shortest path diffusion
- Authors: Ayan Das, Stathi Fotiadis, Anil Batra, Farhang Nabiei, FengTing Liao,
Sattar Vakili, Da-shan Shiu, Alberto Bernacchia
- Abstract summary: We show that the Shortest Path Diffusion (SPD) determines the entire structure of the corruption.
We show that SPD improves on strong baselines without any hypertemporal tuning and outperforms all previous Diffusion Models based on image blurring.
Our work sheds new light on made observations in recent works and provides a new approach to improve diffusion models on images and other types of data.
- Score: 10.041144269046693
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The field of image generation has made significant progress thanks to the
introduction of Diffusion Models, which learn to progressively reverse a given
image corruption. Recently, a few studies introduced alternative ways of
corrupting images in Diffusion Models, with an emphasis on blurring. However,
these studies are purely empirical and it remains unclear what is the optimal
procedure for corrupting an image. In this work, we hypothesize that the
optimal procedure minimizes the length of the path taken when corrupting an
image towards a given final state. We propose the Fisher metric for the path
length, measured in the space of probability distributions. We compute the
shortest path according to this metric, and we show that it corresponds to a
combination of image sharpening, rather than blurring, and noise deblurring.
While the corruption was chosen arbitrarily in previous work, our Shortest Path
Diffusion (SPD) determines uniquely the entire spatiotemporal structure of the
corruption. We show that SPD improves on strong baselines without any
hyperparameter tuning, and outperforms all previous Diffusion Models based on
image blurring. Furthermore, any small deviation from the shortest path leads
to worse performance, suggesting that SPD provides the optimal procedure to
corrupt images. Our work sheds new light on observations made in recent works
and provides a new approach to improve diffusion models on images and other
types of data.
Related papers
- Fast constrained sampling in pre-trained diffusion models [77.21486516041391]
Diffusion models have dominated the field of large, generative image models.
We propose an algorithm for fast-constrained sampling in large pre-trained diffusion models.
arXiv Detail & Related papers (2024-10-24T14:52:38Z) - CODE: Confident Ordinary Differential Editing [62.83365660727034]
Confident Ordinary Differential Editing (CODE) is a novel approach for image synthesis that effectively handles Out-of-Distribution (OoD) guidance images.
CODE enhances images through score-based updates along the probability-flow Ordinary Differential Equation (ODE) trajectory.
Our method operates in a fully blind manner, relying solely on a pre-trained generative model.
arXiv Detail & Related papers (2024-08-22T14:12:20Z) - Blind Image Restoration via Fast Diffusion Inversion [17.139433082780037]
Blind Image Restoration via fast Diffusion (BIRD) is a blind IR method that jointly optimize for the degradation model parameters and the restored image.
A key idea in our method is not to modify the reverse sampling, i.e., not to alter all the intermediate latents, once an initial noise is sampled.
We experimentally validate BIRD on several image restoration tasks and show that it achieves state of the art performance on all of them.
arXiv Detail & Related papers (2024-05-29T23:38:12Z) - Efficient Diffusion-Driven Corruption Editor for Test-Time Adaptation [37.67328706787212]
Test-time adaptation (TTA) addresses the unforeseen distribution shifts occurring during test time.
We propose a novel TTA method that leverages an image editing model based on a latent diffusion model (LDM) and fine-tunes it using our newly introduced corruption modeling scheme.
Our model achieves the best performance with a 100 times faster runtime than that of a diffusion-based baseline.
arXiv Detail & Related papers (2024-03-16T12:18:20Z) - Ambient Diffusion Posterior Sampling: Solving Inverse Problems with
Diffusion Models trained on Corrupted Data [56.81246107125692]
Ambient Diffusion Posterior Sampling (A-DPS) is a generative model pre-trained on one type of corruption.
We show that A-DPS can sometimes outperform models trained on clean data for several image restoration tasks in both speed and performance.
We extend the Ambient Diffusion framework to train MRI models with access only to Fourier subsampled multi-coil MRI measurements.
arXiv Detail & Related papers (2024-03-13T17:28:20Z) - Fast Diffusion EM: a diffusion model for blind inverse problems with
application to deconvolution [0.0]
Current methods assume the degradation to be known and provide impressive results in terms of restoration and diversity.
In this work, we leverage the efficiency of those models to jointly estimate the restored image and unknown parameters of the kernel model.
Our method alternates between approximating the expected log-likelihood of the problem using samples drawn from a diffusion model and a step to estimate unknown model parameters.
arXiv Detail & Related papers (2023-09-01T06:47:13Z) - Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness.
We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
arXiv Detail & Related papers (2023-03-06T11:51:28Z) - Soft Diffusion: Score Matching for General Corruptions [84.26037497404195]
We propose a new objective called Soft Score Matching that provably learns the score function for any linear corruption process.
We show that our objective learns the gradient of the likelihood under suitable regularity conditions for the family of corruption processes.
Our method achieves state-of-the-art FID score $1.85$ on CelebA-64, outperforming all previous linear diffusion models.
arXiv Detail & Related papers (2022-09-12T17:45:03Z) - Perceptual Image Restoration with High-Quality Priori and Degradation
Learning [28.93489249639681]
We show that our model performs well in measuring the similarity between restored and degraded images.
Our simultaneous restoration and enhancement framework generalizes well to real-world complicated degradation types.
arXiv Detail & Related papers (2021-03-04T13:19:50Z) - Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
Blind image restoration is a common yet challenging problem in computer vision.
We propose a novel blind image restoration method, aiming to integrate both the advantages of them.
Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
arXiv Detail & Related papers (2020-08-25T03:30:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.