Diverse super-resolution with pretrained deep hiererarchical VAEs
- URL: http://arxiv.org/abs/2205.10347v4
- Date: Tue, 9 Jan 2024 14:27:13 GMT
- Title: Diverse super-resolution with pretrained deep hiererarchical VAEs
- Authors: Jean Prost, Antoine Houdard, Andr\'es Almansa and Nicolas Papadakis
- Abstract summary: We investigate the problem of producing diverse solutions to an image super-resolution problem.
We train a lightweight encoder to encode low-resolution images in the latent space of a pretrained HVAE.
At inference, we combine the low-resolution encoder and the pretrained generative model to super-resolve an image.
- Score: 6.257821009472099
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We investigate the problem of producing diverse solutions to an image
super-resolution problem. From a probabilistic perspective, this can be done by
sampling from the posterior distribution of an inverse problem, which requires
the definition of a prior distribution on the high-resolution images. In this
work, we propose to use a pretrained hierarchical variational autoencoder
(HVAE) as a prior. We train a lightweight stochastic encoder to encode
low-resolution images in the latent space of a pretrained HVAE. At inference,
we combine the low-resolution encoder and the pretrained generative model to
super-resolve an image. We demonstrate on the task of face super-resolution
that our method provides an advantageous trade-off between the computational
efficiency of conditional normalizing flows techniques and the sample quality
of diffusion based methods.
Related papers
- Efficient Conditional Diffusion Model with Probability Flow Sampling for Image Super-resolution [35.55094110634178]
We propose an efficient conditional diffusion model with probability flow sampling for image super-resolution.
Our method achieves higher super-resolution quality than existing diffusion-based image super-resolution methods.
arXiv Detail & Related papers (2024-04-16T16:08:59Z) - Solving General Noisy Inverse Problem via Posterior Sampling: A Policy Gradient Viewpoint [21.22750301965104]
We leverage a pretrained diffusion generative model to solve a wide range of image inverse tasks without task specific model fine-tuning.
To precisely estimate the guidance score function of the input image, we propose Diffusion Policy Gradient (DPG)
Experiments show that our method is robust to both Gaussian and Poisson noise degradation on multiple linear and non-linear inverse tasks.
arXiv Detail & Related papers (2024-03-15T16:38:47Z) - ACDMSR: Accelerated Conditional Diffusion Models for Single Image
Super-Resolution [84.73658185158222]
We propose a diffusion model-based super-resolution method called ACDMSR.
Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process.
Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
arXiv Detail & Related papers (2023-07-03T06:49:04Z) - Solving Linear Inverse Problems Provably via Posterior Sampling with
Latent Diffusion Models [98.95988351420334]
We present the first framework to solve linear inverse problems leveraging pre-trained latent diffusion models.
We theoretically analyze our algorithm showing provable sample recovery in a linear model setting.
arXiv Detail & Related papers (2023-07-02T17:21:30Z) - A Variational Perspective on Solving Inverse Problems with Diffusion
Models [101.831766524264]
Inverse tasks can be formulated as inferring a posterior distribution over data.
This is however challenging in diffusion models since the nonlinear and iterative nature of the diffusion process renders the posterior intractable.
We propose a variational approach that by design seeks to approximate the true posterior distribution.
arXiv Detail & Related papers (2023-05-07T23:00:47Z) - Deep Model-Based Super-Resolution with Non-uniform Blur [1.7188280334580197]
We propose a state-of-the-art method for super-resolution with non-uniform blur.
We first propose a fast deep plug-and-play algorithm, based on linearized ADMM splitting techniques.
We unfold our iterative algorithm into a single network and train it end-to-end.
arXiv Detail & Related papers (2022-04-21T13:57:21Z) - Uncovering the Over-smoothing Challenge in Image Super-Resolution: Entropy-based Quantification and Contrastive Optimization [67.99082021804145]
We propose an explicit solution to the COO problem, called Detail Enhanced Contrastive Loss (DECLoss)
DECLoss utilizes the clustering property of contrastive learning to directly reduce the variance of the potential high-resolution distribution.
We evaluate DECLoss on multiple super-resolution benchmarks and demonstrate that it improves the perceptual quality of PSNR-oriented models.
arXiv Detail & Related papers (2022-01-04T08:30:09Z) - Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
Blind image restoration is a common yet challenging problem in computer vision.
We propose a novel blind image restoration method, aiming to integrate both the advantages of them.
Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
arXiv Detail & Related papers (2020-08-25T03:30:53Z) - SRFlow: Learning the Super-Resolution Space with Normalizing Flow [176.07982398988747]
Super-resolution is an ill-posed problem, since it allows for multiple predictions for a given low-resolution image.
We propose SRFlow: a normalizing flow based super-resolution method capable of learning the conditional distribution of the output.
Our model is trained in a principled manner using a single loss, namely the negative log-likelihood.
arXiv Detail & Related papers (2020-06-25T06:34:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.