Perception-Distortion Trade-off in the SR Space Spanned by Flow Models
- URL: http://arxiv.org/abs/2209.08564v1
- Date: Sun, 18 Sep 2022 13:12:21 GMT
- Title: Perception-Distortion Trade-off in the SR Space Spanned by Flow Models
- Authors: Cansu Korkmaz, A.Murat Tekalp, Zafer Dogan, Erkut Erdem, Aykut Erdem
- Abstract summary: Flow-based generative super-resolution (SR) models learn to produce a diverse set of feasible SR solutions, called the SR space.
We present a simple but effective image ensembling/fusion approach to obtain a single SR image eliminating random artifacts and improving fidelity without significantly compromising perceptual quality.
- Score: 21.597478894658263
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Flow-based generative super-resolution (SR) models learn to produce a diverse
set of feasible SR solutions, called the SR space. Diversity of SR solutions
increases with the temperature ($\tau$) of latent variables, which introduces
random variations of texture among sample solutions, resulting in visual
artifacts and low fidelity. In this paper, we present a simple but effective
image ensembling/fusion approach to obtain a single SR image eliminating random
artifacts and improving fidelity without significantly compromising perceptual
quality. We achieve this by benefiting from a diverse set of feasible
photo-realistic solutions in the SR space spanned by flow models. We propose
different image ensembling and fusion strategies which offer multiple paths to
move sample solutions in the SR space to more desired destinations in the
perception-distortion plane in a controllable manner depending on the fidelity
vs. perceptual quality requirements of the task at hand. Experimental results
demonstrate that our image ensembling/fusion strategy achieves more promising
perception-distortion trade-off compared to sample SR images produced by flow
models and adversarially trained models in terms of both quantitative metrics
and visual quality.
Related papers
- Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors [75.24313405671433]
Diffusion-based image super-resolution (SR) methods have achieved remarkable success by leveraging large pre-trained text-to-image diffusion models as priors.
We introduce a novel one-step SR model, which significantly addresses the efficiency issue of diffusion-based SR methods.
Unlike existing fine-tuning strategies, we designed a degradation-guided Low-Rank Adaptation (LoRA) module specifically for SR.
arXiv Detail & Related papers (2024-09-25T16:15:21Z) - Semantic Guided Large Scale Factor Remote Sensing Image Super-resolution with Generative Diffusion Prior [13.148815217684277]
Large scale factor super-resolution (SR) algorithms are vital for maximizing the utilization of low-resolution (LR) satellite data captured from orbit.
Existing methods confront challenges in recovering SR images with clear textures and correct ground objects.
We introduce a novel framework, the Semantic Guided Diffusion Model (SGDM), designed for large scale factor remote sensing image super-resolution.
arXiv Detail & Related papers (2024-05-11T16:06:16Z) - Trustworthy SR: Resolving Ambiguity in Image Super-resolution via
Diffusion Models and Human Feedback [5.665865832321032]
Super-resolution (SR) is an ill-posed inverse problem with a large set of feasible solutions that are consistent with a given low-resolution image.
We propose employing human feedback, where we ask human subjects to select a small number of likely samples and we ensemble the averages of selected samples.
Our proposed strategy provides more trustworthy solutions when compared to state-of-the art SR methods.
arXiv Detail & Related papers (2024-02-12T11:55:02Z) - Deep Equilibrium Diffusion Restoration with Parallel Sampling [120.15039525209106]
Diffusion model-based image restoration (IR) aims to use diffusion models to recover high-quality (HQ) images from degraded images, achieving promising performance.
Most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs.
In this work, we aim to rethink the diffusion model-based IR models through a different perspective, i.e., a deep equilibrium (DEQ) fixed point system, called DeqIR.
arXiv Detail & Related papers (2023-11-20T08:27:56Z) - ACDMSR: Accelerated Conditional Diffusion Models for Single Image
Super-Resolution [84.73658185158222]
We propose a diffusion model-based super-resolution method called ACDMSR.
Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process.
Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
arXiv Detail & Related papers (2023-07-03T06:49:04Z) - Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution [82.50210340928173]
randomness of diffusion models results in ineffectiveness and instability, making it challenging for users to guarantee the quality of SR results.
We propose a plug-and-play sampling method that owns the potential to benefit a series of diffusion-based SR methods.
The quality of SR results sampled by the proposed method with fewer steps outperforms the quality of results sampled by current methods with randomness from the same pre-trained diffusion-based SR model.
arXiv Detail & Related papers (2023-05-24T17:09:54Z) - Super-resolution Reconstruction of Single Image for Latent features [8.857209365343646]
Single-image super-resolution (SISR) typically focuses on restoring various degraded low-resolution (LR) images to a single high-resolution (HR) image.
It is often challenging for models to simultaneously maintain high quality and rapid sampling while preserving diversity in details and texture features.
This challenge can lead to issues such as model collapse, lack of rich details and texture features in the reconstructed HR images, and excessive time consumption for model sampling.
arXiv Detail & Related papers (2022-11-16T09:37:07Z) - Blind Super-Resolution for Remote Sensing Images via Conditional
Stochastic Normalizing Flows [14.882417028542855]
We propose a novel blind SR framework based on the normalizing flow (BlindSRSNF) to address the above problems.
BlindSRSNF learns the conditional probability distribution over the high-resolution image space given a low-resolution (LR) image by explicitly optimizing the variational bound on the likelihood.
We show that the proposed algorithm can obtain SR results with excellent visual perception quality on both simulated LR and real-world RSIs.
arXiv Detail & Related papers (2022-10-14T12:37:32Z) - Hierarchical Similarity Learning for Aliasing Suppression Image
Super-Resolution [64.15915577164894]
A hierarchical image super-resolution network (HSRNet) is proposed to suppress the influence of aliasing.
HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.
arXiv Detail & Related papers (2022-06-07T14:55:32Z) - SRWarp: Generalized Image Super-Resolution under Arbitrary
Transformation [65.88321755969677]
Deep CNNs have achieved significant successes in image processing and its applications, including single image super-resolution.
Recent approaches extend the scope to real-valued upsampling factors.
We propose the SRWarp framework to further generalize the SR tasks toward an arbitrary image transformation.
arXiv Detail & Related papers (2021-04-21T02:50:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.