One-to-many Approach for Improving Super-Resolution
- URL: http://arxiv.org/abs/2106.10437v2
- Date: Tue, 22 Jun 2021 01:25:11 GMT
- Title: One-to-many Approach for Improving Super-Resolution
- Authors: Sieun Park, Eunho Lee
- Abstract summary: We propose adding weighted pixel-wise noise after every Residual-in-Residual Dense Block (RRDB) to enable the generator to generate various images.
We modify the strict content loss to not penalize variation in reconstructed images as long as it has consistent content.
We were able to improve the performance of ESRGAN in x4 perceptual SR and achieve the state-of-the-art LPIPS score in x16 perceptual extreme SR.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Super-resolution (SR) is a one-to-many task with multiple possible solutions.
However, previous works were not concerned about this characteristic. For a
one-to-many pipeline, the generator should be able to generate multiple
estimates of the reconstruction, and not be penalized for generating similar
and equally realistic images. To achieve this, we propose adding weighted
pixel-wise noise after every Residual-in-Residual Dense Block (RRDB) to enable
the generator to generate various images. We modify the strict content loss to
not penalize the stochastic variation in reconstructed images as long as it has
consistent content. Additionally, we observe that there are out-of-focus
regions in the DIV2K, DIV8K datasets that provide unhelpful guidelines. We
filter blurry regions in the training data using the method of [10]. Finally,
we modify the discriminator to receive the low-resolution image as a reference
image along with the target image to provide better feedback to the generator.
Using our proposed methods, we were able to improve the performance of ESRGAN
in x4 perceptual SR and achieve the state-of-the-art LPIPS score in x16
perceptual extreme SR.
Related papers
- DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior [70.46245698746874]
We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks.
DiffBIR decouples blind image restoration problem into two stages: 1) degradation removal: removing image-independent content; 2) information regeneration: generating the lost image content.
In the first stage, we use restoration modules to remove degradations and obtain high-fidelity restored results.
For the second stage, we propose IRControlNet that leverages the generative ability of latent diffusion models to generate realistic details.
arXiv Detail & Related papers (2023-08-29T07:11:52Z) - Rethinking the Paradigm of Content Constraints in Unpaired
Image-to-Image Translation [9.900050049833986]
We propose EnCo, a simple but efficient way to maintain the content by constraining the representational similarity in the latent space of patch-level features.
For the similarity function, we use a simple MSE loss instead of contrastive loss, which is currently widely used in I2I tasks.
In addition, we rethink the role played by discriminators in sampling patches and propose a discnative attention-guided (DAG) patch sampling strategy to replace random sampling.
arXiv Detail & Related papers (2022-11-20T04:39:57Z) - Learning Multiple Probabilistic Degradation Generators for Unsupervised
Real World Image Super Resolution [5.987801889633082]
Unsupervised real world super resolution aims at restoring high-resolution (HR) images given low-resolution (LR) inputs when paired data is unavailable.
One of the most common approaches is synthesizing noisy LR images using GANs and utilizing a synthetic dataset to train the model in a supervised manner.
We propose a probabilistic degradation generator to approximate the distribution of LR images given a HR image.
arXiv Detail & Related papers (2022-01-26T04:49:11Z) - Multi-Attention Generative Adversarial Network for Remote Sensing Image
Super-Resolution [17.04588012373861]
Image super-resolution (SR) methods can generate remote sensing images with high spatial resolution without increasing the cost.
We propose a network based on the generative adversarial network (GAN) to generate high resolution remote sensing images.
arXiv Detail & Related papers (2021-07-14T08:06:19Z) - LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single
Image Super-Resolution and Beyond [75.37541439447314]
Single image super-resolution (SISR) deals with a fundamental problem of upsampling a low-resolution (LR) image to its high-resolution (HR) version.
This paper proposes a linearly-assembled pixel-adaptive regression network (LAPAR) to strike a sweet spot of deep model complexity and resulting SISR quality.
arXiv Detail & Related papers (2021-05-21T15:47:18Z) - SRWarp: Generalized Image Super-Resolution under Arbitrary
Transformation [65.88321755969677]
Deep CNNs have achieved significant successes in image processing and its applications, including single image super-resolution.
Recent approaches extend the scope to real-valued upsampling factors.
We propose the SRWarp framework to further generalize the SR tasks toward an arbitrary image transformation.
arXiv Detail & Related papers (2021-04-21T02:50:41Z) - VSpSR: Explorable Super-Resolution via Variational Sparse Representation [15.810502797317502]
Super-resolution (SR) is an ill-posed problem, which means that infinitely many high-resolution (HR) images can be degraded to the same low-resolution (LR) image.
We develop a Vari Sparseational framework for Super-Resolution (VSpSR) via neural networks.
arXiv Detail & Related papers (2021-04-17T15:36:24Z) - Adversarial Generation of Continuous Images [31.92891885615843]
In this paper, we propose two novel architectural techniques for building INR-based image decoders.
We use them to build a state-of-the-art continuous image GAN.
Our proposed INR-GAN architecture improves the performance of continuous image generators by several times.
arXiv Detail & Related papers (2020-11-24T11:06:40Z) - Exploiting Deep Generative Prior for Versatile Image Restoration and
Manipulation [181.08127307338654]
This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images.
The deep generative prior (DGP) provides compelling results to restore missing semantics, e.g., color, patch, resolution, of various degraded images.
arXiv Detail & Related papers (2020-03-30T17:45:07Z) - PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of
Generative Models [77.32079593577821]
PULSE (Photo Upsampling via Latent Space Exploration) generates high-resolution, realistic images at resolutions previously unseen in the literature.
Our method outperforms state-of-the-art methods in perceptual quality at higher resolutions and scale factors than previously possible.
arXiv Detail & Related papers (2020-03-08T16:44:31Z) - Image Fine-grained Inpainting [89.17316318927621]
We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields.
To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss.
We also employ a discriminator with local and global branches to ensure local-global contents consistency.
arXiv Detail & Related papers (2020-02-07T03:45:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.