Deep learning architectural designs for super-resolution of noisy images
- URL: http://arxiv.org/abs/2102.05105v1
- Date: Tue, 9 Feb 2021 20:09:42 GMT
- Title: Deep learning architectural designs for super-resolution of noisy images
- Authors: Angel Villar-Corrales, Franziska Schirrmacher and Christian Riess
- Abstract summary: In this work, we propose to jointly perform denoising and super-resolution.
We investigate two architectural designs: "in-network" combines both tasks at feature level, while "pre-network" first performs denoising and then super-resolution.
The pre-network design exhibits superior performance on unseen types of image corruption, which is a pathological failure case of existing super-resolution models.
- Score: 7.657378889055478
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recent advances in deep learning have led to significant improvements in
single image super-resolution (SR) research. However, due to the amplification
of noise during the upsampling steps, state-of-the-art methods often fail at
reconstructing high-resolution images from noisy versions of their
low-resolution counterparts. However, this is especially important for images
from unknown cameras with unseen types of image degradation. In this work, we
propose to jointly perform denoising and super-resolution. To this end, we
investigate two architectural designs: "in-network" combines both tasks at
feature level, while "pre-network" first performs denoising and then
super-resolution. Our experiments show that both variants have specific
advantages: The in-network design obtains the strongest results when the type
of image corruption is aligned in the training and testing dataset, for any
choice of denoiser. The pre-network design exhibits superior performance on
unseen types of image corruption, which is a pathological failure case of
existing super-resolution models. We hope that these findings help to enable
super-resolution also in less constrained scenarios where source camera or
imaging conditions are not well controlled. Source code and pretrained models
are available at https://github.com/
angelvillar96/super-resolution-noisy-images.
Related papers
- ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with
Diffusion Models [126.35334860896373]
We investigate the capability of generating images from pre-trained diffusion models at much higher resolutions than the training image sizes.
Existing works for higher-resolution generation, such as attention-based and joint-diffusion approaches, cannot well address these issues.
We propose a simple yet effective re-dilation that can dynamically adjust the convolutional perception field during inference.
arXiv Detail & Related papers (2023-10-11T17:52:39Z) - The Devil is in the Upsampling: Architectural Decisions Made Simpler for
Denoising with Deep Image Prior [20.1435183909115]
Deep Image Prior (DIP) shows that some network architectures naturally bias towards smooth images and resist noises.
Although DIP has removed the requirement of large training sets, it still presents two practical challenges for denoising: architectural design and noise-fitting.
In this study, we analyze from a frequency perspective to demonstrate that the unlearnt upsampling is the main driving force behind the denoising phenomenon in DIP.
arXiv Detail & Related papers (2023-04-22T13:50:27Z) - Masked Image Training for Generalizable Deep Image Denoising [53.03126421917465]
We present a novel approach to enhance the generalization performance of denoising networks.
Our method involves masking random pixels of the input image and reconstructing the missing information during training.
Our approach exhibits better generalization ability than other deep learning models and is directly applicable to real-world scenarios.
arXiv Detail & Related papers (2023-03-23T09:33:44Z) - Exploring Resolution and Degradation Clues as Self-supervised Signal for
Low Quality Object Detection [77.3530907443279]
We propose a novel self-supervised framework to detect objects in degraded low resolution images.
Our methods has achieved superior performance compared with existing methods when facing variant degradation situations.
arXiv Detail & Related papers (2022-08-05T09:36:13Z) - Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis [148.16279746287452]
We propose a swin-conv block to incorporate the local modeling ability of residual convolutional layer and non-local modeling ability of swin transformer block.
For the training data synthesis, we design a practical noise degradation model which takes into consideration different kinds of noise.
Experiments on AGWN removal and real image denoising demonstrate that the new network architecture design achieves state-of-the-art performance.
arXiv Detail & Related papers (2022-03-24T18:11:31Z) - Exploiting Raw Images for Real-Scene Super-Resolution [105.18021110372133]
We study the problem of real-scene single image super-resolution to bridge the gap between synthetic data and real captured images.
We propose a method to generate more realistic training data by mimicking the imaging process of digital cameras.
We also develop a two-branch convolutional neural network to exploit the radiance information originally-recorded in raw images.
arXiv Detail & Related papers (2021-02-02T16:10:15Z) - Blind Image Super-Resolution with Spatial Context Hallucination [5.849485167287474]
We propose a novel Spatial Context Hallucination Network (SCHN) for blind super-resolution without knowing the degradation kernel.
We train our model on two high quality datasets, DIV2K and Flickr2K.
Our method performs better than state-of-the-art methods when input images are corrupted with random blur and noise.
arXiv Detail & Related papers (2020-09-25T22:36:07Z) - Unsupervised Real Image Super-Resolution via Generative Variational
AutoEncoder [47.53609520395504]
We revisit the classic example based image super-resolution approaches and come up with a novel generative model for perceptual image super-resolution.
We propose a joint image denoising and super-resolution model via Variational AutoEncoder.
With the aid of the discriminator, an additional overhead of super-resolution subnetwork is attached to super-resolve the denoised image with photo-realistic visual quality.
arXiv Detail & Related papers (2020-04-27T13:49:36Z) - Stochastic Frequency Masking to Improve Super-Resolution and Denoising
Networks [20.23203428843382]
We present an analysis, in the frequency domain, of degradation- Kernel overfitting in super-resolution.
We propose a frequency masking of images used in training to regularize the networks and address the overfitting problem.
Our technique improves state-of-the-art methods on blind super-resolution with different synthetic kernels, real super-resolution, blind Gaussian denoising, and real-image denoising.
arXiv Detail & Related papers (2020-03-16T11:21:20Z) - Gated Fusion Network for Degraded Image Super Resolution [78.67168802945069]
We propose a dual-branch convolutional neural network to extract base features and recovered features separately.
By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process.
arXiv Detail & Related papers (2020-03-02T13:28:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.