AdaDM: Enabling Normalization for Image Super-Resolution
- URL: http://arxiv.org/abs/2111.13905v1
- Date: Sat, 27 Nov 2021 14:16:11 GMT
- Title: AdaDM: Enabling Normalization for Image Super-Resolution
- Authors: Jie Liu, Jie Tang, Gangshan Wu
- Abstract summary: In fidelity image Super-Resolution (SR), it is believed that normalization layers get rid of range flexibility by normalizing the features and they are simply removed from modern SR networks.
We found that the standard deviation of the residual feature shrinks a lot after normalization layers, which causes the performance degradation in SR networks.
We propose an Adaptive Deviation Modulator (AdaDM) in which a modulation factor is adaptively predicted to amplify the pixel deviation.
- Score: 31.033444663076498
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Normalization like Batch Normalization (BN) is a milestone technique to
normalize the distributions of intermediate layers in deep learning, enabling
faster training and better generalization accuracy. However, in fidelity image
Super-Resolution (SR), it is believed that normalization layers get rid of
range flexibility by normalizing the features and they are simply removed from
modern SR networks. In this paper, we study this phenomenon quantitatively and
qualitatively. We found that the standard deviation of the residual feature
shrinks a lot after normalization layers, which causes the performance
degradation in SR networks. Standard deviation reflects the amount of variation
of pixel values. When the variation becomes smaller, the edges will become less
discriminative for the network to resolve. To address this problem, we propose
an Adaptive Deviation Modulator (AdaDM), in which a modulation factor is
adaptively predicted to amplify the pixel deviation. For better generalization
performance, we apply BN in state-of-the-art SR networks with the proposed
AdaDM. Meanwhile, the deviation amplification strategy in AdaDM makes the edge
information in the feature more distinguishable. As a consequence, SR networks
with BN and our AdaDM can get substantial performance improvements on benchmark
datasets. Extensive experiments have been conducted to show the effectiveness
of our method.
Related papers
- Invariant Representation via Decoupling Style and Spurious Features from Images [27.965593857283316]
This paper considers the out-of-distribution (OOD) generalization problem under the setting that both style distribution shift and spurious features exist and domain labels are missing.
We propose a structural causal model (SCM) for the image generation process, which captures both style distribution shift and spurious features.
The proposed SCM enables us to design a new framework called IRSS, which can gradually separate style distribution and spurious features from images.
arXiv Detail & Related papers (2023-12-11T09:14:42Z) - Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networks [53.23803932357899]
quantization leads to accuracy loss in image super-resolution (SR) networks.
Existing works address this distribution mismatch problem by dynamically adapting quantization ranges during test time.
We propose a new quantization-aware training scheme that effectively Overcomes the Distribution Mismatch problem in SR networks.
arXiv Detail & Related papers (2023-07-25T08:50:01Z) - Blind Super-Resolution for Remote Sensing Images via Conditional
Stochastic Normalizing Flows [14.882417028542855]
We propose a novel blind SR framework based on the normalizing flow (BlindSRSNF) to address the above problems.
BlindSRSNF learns the conditional probability distribution over the high-resolution image space given a low-resolution (LR) image by explicitly optimizing the variational bound on the likelihood.
We show that the proposed algorithm can obtain SR results with excellent visual perception quality on both simulated LR and real-world RSIs.
arXiv Detail & Related papers (2022-10-14T12:37:32Z) - Meta-Learning based Degradation Representation for Blind
Super-Resolution [54.93926549648434]
We propose a Meta-Learning based Region Degradation Aware SR Network (MRDA)
We use the MRDA to rapidly adapt to the specific complex degradation after several iterations and extract implicit degradation information.
A teacher network MRDA$_T$ is designed to further utilize the degradation information extracted by MLN for SR.
arXiv Detail & Related papers (2022-07-28T09:03:00Z) - CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolution [55.50793823060282]
We propose a novel Content-Aware Dynamic Quantization (CADyQ) method for image super-resolution (SR) networks.
CADyQ allocates optimal bits to local regions and layers adaptively based on the local contents of an input image.
The pipeline has been tested on various SR networks and evaluated on several standard benchmarks.
arXiv Detail & Related papers (2022-07-21T07:50:50Z) - Improving Super-Resolution Performance using Meta-Attention Layers [17.870338228921327]
Convolutional Neural Networks (CNNs) have achieved impressive results across many super-resolution (SR) and image restoration tasks.
Ill-posed nature of SR can make it difficult to accurately super-resolve an image which has undergone multiple different degradations.
We introduce meta-attention, a mechanism which allows any SR CNN to exploit the information available in relevant degradation parameters.
arXiv Detail & Related papers (2021-10-27T09:20:21Z) - Style Normalization and Restitution for DomainGeneralization and
Adaptation [88.86865069583149]
An effective domain generalizable model is expected to learn feature representations that are both generalizable and discriminative.
In this paper, we design a novel Style Normalization and Restitution module (SNR) to ensure both high generalization and discrimination capability of the networks.
arXiv Detail & Related papers (2021-01-03T09:01:39Z) - SRFlow: Learning the Super-Resolution Space with Normalizing Flow [176.07982398988747]
Super-resolution is an ill-posed problem, since it allows for multiple predictions for a given low-resolution image.
We propose SRFlow: a normalizing flow based super-resolution method capable of learning the conditional distribution of the output.
Our model is trained in a principled manner using a single loss, namely the negative log-likelihood.
arXiv Detail & Related papers (2020-06-25T06:34:04Z) - Iterative Network for Image Super-Resolution [69.07361550998318]
Single image super-resolution (SISR) has been greatly revitalized by the recent development of convolutional neural networks (CNN)
This paper provides a new insight on conventional SISR algorithm, and proposes a substantially different approach relying on the iterative optimization.
A novel iterative super-resolution network (ISRN) is proposed on top of the iterative optimization.
arXiv Detail & Related papers (2020-05-20T11:11:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.