Scale-Aware Dynamic Network for Continuous-Scale Super-Resolution
- URL: http://arxiv.org/abs/2110.15655v1
- Date: Fri, 29 Oct 2021 09:57:48 GMT
- Title: Scale-Aware Dynamic Network for Continuous-Scale Super-Resolution
- Authors: Hanlin Wu, Ning Ni, Libao Zhang
- Abstract summary: We propose a scale-aware dynamic network (SADN) for continuous-scale SR.
First, we propose a scale-aware dynamic convolutional (SAD-Conv) layer for the feature learning of multiple SR tasks with various scales.
Second, we devise a continuous-scale upsampling module (CSUM) with the multi-bilinear local implicit function (MBLIF) for any-scale upsampling.
- Score: 16.67263192454279
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Single-image super-resolution (SR) with fixed and discrete scale factors has
achieved great progress due to the development of deep learning technology.
However, the continuous-scale SR, which aims to use a single model to process
arbitrary (integer or non-integer) scale factors, is still a challenging task.
The existing SR models generally adopt static convolution to extract features,
and thus unable to effectively perceive the change of scale factor, resulting
in limited generalization performance on multi-scale SR tasks. Moreover, the
existing continuous-scale upsampling modules do not make full use of
multi-scale features and face problems such as checkerboard artifacts in the SR
results and high computational complexity. To address the above problems, we
propose a scale-aware dynamic network (SADN) for continuous-scale SR. First, we
propose a scale-aware dynamic convolutional (SAD-Conv) layer for the feature
learning of multiple SR tasks with various scales. The SAD-Conv layer can
adaptively adjust the attention weights of multiple convolution kernels based
on the scale factor, which enhances the expressive power of the model with a
negligible extra computational cost. Second, we devise a continuous-scale
upsampling module (CSUM) with the multi-bilinear local implicit function
(MBLIF) for any-scale upsampling. The CSUM constructs multiple feature spaces
with gradually increasing scales to approximate the continuous feature
representation of an image, and then the MBLIF makes full use of multi-scale
features to map arbitrary coordinates to RGB values in high-resolution space.
We evaluate our SADN using various benchmarks. The experimental results show
that the CSUM can replace the previous fixed-scale upsampling layers and obtain
a continuous-scale SR network while maintaining performance. Our SADN uses much
fewer parameters and outperforms the state-of-the-art SR methods.
Related papers
- Task-Aware Dynamic Transformer for Efficient Arbitrary-Scale Image Super-Resolution [8.78015409192613]
Arbitrary-scale super-resolution (ASSR) aims to learn a single model for image super-resolution at arbitrary magnifying scales.
Existing ASSR networks typically comprise an off-the-shelf scale-agnostic feature extractor and an arbitrary scale upsampler.
We propose a Task-Aware Dynamic Transformer (TADT) as an input-adaptive feature extractor for efficient image ASSR.
arXiv Detail & Related papers (2024-08-16T13:35:52Z) - AnySR: Realizing Image Super-Resolution as Any-Scale, Any-Resource [84.74855803555677]
We introduce AnySR, to rebuild existing arbitrary-scale SR methods into any-scale, any-resource implementation.
Our AnySR innovates in: 1) building arbitrary-scale tasks as any-resource implementation, reducing resource requirements for smaller scales without additional parameters; 2) enhancing any-scale performance in a feature-interweaving fashion.
Results show that our AnySR implements SISR tasks in a computing-more-efficient fashion, and performs on par with existing arbitrary-scale SISR methods.
arXiv Detail & Related papers (2024-07-05T04:00:14Z) - Frequency-Assisted Mamba for Remote Sensing Image Super-Resolution [49.902047563260496]
We develop the first attempt to integrate the Vision State Space Model (Mamba) for remote sensing image (RSI) super-resolution.
To achieve better SR reconstruction, building upon Mamba, we devise a Frequency-assisted Mamba framework, dubbed FMSR.
Our FMSR features a multi-level fusion architecture equipped with the Frequency Selection Module (FSM), Vision State Space Module (VSSM), and Hybrid Gate Module (HGM)
arXiv Detail & Related papers (2024-05-08T11:09:24Z) - CiaoSR: Continuous Implicit Attention-in-Attention Network for
Arbitrary-Scale Image Super-Resolution [158.2282163651066]
This paper proposes a continuous implicit attention-in-attention network, called CiaoSR.
We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features.
We embed a scale-aware attention in this implicit attention network to exploit additional non-local information.
arXiv Detail & Related papers (2022-12-08T15:57:46Z) - Lightweight Stepless Super-Resolution of Remote Sensing Images via
Saliency-Aware Dynamic Routing Strategy [15.587621728422414]
Deep learning algorithms have greatly improved the performance of remote sensing image (RSI) super-resolution (SR)
However, increasing network depth and parameters cause a huge burden of computing and storage.
We propose a saliency-aware dynamic routing network (SalDRN) for lightweight and stepless SR of RSIs.
arXiv Detail & Related papers (2022-10-14T07:49:03Z) - Accurate and Lightweight Image Super-Resolution with Model-Guided Deep
Unfolding Network [63.69237156340457]
We present and advocate an explainable approach toward SISR named model-guided deep unfolding network (MoG-DUN)
MoG-DUN is accurate (producing fewer aliasing artifacts), computationally efficient (with reduced model parameters), and versatile (capable of handling multiple degradations)
The superiority of the proposed MoG-DUN method to existing state-of-theart image methods including RCAN, SRDNF, and SRFBN is substantiated by extensive experiments on several popular datasets and various degradation scenarios.
arXiv Detail & Related papers (2020-09-14T08:23:37Z) - Sequential Hierarchical Learning with Distribution Transformation for
Image Super-Resolution [83.70890515772456]
We build a sequential hierarchical learning super-resolution network (SHSR) for effective image SR.
We consider the inter-scale correlations of features, and devise a sequential multi-scale block (SMB) to progressively explore the hierarchical information.
Experiment results show SHSR achieves superior quantitative performance and visual quality to state-of-the-art methods.
arXiv Detail & Related papers (2020-07-19T01:35:53Z) - Crowd Counting via Hierarchical Scale Recalibration Network [61.09833400167511]
We propose a novel Hierarchical Scale Recalibration Network (HSRNet) to tackle the task of crowd counting.
HSRNet models rich contextual dependencies and recalibrating multiple scale-associated information.
Our approach can ignore various noises selectively and focus on appropriate crowd scales automatically.
arXiv Detail & Related papers (2020-03-07T10:06:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.