Transforming Image Super-Resolution: A ConvFormer-based Efficient
Approach
- URL: http://arxiv.org/abs/2401.05633v1
- Date: Thu, 11 Jan 2024 03:08:00 GMT
- Title: Transforming Image Super-Resolution: A ConvFormer-based Efficient
Approach
- Authors: Gang Wu, Junjun Jiang, Junpeng Jiang, Xianming Liu
- Abstract summary: We introduce the Convolutional Transformer layer (ConvFormer) and the ConvFormer-based Super-Resolution network (CFSR)
CFSR efficiently models long-range dependencies and extensive receptive fields with a slight computational cost.
It achieves 0.39 dB gains on Urban100 dataset for x2 SR task while containing 26% and 31% fewer parameters and FLOPs, respectively.
- Score: 63.98380888730723
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent progress in single-image super-resolution (SISR) has achieved
remarkable performance, yet the computational costs of these methods remain a
challenge for deployment on resource-constrained devices. Especially for
transformer-based methods, the self-attention mechanism in such models brings
great breakthroughs while incurring substantial computational costs. To tackle
this issue, we introduce the Convolutional Transformer layer (ConvFormer) and
the ConvFormer-based Super-Resolution network (CFSR), which offer an effective
and efficient solution for lightweight image super-resolution tasks. In detail,
CFSR leverages the large kernel convolution as the feature mixer to replace the
self-attention module, efficiently modeling long-range dependencies and
extensive receptive fields with a slight computational cost. Furthermore, we
propose an edge-preserving feed-forward network, simplified as EFN, to obtain
local feature aggregation and simultaneously preserve more high-frequency
information. Extensive experiments demonstrate that CFSR can achieve an
advanced trade-off between computational cost and performance when compared to
existing lightweight SR methods. Compared to state-of-the-art methods, e.g.
ShuffleMixer, the proposed CFSR achieves 0.39 dB gains on Urban100 dataset for
x2 SR task while containing 26% and 31% fewer parameters and FLOPs,
respectively. Code and pre-trained models are available at
https://github.com/Aitical/CFSR.
Related papers
- Unfolding Once is Enough: A Deployment-Friendly Transformer Unit for
Super-Resolution [16.54421804141835]
High resolution of intermediate features in SISR models increases memory and computational requirements.
We propose a Deployment-friendly Inner-patch Transformer Network (DITN) for the SISR task.
Our models can achieve competitive results in terms of qualitative and quantitative performance with high deployment efficiency.
arXiv Detail & Related papers (2023-08-05T05:42:51Z) - Incorporating Transformer Designs into Convolutions for Lightweight
Image Super-Resolution [46.32359056424278]
Large convolutional kernels have become popular in designing convolutional neural networks.
The increase in kernel size also leads to a quadratic growth in the number of parameters, resulting in heavy computation and memory requirements.
We propose a neighborhood attention (NA) module that upgrades the standard convolution with a self-attention mechanism.
Building upon the NA module, we propose a lightweight single image super-resolution (SISR) network named TCSR.
arXiv Detail & Related papers (2023-03-25T01:32:18Z) - Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures.
This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead.
We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
arXiv Detail & Related papers (2023-03-16T21:06:13Z) - Spatially-Adaptive Feature Modulation for Efficient Image
Super-Resolution [90.16462805389943]
We develop a spatially-adaptive feature modulation (SAFM) mechanism upon a vision transformer (ViT)-like block.
Proposed method is $3times$ smaller than state-of-the-art efficient SR methods.
arXiv Detail & Related papers (2023-02-27T14:19:31Z) - ShuffleMixer: An Efficient ConvNet for Image Super-Resolution [88.86376017828773]
We propose ShuffleMixer, for lightweight image super-resolution that explores large convolution and channel split-shuffle operation.
Specifically, we develop a large depth-wise convolution and two projection layers based on channel splitting and shuffling as the basic component to mix features efficiently.
Experimental results demonstrate that the proposed ShuffleMixer is about 6x smaller than the state-of-the-art methods in terms of model parameters and FLOPs.
arXiv Detail & Related papers (2022-05-30T15:26:52Z) - Residual Local Feature Network for Efficient Super-Resolution [20.62809970985125]
In this work, we propose a novel Residual Local Feature Network (RLFN)
The main idea is using three convolutional layers for residual local feature learning to simplify feature aggregation.
In addition, we won the first place in the runtime track of the NTIRE 2022 efficient super-resolution challenge.
arXiv Detail & Related papers (2022-05-16T08:46:34Z) - Learning Frequency-aware Dynamic Network for Efficient Super-Resolution [56.98668484450857]
This paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain.
In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden.
Experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures.
arXiv Detail & Related papers (2021-03-15T12:54:26Z) - Fully Quantized Image Super-Resolution Networks [81.75002888152159]
We propose a Fully Quantized image Super-Resolution framework (FQSR) to jointly optimize efficiency and accuracy.
We apply our quantization scheme on multiple mainstream super-resolution architectures, including SRResNet, SRGAN and EDSR.
Our FQSR using low bits quantization can achieve on par performance compared with the full-precision counterparts on five benchmark datasets.
arXiv Detail & Related papers (2020-11-29T03:53:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.