Single Image Super-Resolution Using Lightweight Networks Based on Swin Transformer

• URL: http://arxiv.org/abs/2210.11019v1
• Date: Thu, 20 Oct 2022 05:03:16 GMT
• Title: Single Image Super-Resolution Using Lightweight Networks Based on Swin Transformer
• Authors: Bolong Zhang and Juan Chen and Quan Wen
• Abstract summary: We propose two lightweight models named as MSwinSR and UGSwinSR based on Swin Transformer. MSwinSR increases PSNR by $mathbf0.07dB$ compared with the state-of-the-art model SwinIR. The number of parameters can reduced by $mathbf30.68%$, and the calculation cost can reduced by $mathbf9.936%$.
• Score: 2.9649783577150837
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image super-resolution reconstruction is an important task in the field of image processing technology, which can restore low resolution image to high quality image with high resolution. In recent years, deep learning has been applied in the field of image super-resolution reconstruction. With the continuous development of deep neural network, the quality of the reconstructed images has been greatly improved, but the model complexity has also been increased. In this paper, we propose two lightweight models named as MSwinSR and UGSwinSR based on Swin Transformer. The most important structure in MSwinSR is called Multi-size Swin Transformer Block (MSTB), which mainly contains four parallel multi-head self-attention (MSA) blocks. UGSwinSR combines U-Net and GAN with Swin Transformer. Both of them can reduce the model complexity, but MSwinSR can reach a higher objective quality, while UGSwinSR can reach a higher perceptual quality. The experimental results demonstrate that MSwinSR increases PSNR by $\mathbf{0.07dB}$ compared with the state-of-the-art model SwinIR, while the number of parameters can reduced by $\mathbf{30.68\%}$, and the calculation cost can reduced by $\mathbf{9.936\%}$. UGSwinSR can effectively reduce the amount of calculation of the network, which can reduced by $\mathbf{90.92\%}$ compared with SwinIR.

Related papers

• Efficient Model Agnostic Approach for Implicit Neural Representation Based Arbitrary-Scale Image Super-Resolution [5.704360536038803]
  Single image super-resolution (SISR) has experienced significant advancements, primarily driven by deep convolutional networks. Traditional networks are limited to upscaling images to a fixed scale, leading to the utilization of implicit neural functions for generating arbitrarily scaled images. We introduce a novel and efficient framework, the Mixture of Experts Implicit Super-Resolution (MoEISR), which enables super-resolution at arbitrary scales.
  arXiv  Detail & Related papers  (2023-11-20T05:34:36Z)
• PTSR: Patch Translator for Image Super-Resolution [16.243363392717434]
  We propose a patch translator for image super-resolution (PTSR) to address this problem. The proposed PTSR is a transformer-based GAN network with no convolution operation. We introduce a novel patch translator module for regenerating the improved patches utilising multi-head attention.
  arXiv  Detail & Related papers  (2023-10-20T01:45:00Z)
• Lightweight Bimodal Network for Single-Image Super-Resolution via Symmetric CNN and Recursive Transformer [27.51790638626891]
  Single-image super-resolution (SISR) has achieved significant breakthroughs with the development of deep learning. To solve this issue, we propose a Lightweight Bimodal Network (LBNet) for SISR. Specifically, an effective Symmetric CNN is designed for local feature extraction and coarse image reconstruction.
  arXiv  Detail & Related papers  (2022-04-28T04:43:22Z)
• BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment [84.82352123245488]
  This work addresses the Burst Super-Resolution (BurstSR) task using a new architecture, which requires restoring a high-quality image from a sequence of noisy, misaligned, and low-resolution RAW bursts. We propose a Burst Super-Resolution Transformer (BSRT), which can significantly improve the capability of extracting inter-frame information and reconstruction. Our BSRT wins the championship in the NTIRE2022 Burst Super-Resolution Challenge.
  arXiv  Detail & Related papers  (2022-04-18T14:23:10Z)
• Hybrid Pixel-Unshuffled Network for Lightweight Image Super-Resolution [64.54162195322246]
  Convolutional neural network (CNN) has achieved great success on image super-resolution (SR) Most deep CNN-based SR models take massive computations to obtain high performance. We propose a novel Hybrid Pixel-Unshuffled Network (HPUN) by introducing an efficient and effective downsampling module into the SR task.
  arXiv  Detail & Related papers  (2022-03-16T20:10:41Z)
• Restormer: Efficient Transformer for High-Resolution Image Restoration [118.9617735769827]
  convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data. Transformers have shown significant performance gains on natural language and high-level vision tasks. Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks.
  arXiv  Detail & Related papers  (2021-11-18T18:59:10Z)
• SwinIR: Image Restoration Using Swin Transformer [124.8794221439392]
  We propose a strong baseline model SwinIR for image restoration based on the Swin Transformer. SwinIR consists of three parts: shallow feature extraction, deep feature extraction and high-quality image reconstruction. We conduct experiments on three representative tasks: image super-resolution, image denoising and JPEG compression artifact reduction.
  arXiv  Detail & Related papers  (2021-08-23T15:55:32Z)
• Learning Frequency-aware Dynamic Network for Efficient Super-Resolution [56.98668484450857]
  This paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain. In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden. Experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures.
  arXiv  Detail & Related papers  (2021-03-15T12:54:26Z)
• FAN: Frequency Aggregation Network for Real Image Super-resolution [33.30542701042704]
  Single image super-resolution (SISR) aims to recover the high-resolution (HR) image from its low-resolution (LR) input image. We propose FAN, a frequency aggregation network, to address the real-world image super-resolu-tion problem.
  arXiv  Detail & Related papers  (2020-09-30T10:18:41Z)
• Lightweight image super-resolution with enhanced CNN [82.36883027158308]
  Deep convolutional neural networks (CNNs) with strong expressive ability have achieved impressive performances on single image super-resolution (SISR) We propose a lightweight enhanced SR CNN (LESRCNN) with three successive sub-blocks, an information extraction and enhancement block (IEEB), a reconstruction block (RB) and an information refinement block (IRB) IEEB extracts hierarchical low-resolution (LR) features and aggregates the obtained features step-by-step to increase the memory ability of the shallow layers on deep layers for SISR. RB converts low-frequency features into high-frequency features by fusing global
  arXiv  Detail & Related papers  (2020-07-08T18:03:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.