Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution

• URL: http://arxiv.org/abs/2501.15774v2
• Date: Tue, 04 Feb 2025 03:24:45 GMT
• Title: Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution
• Authors: Karam Park, Jae Woong Soh, Nam Ik Cho,
• Abstract summary: Transformer-based Super-Resolution (SR) methods have demonstrated superior performance compared to convolutional neural network (CNN)-based SR approaches. We propose a lightweight SR network that integrates attention-sharing and an information distillation structure specifically designed for Transformer-based SR methods.
• Score: 23.265907475054156
• License:
• Abstract: Transformer-based Super-Resolution (SR) methods have demonstrated superior performance compared to convolutional neural network (CNN)-based SR approaches due to their capability to capture long-range dependencies. However, their high computational complexity necessitates the development of lightweight approaches for practical use. To address this challenge, we propose the Attention-Sharing Information Distillation (ASID) network, a lightweight SR network that integrates attention-sharing and an information distillation structure specifically designed for Transformer-based SR methods. We modify the information distillation scheme, originally designed for efficient CNN operations, to reduce the computational load of stacked self-attention layers, effectively addressing the efficiency bottleneck. Additionally, we introduce attention-sharing across blocks to further minimize the computational cost of self-attention operations. By combining these strategies, ASID achieves competitive performance with existing SR methods while requiring only around 300K parameters - significantly fewer than existing CNN-based and Transformer-based SR models. Furthermore, ASID outperforms state-of-the-art SR methods when the number of parameters is matched, demonstrating its efficiency and effectiveness. The code and supplementary material are available on the project page.

Related papers

• Efficient Single Image Super-Resolution with Entropy Attention and Receptive Field Augmentation [34.50541063621832]
  We present an efficient single image super-resolution (SISR) model composed of a novel entropy attention (EA) and a shifting large kernel attention (SLKA) EA increases the entropy of intermediate features conditioned on a Gaussian distribution, providing more informative input for subsequent reasoning. SLKA extends the receptive field of SR models with the assistance of channel shifting, which also favors to boost the diversity of hierarchical features.
  arXiv  Detail & Related papers  (2024-08-08T02:03:10Z)
• HiT-SR: Hierarchical Transformer for Efficient Image Super-Resolution [70.52256118833583]
  We present a strategy to convert transformer-based SR networks to hierarchical transformers (HiT-SR) Specifically, we first replace the commonly used fixed small windows with expanding hierarchical windows to aggregate features at different scales. Considering the intensive computation required for large windows, we further design a spatial-channel correlation method with linear complexity to window sizes.
  arXiv  Detail & Related papers  (2024-07-08T12:42:10Z)
• Transforming Image Super-Resolution: A ConvFormer-based Efficient Approach [58.57026686186709]
  We introduce the Convolutional Transformer layer (ConvFormer) and propose a ConvFormer-based Super-Resolution network (CFSR) CFSR inherits the advantages of both convolution-based and transformer-based approaches. Experiments demonstrate that CFSR strikes an optimal balance between computational cost and performance.
  arXiv  Detail & Related papers  (2024-01-11T03:08:00Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
  Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures. This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead. We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
  arXiv  Detail & Related papers  (2023-03-16T21:06:13Z)
• A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution [17.094665593472214]
  We propose a dynamic residual self-attention network (DRSAN) for lightweight single-image super-resolution (SISR) DRSAN has dynamic residual connections based on dynamic residual attention (DRA), which adaptively changes its structure according to input statistics. We also propose a residual self-attention (RSA) module to further boost the performance, which produces 3-dimensional attention maps without additional parameters.
  arXiv  Detail & Related papers  (2021-12-08T06:41:21Z)
• Local-Selective Feature Distillation for Single Image Super-Resolution [42.83228585332463]
  We propose a novel feature distillation (FD) method which is suitable for single image super-resolution (SISR) We show the limitations of the existing FitNet-based FD method that it suffers in the SISR task, and propose to modify the existing FD algorithm to focus on local feature information. We call our method local-selective feature distillation (LSFD) and verify that our method outperforms conventional FD methods in SISR problems.
  arXiv  Detail & Related papers  (2021-11-22T05:05:37Z)
• Residual Feature Distillation Network for Lightweight Image Super-Resolution [40.52635571871426]
  We propose a lightweight and accurate SISR model called residual feature distillation network (RFDN) RFDN uses multiple feature distillation connections to learn more discriminative feature representations. We also propose a shallow residual block (SRB) as the main building block of RFDN so that the network can benefit most from residual learning.
  arXiv  Detail & Related papers  (2020-09-24T08:46:40Z)
• Lightweight image super-resolution with enhanced CNN [82.36883027158308]
  Deep convolutional neural networks (CNNs) with strong expressive ability have achieved impressive performances on single image super-resolution (SISR) We propose a lightweight enhanced SR CNN (LESRCNN) with three successive sub-blocks, an information extraction and enhancement block (IEEB), a reconstruction block (RB) and an information refinement block (IRB) IEEB extracts hierarchical low-resolution (LR) features and aggregates the obtained features step-by-step to increase the memory ability of the shallow layers on deep layers for SISR. RB converts low-frequency features into high-frequency features by fusing global
  arXiv  Detail & Related papers  (2020-07-08T18:03:40Z)
• Deep Adaptive Inference Networks for Single Image Super-Resolution [72.7304455761067]
  Single image super-resolution (SISR) has witnessed tremendous progress in recent years owing to the deployment of deep convolutional neural networks (CNNs) In this paper, we take a step forward to address this issue by leveraging the adaptive inference networks for deep SISR (AdaDSR) Our AdaDSR involves an SISR model as backbone and a lightweight adapter module which takes image features and resource constraint as input and predicts a map of local network depth.
  arXiv  Detail & Related papers  (2020-04-08T10:08:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.