Lightweight Stepless Super-Resolution of Remote Sensing Images via Saliency-Aware Dynamic Routing Strategy

• URL: http://arxiv.org/abs/2210.07598v1
• Date: Fri, 14 Oct 2022 07:49:03 GMT
• Title: Lightweight Stepless Super-Resolution of Remote Sensing Images via Saliency-Aware Dynamic Routing Strategy
• Authors: Hanlin Wu, Ning Ni, Libao Zhang
• Abstract summary: Deep learning algorithms have greatly improved the performance of remote sensing image (RSI) super-resolution (SR) However, increasing network depth and parameters cause a huge burden of computing and storage. We propose a saliency-aware dynamic routing network (SalDRN) for lightweight and stepless SR of RSIs.
• Score: 15.587621728422414
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning-based algorithms have greatly improved the performance of remote sensing image (RSI) super-resolution (SR). However, increasing network depth and parameters cause a huge burden of computing and storage. Directly reducing the depth or width of existing models results in a large performance drop. We observe that the SR difficulty of different regions in an RSI varies greatly, and existing methods use the same deep network to process all regions in an image, resulting in a waste of computing resources. In addition, existing SR methods generally predefine integer scale factors and cannot perform stepless SR, i.e., a single model can deal with any potential scale factor. Retraining the model on each scale factor wastes considerable computing resources and model storage space. To address the above problems, we propose a saliency-aware dynamic routing network (SalDRN) for lightweight and stepless SR of RSIs. First, we introduce visual saliency as an indicator of region-level SR difficulty and integrate a lightweight saliency detector into the SalDRN to capture pixel-level visual characteristics. Then, we devise a saliency-aware dynamic routing strategy that employs path selection switches to adaptively select feature extraction paths of appropriate depth according to the SR difficulty of sub-image patches. Finally, we propose a novel lightweight stepless upsampling module whose core is an implicit feature function for realizing mapping from low-resolution feature space to high-resolution feature space. Comprehensive experiments verify that the SalDRN can achieve a good trade-off between performance and complexity. The code is available at \url{https://github.com/hanlinwu/SalDRN}.

Related papers

• DCS-RISR: Dynamic Channel Splitting for Efficient Real-world Image Super-Resolution [15.694407977871341]
  Real-world image super-resolution (RISR) has received increased focus for improving the quality of SR images under unknown complex degradation. Existing methods rely on the heavy SR models to enhance low-resolution (LR) images of different degradation levels. We propose a novel Dynamic Channel Splitting scheme for efficient Real-world Image Super-Resolution, termed DCS-RISR.
  arXiv  Detail & Related papers  (2022-12-15T04:34:57Z)
• CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution [158.2282163651066]
  This paper proposes a continuous implicit attention-in-attention network, called CiaoSR. We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features. We embed a scale-aware attention in this implicit attention network to exploit additional non-local information.
  arXiv  Detail & Related papers  (2022-12-08T15:57:46Z)
• CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolution [55.50793823060282]
  We propose a novel Content-Aware Dynamic Quantization (CADyQ) method for image super-resolution (SR) networks. CADyQ allocates optimal bits to local regions and layers adaptively based on the local contents of an input image. The pipeline has been tested on various SR networks and evaluated on several standard benchmarks.
  arXiv  Detail & Related papers  (2022-07-21T07:50:50Z)
• Lightweight Image Super-Resolution with Hierarchical and Differentiable Neural Architecture Search [38.83764580480486]
  Single Image Super-Resolution (SISR) tasks have achieved significant performance with deep neural networks. We propose a novel differentiable Neural Architecture Search (NAS) approach on both the cell-level and network-level to search for lightweight SISR models.
  arXiv  Detail & Related papers  (2021-05-09T13:30:16Z)
• MPRNet: Multi-Path Residual Network for Lightweight Image Super Resolution [2.3576437999036473]
  A novel lightweight super resolution network is proposed, which improves the SOTA performance in lightweight SR. The proposed architecture also contains a new attention mechanism, Two-Fold Attention Module, to maximize the representation ability of the model.
  arXiv  Detail & Related papers  (2020-11-09T17:11:15Z)
• Real Image Super Resolution Via Heterogeneous Model Ensemble using GP-NAS [63.48801313087118]
  We propose a new method for image superresolution using deep residual network with dense skip connections. The proposed method won the first place in all three tracks of the AIM 2020 Real Image Super-Resolution Challenge.
  arXiv  Detail & Related papers  (2020-09-02T22:33:23Z)
• Lightweight image super-resolution with enhanced CNN [82.36883027158308]
  Deep convolutional neural networks (CNNs) with strong expressive ability have achieved impressive performances on single image super-resolution (SISR) We propose a lightweight enhanced SR CNN (LESRCNN) with three successive sub-blocks, an information extraction and enhancement block (IEEB), a reconstruction block (RB) and an information refinement block (IRB) IEEB extracts hierarchical low-resolution (LR) features and aggregates the obtained features step-by-step to increase the memory ability of the shallow layers on deep layers for SISR. RB converts low-frequency features into high-frequency features by fusing global
  arXiv  Detail & Related papers  (2020-07-08T18:03:40Z)
• Hyperspectral Image Super-resolution via Deep Progressive Zero-centric Residual Learning [62.52242684874278]
  Cross-modality distribution of spatial and spectral information makes the problem challenging. We propose a novel textitlightweight deep neural network-based framework, namely PZRes-Net. Our framework learns a high resolution and textitzero-centric residual image, which contains high-frequency spatial details of the scene.
  arXiv  Detail & Related papers  (2020-06-18T06:32:11Z)
• Deep Adaptive Inference Networks for Single Image Super-Resolution [72.7304455761067]
  Single image super-resolution (SISR) has witnessed tremendous progress in recent years owing to the deployment of deep convolutional neural networks (CNNs) In this paper, we take a step forward to address this issue by leveraging the adaptive inference networks for deep SISR (AdaDSR) Our AdaDSR involves an SISR model as backbone and a lightweight adapter module which takes image features and resource constraint as input and predicts a map of local network depth.
  arXiv  Detail & Related papers  (2020-04-08T10:08:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.