ClassSR: A General Framework to Accelerate Super-Resolution Networks by Data Characteristic

• URL: http://arxiv.org/abs/2103.04039v1
• Date: Sat, 6 Mar 2021 06:00:31 GMT
• Title: ClassSR: A General Framework to Accelerate Super-Resolution Networks by Data Characteristic
• Authors: Xiangtao Kong, Hengyuan Zhao, Yu Qiao, Chao Dong
• Abstract summary: We aim at accelerating super-resolution (SR) networks on large images (2K-8K) We find that different image regions have different restoration difficulties and can be processed by networks with different capacities. We propose a new solution pipeline -- ClassSR that combines classification and SR in a unified framework.
• Score: 35.02837100573671
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We aim at accelerating super-resolution (SR) networks on large images (2K-8K). The large images are usually decomposed into small sub-images in practical usages. Based on this processing, we found that different image regions have different restoration difficulties and can be processed by networks with different capacities. Intuitively, smooth areas are easier to super-solve than complex textures. To utilize this property, we can adopt appropriate SR networks to process different sub-images after the decomposition. On this basis, we propose a new solution pipeline -- ClassSR that combines classification and SR in a unified framework. In particular, it first uses a Class-Module to classify the sub-images into different classes according to restoration difficulties, then applies an SR-Module to perform SR for different classes. The Class-Module is a conventional classification network, while the SR-Module is a network container that consists of the to-be-accelerated SR network and its simplified versions. We further introduce a new classification method with two losses -- Class-Loss and Average-Loss to produce the classification results. After joint training, a majority of sub-images will pass through smaller networks, thus the computational cost can be significantly reduced. Experiments show that our ClassSR can help most existing methods (e.g., FSRCNN, CARN, SRResNet, RCAN) save up to 50% FLOPs on DIV8K datasets. This general framework can also be applied in other low-level vision tasks.

Related papers

• Learning Many-to-Many Mapping for Unpaired Real-World Image Super-resolution and Downscaling [60.80788144261183]
  We propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2023-10-08T01:48:34Z)
• Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
  Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures. This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead. We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
  arXiv  Detail & Related papers  (2023-03-16T21:06:13Z)
• Improving Super-Resolution Performance using Meta-Attention Layers [17.870338228921327]
  Convolutional Neural Networks (CNNs) have achieved impressive results across many super-resolution (SR) and image restoration tasks. Ill-posed nature of SR can make it difficult to accurately super-resolve an image which has undergone multiple different degradations. We introduce meta-attention, a mechanism which allows any SR CNN to exploit the information available in relevant degradation parameters.
  arXiv  Detail & Related papers  (2021-10-27T09:20:21Z)
• Structure-Preserving Image Super-Resolution [94.16949589128296]
  Structures matter in single image super-resolution (SISR) Recent studies have promoted the development of SISR by recovering photo-realistic images. However, there are still undesired structural distortions in the recovered images.
  arXiv  Detail & Related papers  (2021-09-26T08:48:27Z)
• Over-and-Under Complete Convolutional RNN for MRI Reconstruction [57.95363471940937]
  Recent deep learning-based methods for MR image reconstruction usually leverage a generic auto-encoder architecture. We propose an Over-and-Under Complete Convolu?tional Recurrent Neural Network (OUCR), which consists of an overcomplete and an undercomplete Convolutional Recurrent Neural Network(CRNN) The proposed method achieves significant improvements over the compressed sensing and popular deep learning-based methods with less number of trainable parameters.
  arXiv  Detail & Related papers  (2021-06-16T15:56:34Z)
• Component Divide-and-Conquer for Real-World Image Super-Resolution [143.24770911629807]
  We present a large-scale Diverse Real-world image Super-Resolution dataset, i.e., DRealSR, as well as a divide-and-conquer Super-Resolution network. DRealSR establishes a new SR benchmark with diverse real-world degradation processes. We propose a Component Divide-and-Conquer (CDC) model and a Gradient-Weighted (GW) loss for SR.
  arXiv  Detail & Related papers  (2020-08-05T04:26:26Z)
• Learning with Privileged Information for Efficient Image Super-Resolution [35.599731963795875]
  We introduce in this paper a novel distillation framework, consisting of teacher and student networks, that allows to boost the performance of FSRCNN drastically. The encoder in the teacher learns the degradation process, subsampling of HR images, using an imitation loss. The student and the decoder in the teacher, having the same network architecture as FSRCNN, try to reconstruct HR images.
  arXiv  Detail & Related papers  (2020-07-15T07:44:18Z)
• Learning A Single Network for Scale-Arbitrary Super-Resolution [43.025921944418485]
  We propose to learn a scale-arbitrary image SR network from scale-specific networks. Our plug-in module can be easily adapted to existing networks to achieve scale-arbitrary SR.
  arXiv  Detail & Related papers  (2020-04-08T03:40:15Z)
• Unpaired Image Super-Resolution using Pseudo-Supervision [12.18340575383456]
  We propose an unpaired image super-resolution (SR) method using a generative adversarial network. Our network consists of an unpaired kernel/noise correction network and a pseudo-paired SR network. Experiments on diverse datasets show that the proposed method is superior to existing solutions to the unpaired SR problem.
  arXiv  Detail & Related papers  (2020-02-26T10:30:52Z)
• DDet: Dual-path Dynamic Enhancement Network for Real-World Image Super-Resolution [69.2432352477966]
  Real image super-resolution(Real-SR) focus on the relationship between real-world high-resolution(HR) and low-resolution(LR) image. In this article, we propose a Dual-path Dynamic Enhancement Network(DDet) for Real-SR. Unlike conventional methods which stack up massive convolutional blocks for feature representation, we introduce a content-aware framework to study non-inherently aligned image pair.
  arXiv  Detail & Related papers  (2020-02-25T18:24:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.