Structured Sparsity Learning for Efficient Video Super-Resolution

• URL: http://arxiv.org/abs/2206.07687v3
• Date: Sat, 25 Mar 2023 13:28:14 GMT
• Title: Structured Sparsity Learning for Efficient Video Super-Resolution
• Authors: Bin Xia, Jingwen He, Yulun Zhang, Yitong Wang, Yapeng Tian, Wenming Yang, and Luc Van Gool
• Abstract summary: We develop a structured pruning scheme called Structured Sparsity Learning (SSL) according to the properties of video super-resolution (VSR) models. In SSL, we design pruning schemes for several key components in VSR models, including residual blocks, recurrent networks, and upsampling networks.
• Score: 99.1632164448236
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The high computational costs of video super-resolution (VSR) models hinder their deployment on resource-limited devices, (e.g., smartphones and drones). Existing VSR models contain considerable redundant filters, which drag down the inference efficiency. To prune these unimportant filters, we develop a structured pruning scheme called Structured Sparsity Learning (SSL) according to the properties of VSR. In SSL, we design pruning schemes for several key components in VSR models, including residual blocks, recurrent networks, and upsampling networks. Specifically, we develop a Residual Sparsity Connection (RSC) scheme for residual blocks of recurrent networks to liberate pruning restrictions and preserve the restoration information. For upsampling networks, we design a pixel-shuffle pruning scheme to guarantee the accuracy of feature channel-space conversion. In addition, we observe that pruning error would be amplified as the hidden states propagate along with recurrent networks. To alleviate the issue, we design Temporal Finetuning (TF). Extensive experiments show that SSL can significantly outperform recent methods quantitatively and qualitatively.

Related papers

• Cascaded Temporal Updating Network for Efficient Video Super-Resolution [47.63267159007611]
  Key components in recurrent-based VSR networks significantly impact model efficiency. We propose a cascaded temporal updating network (CTUN) for efficient VSR. CTUN achieves a favorable trade-off between efficiency and performance compared to existing methods.
  arXiv  Detail & Related papers  (2024-08-26T12:59:32Z)
• A Lightweight Recurrent Learning Network for Sustainable Compressed Sensing [27.964167481909588]
  We propose a lightweight but effective deep neural network based on recurrent learning to achieve a sustainable CS system. Our proposed model can achieve a better reconstruction quality than existing state-of-the-art CS algorithms.
  arXiv  Detail & Related papers  (2023-04-23T14:54:15Z)
• Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
  Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures. This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead. We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
  arXiv  Detail & Related papers  (2023-03-16T21:06:13Z)
• RDRN: Recursively Defined Residual Network for Image Super-Resolution [58.64907136562178]
  Deep convolutional neural networks (CNNs) have obtained remarkable performance in single image super-resolution. We propose a novel network architecture which utilizes attention blocks efficiently.
  arXiv  Detail & Related papers  (2022-11-17T11:06:29Z)
• Image Superresolution using Scale-Recurrent Dense Network [30.75380029218373]
  Recent advances in the design of convolutional neural network (CNN) have yielded significant improvements in the performance of image super-resolution (SR) We propose a scale recurrent SR architecture built upon units containing series of dense connections within a residual block (Residual Dense Blocks (RDBs)) Our scale recurrent design delivers competitive performance for higher scale factors while being parametrically more efficient as compared to current state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-01-28T09:18:43Z)
• Deep Networks for Image and Video Super-Resolution [30.75380029218373]
  Single image super-resolution (SISR) is built using efficient convolutional units we refer to as mixed-dense connection blocks (MDCB) We train two versions of our network to enhance complementary image qualities using different loss configurations. We further employ our network for super-resolution task, where our network learns to aggregate information from multiple frames and maintain-temporal consistency.
  arXiv  Detail & Related papers  (2022-01-28T09:15:21Z)
• BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond [75.62146968824682]
  Video super-resolution (VSR) approaches tend to have more components than the image counterparts. We show a succinct pipeline, BasicVSR, that achieves appealing improvements in terms of speed and restoration quality.
  arXiv  Detail & Related papers  (2020-12-03T18:56:14Z)
• Lightweight image super-resolution with enhanced CNN [82.36883027158308]
  Deep convolutional neural networks (CNNs) with strong expressive ability have achieved impressive performances on single image super-resolution (SISR) We propose a lightweight enhanced SR CNN (LESRCNN) with three successive sub-blocks, an information extraction and enhancement block (IEEB), a reconstruction block (RB) and an information refinement block (IRB) IEEB extracts hierarchical low-resolution (LR) features and aggregates the obtained features step-by-step to increase the memory ability of the shallow layers on deep layers for SISR. RB converts low-frequency features into high-frequency features by fusing global
  arXiv  Detail & Related papers  (2020-07-08T18:03:40Z)
• iSeeBetter: Spatio-temporal video super-resolution using recurrent generative back-projection networks [0.0]
  We present iSeeBetter, a novel GAN-based structural-temporal approach to video super-resolution (VSR) iSeeBetter extracts spatial and temporal information from the current and neighboring frames using the concept of recurrent back-projection networks as its generator. Our results demonstrate that iSeeBetter offers superior VSR fidelity and surpasses state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-06-13T01:36:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.