Compressing Deep Image Super-resolution Models
- URL: http://arxiv.org/abs/2401.00523v2
- Date: Wed, 21 Feb 2024 20:25:53 GMT
- Title: Compressing Deep Image Super-resolution Models
- Authors: Yuxuan Jiang, Jakub Nawala, Fan Zhang, and David Bull
- Abstract summary: This work employs a three-stage workflow for compressing deep SR models which significantly reduces their memory requirement.
We have applied this approach to two popular image super-resolution networks, SwinIR and EDSR, to demonstrate its effectiveness.
The resulting compact models, SwinIRmini and EDSRmini, attain an 89% and 96% reduction in both model size and floating-point operations.
- Score: 2.895266689123347
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Deep learning techniques have been applied in the context of image
super-resolution (SR), achieving remarkable advances in terms of reconstruction
performance. Existing techniques typically employ highly complex model
structures which result in large model sizes and slow inference speeds. This
often leads to high energy consumption and restricts their adoption for
practical applications. To address this issue, this work employs a three-stage
workflow for compressing deep SR models which significantly reduces their
memory requirement. Restoration performance has been maintained through
teacher-student knowledge distillation using a newly designed distillation
loss. We have applied this approach to two popular image super-resolution
networks, SwinIR and EDSR, to demonstrate its effectiveness. The resulting
compact models, SwinIRmini and EDSRmini, attain an 89% and 96% reduction in
both model size and floating-point operations (FLOPs) respectively, compared to
their original versions. They also retain competitive super-resolution
performance compared to their original models and other commonly used SR
approaches. The source code and pre-trained models for these two lightweight SR
approaches are released at https://pikapi22.github.io/CDISM/.
Related papers
- Large Kernel Distillation Network for Efficient Single Image Super-Resolution [8.094254341695684]
Single-image super-resolution (SISR) has achieved remarkable performance in recent years.
Current state-of-the-art (SOTA) models still face problems such as high computational costs.
We propose the Large Kernel Distillation Network (LKDN) in this paper.
arXiv Detail & Related papers (2024-07-19T14:21:56Z) - Task-Oriented Diffusion Model Compression [27.813361445528397]
Large-scale Text-to-Image (T2I) diffusion models have yielded remarkable high-quality image generation, diverse downstream Image-to-Image (I2I) applications have emerged.
Despite the impressive results achieved by these I2I models, their practical utility is hampered by their large model size and the computational burden of the iterative denoising process.
In this paper, we explore the compression potential of these I2I models in a task-oriented manner and introduce a novel method for reducing both model size and the number of timesteps.
arXiv Detail & Related papers (2024-01-31T02:25:52Z) - A-SDM: Accelerating Stable Diffusion through Redundancy Removal and
Performance Optimization [54.113083217869516]
In this work, we first explore the computational redundancy part of the network.
We then prune the redundancy blocks of the model and maintain the network performance.
Thirdly, we propose a global-regional interactive (GRI) attention to speed up the computationally intensive attention part.
arXiv Detail & Related papers (2023-12-24T15:37:47Z) - Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures.
This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead.
We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
arXiv Detail & Related papers (2023-03-16T21:06:13Z) - Towards Lightweight Super-Resolution with Dual Regression Learning [58.98801753555746]
Deep neural networks have exhibited remarkable performance in image super-resolution (SR) tasks.
The SR problem is typically an ill-posed problem and existing methods would come with several limitations.
We propose a dual regression learning scheme to reduce the space of possible SR mappings.
arXiv Detail & Related papers (2022-07-16T12:46:10Z) - Accurate and Lightweight Image Super-Resolution with Model-Guided Deep
Unfolding Network [63.69237156340457]
We present and advocate an explainable approach toward SISR named model-guided deep unfolding network (MoG-DUN)
MoG-DUN is accurate (producing fewer aliasing artifacts), computationally efficient (with reduced model parameters), and versatile (capable of handling multiple degradations)
The superiority of the proposed MoG-DUN method to existing state-of-theart image methods including RCAN, SRDNF, and SRFBN is substantiated by extensive experiments on several popular datasets and various degradation scenarios.
arXiv Detail & Related papers (2020-09-14T08:23:37Z) - Towards Practical Lipreading with Distilled and Efficient Models [57.41253104365274]
Lipreading has witnessed a lot of progress due to the resurgence of neural networks.
Recent works have placed emphasis on aspects such as improving performance by finding the optimal architecture or improving generalization.
There is still a significant gap between the current methodologies and the requirements for an effective deployment of lipreading in practical scenarios.
We propose a series of innovations that significantly bridge that gap: first, we raise the state-of-the-art performance by a wide margin on LRW and LRW-1000 to 88.5% and 46.6%, respectively using self-distillation.
arXiv Detail & Related papers (2020-07-13T16:56:27Z) - Characteristic Regularisation for Super-Resolving Face Images [81.84939112201377]
Existing facial image super-resolution (SR) methods focus mostly on improving artificially down-sampled low-resolution (LR) imagery.
Previous unsupervised domain adaptation (UDA) methods address this issue by training a model using unpaired genuine LR and HR data.
This renders the model overstretched with two tasks: consistifying the visual characteristics and enhancing the image resolution.
We formulate a method that joins the advantages of conventional SR and UDA models.
arXiv Detail & Related papers (2019-12-30T16:27:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.