SwinIR: Image Restoration Using Swin Transformer
- URL: http://arxiv.org/abs/2108.10257v1
- Date: Mon, 23 Aug 2021 15:55:32 GMT
- Title: SwinIR: Image Restoration Using Swin Transformer
- Authors: Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, Radu
Timofte
- Abstract summary: We propose a strong baseline model SwinIR for image restoration based on the Swin Transformer.
SwinIR consists of three parts: shallow feature extraction, deep feature extraction and high-quality image reconstruction.
We conduct experiments on three representative tasks: image super-resolution, image denoising and JPEG compression artifact reduction.
- Score: 124.8794221439392
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Image restoration is a long-standing low-level vision problem that aims to
restore high-quality images from low-quality images (e.g., downscaled, noisy
and compressed images). While state-of-the-art image restoration methods are
based on convolutional neural networks, few attempts have been made with
Transformers which show impressive performance on high-level vision tasks. In
this paper, we propose a strong baseline model SwinIR for image restoration
based on the Swin Transformer. SwinIR consists of three parts: shallow feature
extraction, deep feature extraction and high-quality image reconstruction. In
particular, the deep feature extraction module is composed of several residual
Swin Transformer blocks (RSTB), each of which has several Swin Transformer
layers together with a residual connection. We conduct experiments on three
representative tasks: image super-resolution (including classical, lightweight
and real-world image super-resolution), image denoising (including grayscale
and color image denoising) and JPEG compression artifact reduction.
Experimental results demonstrate that SwinIR outperforms state-of-the-art
methods on different tasks by $\textbf{up to 0.14$\sim$0.45dB}$, while the
total number of parameters can be reduced by $\textbf{up to 67%}$.
Related papers
- Multi-Scale Representation Learning for Image Restoration with State-Space Model [13.622411683295686]
We propose a novel Multi-Scale State-Space Model-based (MS-Mamba) for efficient image restoration.
Our proposed method achieves new state-of-the-art performance while maintaining low computational complexity.
arXiv Detail & Related papers (2024-08-19T16:42:58Z) - Gated Multi-Resolution Transfer Network for Burst Restoration and
Enhancement [75.25451566988565]
We propose a novel Gated Multi-Resolution Transfer Network (GMTNet) to reconstruct a spatially precise high-quality image from a burst of low-quality raw images.
Detailed experimental analysis on five datasets validates our approach and sets a state-of-the-art for burst super-resolution, burst denoising, and low-light burst enhancement.
arXiv Detail & Related papers (2023-04-13T17:54:00Z) - Image Deblurring by Exploring In-depth Properties of Transformer [86.7039249037193]
We leverage deep features extracted from a pretrained vision transformer (ViT) to encourage recovered images to be sharp without sacrificing the performance measured by the quantitative metrics.
By comparing the transformer features between recovered image and target one, the pretrained transformer provides high-resolution blur-sensitive semantic information.
One regards the features as vectors and computes the discrepancy between representations extracted from recovered image and target one in Euclidean space.
arXiv Detail & Related papers (2023-03-24T14:14:25Z) - Single Image Super-Resolution Using Lightweight Networks Based on Swin
Transformer [2.9649783577150837]
We propose two lightweight models named as MSwinSR and UGSwinSR based on Swin Transformer.
MSwinSR increases PSNR by $mathbf0.07dB$ compared with the state-of-the-art model SwinIR.
The number of parameters can reduced by $mathbf30.68%$, and the calculation cost can reduced by $mathbf9.936%$.
arXiv Detail & Related papers (2022-10-20T05:03:16Z) - Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and
Restoration [71.6879432974126]
In this paper, we explore the novel Swin Transformer V2, to improve SwinIR for image super-resolution.
We conduct experiments on three representative tasks: JPEG compression artifacts removal, image super-resolution (classical and lightweight), and compressed image super-resolution.
Experimental results demonstrate that our method, Swin2SR, can improve the training convergence and performance of SwinIR.
arXiv Detail & Related papers (2022-09-22T23:25:08Z) - HST: Hierarchical Swin Transformer for Compressed Image Super-resolution [26.589370745694502]
We propose the Hierarchical Swin Transformer (HST) network to restore the low-resolution compressed image.
Our HST achieves the fifth place in AIM 2022 challenge on the low-quality compressed image super-resolution track, with the PSNR of 23.51dB.
arXiv Detail & Related papers (2022-08-21T13:41:51Z) - Restormer: Efficient Transformer for High-Resolution Image Restoration [118.9617735769827]
convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data.
Transformers have shown significant performance gains on natural language and high-level vision tasks.
Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks.
arXiv Detail & Related papers (2021-11-18T18:59:10Z) - Exploiting Deep Generative Prior for Versatile Image Restoration and
Manipulation [181.08127307338654]
This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images.
The deep generative prior (DGP) provides compelling results to restore missing semantics, e.g., color, patch, resolution, of various degraded images.
arXiv Detail & Related papers (2020-03-30T17:45:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.