GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions

• URL: http://arxiv.org/abs/2305.17863v2
• Date: Fri, 21 Jun 2024 07:46:10 GMT
• Title: GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions
• Authors: Tao Wang, Kaihao Zhang, Ziqian Shao, Wenhan Luo, Bjorn Stenger, Tong Lu, Tae-Kyun Kim, Wei Liu, Hongdong Li,
• Abstract summary: We propose a novel transformer-based framework called GridFormer. GridFormer serves as a backbone for image restoration under adverse weather conditions. The framework achieves state-of-the-art results on five diverse image restoration tasks.
• Score: 97.45751035126548
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image restoration in adverse weather conditions is a difficult task in computer vision. In this paper, we propose a novel transformer-based framework called GridFormer which serves as a backbone for image restoration under adverse weather conditions. GridFormer is designed in a grid structure using a residual dense transformer block, and it introduces two core designs. First, it uses an enhanced attention mechanism in the transformer layer. The mechanism includes stages of the sampler and compact self-attention to improve efficiency, and a local enhancement stage to strengthen local information. Second, we introduce a residual dense transformer block (RDTB) as the final GridFormer layer. This design further improves the network's ability to learn effective features from both preceding and current local features. The GridFormer framework achieves state-of-the-art results on five diverse image restoration tasks in adverse weather conditions, including image deraining, dehazing, deraining \& dehazing, desnowing, and multi-weather restoration. The source code and pre-trained models are available at https://github.com/TaoWangzj/GridFormer.

Related papers

• SwinStyleformer is a favorable choice for image inversion [2.8115030277940947]
  This paper proposes the first pure Transformer structure inversion network called SwinStyleformer. Experiments found that the inversion network with the Transformer backbone could not successfully invert the image.
  arXiv  Detail & Related papers  (2024-06-19T02:08:45Z)
• Segmentation Guided Sparse Transformer for Under-Display Camera Image Restoration [91.65248635837145]
  Under-Display Camera (UDC) is an emerging technology that achieves full-screen display via hiding the camera under the display panel. In this paper, we observe that when using the Vision Transformer for UDC degraded image restoration, the global attention samples a large amount of redundant information and noise. We propose a Guided Sparse Transformer method (SGSFormer) for the task of restoring high-quality images from UDC degraded images.
  arXiv  Detail & Related papers  (2024-03-09T13:11:59Z)
• Gradient Adjusting Networks for Domain Inversion [82.72289618025084]
  StyleGAN2 was demonstrated to be a powerful image generation engine that supports semantic editing. We present a per-image optimization method that tunes a StyleGAN2 generator such that it achieves a local edit to the generator's weights. Our experiments show a sizable gap in performance over the current state of the art in this very active domain.
  arXiv  Detail & Related papers  (2023-02-22T14:47:57Z)
• RePre: Improving Self-Supervised Vision Transformer with Reconstructive Pre-training [80.44284270879028]
  This paper incorporates local feature learning into self-supervised vision transformers via Reconstructive Pre-training (RePre) Our RePre extends contrastive frameworks by adding a branch for reconstructing raw image pixels in parallel with the existing contrastive objective.
  arXiv  Detail & Related papers  (2022-01-18T10:24:58Z)
• U2-Former: A Nested U-shaped Transformer for Image Restoration [30.187257111046556]
  We present a deep and effective Transformer-based network for image restoration, termed as U2-Former. It is able to employ Transformer as the core operation to perform image restoration in a deep encoding and decoding space.
  arXiv  Detail & Related papers  (2021-12-04T08:37:04Z)
• Restormer: Efficient Transformer for High-Resolution Image Restoration [118.9617735769827]
  convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data. Transformers have shown significant performance gains on natural language and high-level vision tasks. Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks.
  arXiv  Detail & Related papers  (2021-11-18T18:59:10Z)
• PPT Fusion: Pyramid Patch Transformerfor a Case Study in Image Fusion [37.993611194758195]
  We propose a Patch PyramidTransformer(PPT) to address the issues of extracting semantic information from an image. The experimental results demonstrate its superior performance against the state-of-the-art fusion approaches.
  arXiv  Detail & Related papers  (2021-07-29T13:57:45Z)
• Uformer: A General U-Shaped Transformer for Image Restoration [47.60420806106756]
  We build a hierarchical encoder-decoder network using the Transformer block for image restoration. Experiments on several image restoration tasks demonstrate the superiority of Uformer.
  arXiv  Detail & Related papers  (2021-06-06T12:33:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.