Diving Deep into Regions: Exploiting Regional Information Transformer
for Single Image Deraining
- URL: http://arxiv.org/abs/2402.16033v1
- Date: Sun, 25 Feb 2024 09:09:30 GMT
- Title: Diving Deep into Regions: Exploiting Regional Information Transformer
for Single Image Deraining
- Authors: Baiang Li, Zhao Zhang, Huan Zheng, Xiaogang Xu, Yanyan Wei, Jingyi
Zhang, Jicong Fan and Meng Wang
- Abstract summary: Region Transformer Block (RTB) integrates a Region Masked Attention (RMA) mechanism and a Mixed Gate Forward Block (MGFB)
Our model reaches state-of-the-art performance, significantly improving the image deraining quality.
- Score: 42.72715866295803
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Transformer-based Single Image Deraining (SID) methods have achieved
remarkable success, primarily attributed to their robust capability in
capturing long-range interactions. However, we've noticed that current methods
handle rain-affected and unaffected regions concurrently, overlooking the
disparities between these areas, resulting in confusion between rain streaks
and background parts, and inabilities to obtain effective interactions,
ultimately resulting in suboptimal deraining outcomes. To address the above
issue, we introduce the Region Transformer (Regformer), a novel SID method that
underlines the importance of independently processing rain-affected and
unaffected regions while considering their combined impact for high-quality
image reconstruction. The crux of our method is the innovative Region
Transformer Block (RTB), which integrates a Region Masked Attention (RMA)
mechanism and a Mixed Gate Forward Block (MGFB). Our RTB is used for attention
selection of rain-affected and unaffected regions and local modeling of mixed
scales. The RMA generates attention maps tailored to these two regions and
their interactions, enabling our model to capture comprehensive features
essential for rain removal. To better recover high-frequency textures and
capture more local details, we develop the MGFB as a compensation module to
complete local mixed scale modeling. Extensive experiments demonstrate that our
model reaches state-of-the-art performance, significantly improving the image
deraining quality. Our code and trained models are publicly available.
Related papers
- Region Attention Transformer for Medical Image Restoration [16.33261165253018]
We introduce a novel Region Attention Transformer (RAT) that utilizes a region-based multi-head self-attention mechanism (R-MSA)
RAT partitions the input image into non-overlapping semantic regions using the robust Segment Anything Model (SAM)
This region is more flexible and interpretable, ensuring that only pixels from similar semantic regions complement each other, eliminating interference from irrelevant regions.
Experiments demonstrate the effectiveness of RAT in various medical image restoration tasks, including PET image synthesis, CT image denoising, and pathological image super-resolution.
arXiv Detail & Related papers (2024-07-12T13:52:05Z) - Coherent and Multi-modality Image Inpainting via Latent Space Optimization [61.99406669027195]
PILOT (intextbfPainting vtextbfIa textbfLatent textbfOptextbfTimization) is an optimization approach grounded on a novel textitsemantic centralization and textitbackground preservation loss.
Our method searches latent spaces capable of generating inpainted regions that exhibit high fidelity to user-provided prompts while maintaining coherence with the background.
arXiv Detail & Related papers (2024-07-10T19:58:04Z) - Image Deraining with Frequency-Enhanced State Space Model [2.9465623430708905]
This study introduces State Space Models (SSMs) to rain removal and proposes a Deraining Frequency-Enhanced State Space Model (DFSSM)
To effectively remove rain streaks, which produce high-intensity frequency components in specific directions, we employ frequency domain processing concurrently with SSM.
We develop a novel mixed-scale gated-convolutional block, which uses convolutions with multiple kernel sizes to capture various scale degradations effectively.
arXiv Detail & Related papers (2024-05-26T07:45:12Z) - Look-Around Before You Leap: High-Frequency Injected Transformer for Image Restoration [46.96362010335177]
In this paper, we propose HIT, a simple yet effective High-frequency Injected Transformer for image restoration.
Specifically, we design a window-wise injection module (WIM), which incorporates abundant high-frequency details into the feature map, to provide reliable references for restoring high-quality images.
In addition, we introduce a spatial enhancement unit (SEU) to preserve essential spatial relationships that may be lost due to the computations carried out across channel dimensions in the BIM.
arXiv Detail & Related papers (2024-03-30T08:05:00Z) - Gabor-guided transformer for single image deraining [2.330361251490783]
We propose a Gabor-guided tranformer (Gabformer) for single image deraining.
The focus on local texture features is enhanced by incorporating the information processed by the Gabor filter into the query vector.
Our method outperforms state-of-the-art approaches.
arXiv Detail & Related papers (2024-03-12T07:41:51Z) - DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image
Enhancement [77.0360085530701]
Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments.
Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features.
Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
arXiv Detail & Related papers (2023-12-12T06:07:21Z) - Magic ELF: Image Deraining Meets Association Learning and Transformer [63.761812092934576]
This paper aims to unify CNN and Transformer to take advantage of their learning merits for image deraining.
A novel multi-input attention module (MAM) is proposed to associate rain removal and background recovery.
Our proposed method (dubbed as ELF) outperforms the state-of-the-art approach (MPRNet) by 0.25 dB on average.
arXiv Detail & Related papers (2022-07-21T12:50:54Z) - From Rain Generation to Rain Removal [67.71728610434698]
We build a full Bayesian generative model for rainy image where the rain layer is parameterized as a generator.
We employ the variational inference framework to approximate the expected statistical distribution of rainy image.
Comprehensive experiments substantiate that the proposed model can faithfully extract the complex rain distribution.
arXiv Detail & Related papers (2020-08-08T18:56:51Z) - Image Fine-grained Inpainting [89.17316318927621]
We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields.
To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss.
We also employ a discriminator with local and global branches to ensure local-global contents consistency.
arXiv Detail & Related papers (2020-02-07T03:45:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.