Iterative Optimal Attention and Local Model for Single Image Rain Streak Removal

• URL: http://arxiv.org/abs/2503.16165v1
• Date: Thu, 20 Mar 2025 14:06:53 GMT
• Title: Iterative Optimal Attention and Local Model for Single Image Rain Streak Removal
• Authors: Xiangyu Li, Wanshu Fan, Yue Shen, Cong Wang, Wei Wang, Xin Yang, Qiang Zhang, Dongsheng Zhou,
• Abstract summary: We propose an Expectation Maximization Reconstruction Transformer (EMResformer) for single image rain streak removal.<n>The EMResformer retains the key self-attention values for feature aggregation, enhancing local features to produce superior image reconstruction.<n>Our proposed EMResformer surpasses current state-of-the-art single image rain streak removal methods on both synthetic and real-world datasets.
• Score: 17.153392956724783
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-fidelity imaging is crucial for the successful safety supervision and intelligent deployment of vision-based measurement systems (VBMS). It ensures high-quality imaging in VBMS, which is fundamental for reliable visual measurement and analysis. However, imaging quality can be significantly impaired by adverse weather conditions, particularly rain, leading to blurred images and reduced contrast. Such impairments increase the risk of inaccurate evaluations and misinterpretations in VBMS. To address these limitations, we propose an Expectation Maximization Reconstruction Transformer (EMResformer) for single image rain streak removal. The EMResformer retains the key self-attention values for feature aggregation, enhancing local features to produce superior image reconstruction. Specifically, we propose an Expectation Maximization Block seamlessly integrated into the single image rain streak removal network, enhancing its ability to eliminate superfluous information and restore a cleaner background image. Additionally, to further enhance local information for improved detail rendition, we introduce a Local Model Residual Block, which integrates two local model blocks along with a sequence of convolutions and activation functions. This integration synergistically facilitates the extraction of more pertinent features for enhanced single image rain streak removal. Extensive experiments validate that our proposed EMResformer surpasses current state-of-the-art single image rain streak removal methods on both synthetic and real-world datasets, achieving an improved balance between model complexity and single image deraining performance. Furthermore, we evaluate the effectiveness of our method in VBMS scenarios, demonstrating that high-quality imaging significantly improves the accuracy and reliability of VBMS tasks.

Related papers

• DPMambaIR:All-in-One Image Restoration via Degradation-Aware Prompt State Space Model [36.979833523678614]
  All-in-One image restoration aims to address multiple image degradation problems. Existing approaches rely on Degradation-specific models or coarse-grained degradation prompts to guide image restoration. We propose DPMambaIR, a novel All-in-One image restoration framework.
  arXiv  Detail & Related papers  (2025-04-24T16:46:32Z)
• Timestep-Aware Diffusion Model for Extreme Image Rescaling [47.89362819768323]
  We propose a novel framework called Timestep-Aware Diffusion Model (TADM) for extreme image rescaling.<n>TADM performs rescaling operations in the latent space of a pre-trained autoencoder.<n>It effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-08-17T09:51:42Z)
• Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration. We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• MvKSR: Multi-view Knowledge-guided Scene Recovery for Hazy and Rainy Degradation [8.955174143860681]
  High-quality imaging is crucial for ensuring safety supervision and intelligent deployment in fields like transportation and industry. Bad weather conditions, such as atmospheric haziness and precipitation, can have a significant impact on image quality. This paper proposes a novel knowledge-guided scene recovery network (termed MvKSR) to restore degraded images in hazy and rainy conditions.
  arXiv  Detail & Related papers  (2024-01-08T10:41:45Z)
• Multi-task Image Restoration Guided By Robust DINO Features [88.74005987908443]
  We propose mboxtextbfDINO-IR, a multi-task image restoration approach leveraging robust features extracted from DINOv2. We first propose a pixel-semantic fusion (PSF) module to dynamically fuse DINOV2's shallow features. By formulating these modules into a unified deep model, we propose a DINO perception contrastive loss to constrain the model training.
  arXiv  Detail & Related papers  (2023-12-04T06:59:55Z)
• A Unified Conditional Framework for Diffusion-based Image Restoration [39.418415473235235]
  We present a unified conditional framework based on diffusion models for image restoration. We leverage a lightweight UNet to predict initial guidance and the diffusion model to learn the residual of the guidance. To handle high-resolution images, we propose a simple yet effective inter-step patch-splitting strategy.
  arXiv  Detail & Related papers  (2023-05-31T17:22:24Z)
• Robust Single Image Dehazing Based on Consistent and Contrast-Assisted Reconstruction [95.5735805072852]
  We propose a novel density-variational learning framework to improve the robustness of the image dehzing model. Specifically, the dehazing network is optimized under the consistency-regularized framework. Our method significantly surpasses the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-29T08:11:04Z)
• From Rain Generation to Rain Removal [67.71728610434698]
  We build a full Bayesian generative model for rainy image where the rain layer is parameterized as a generator. We employ the variational inference framework to approximate the expected statistical distribution of rainy image. Comprehensive experiments substantiate that the proposed model can faithfully extract the complex rain distribution.
  arXiv  Detail & Related papers  (2020-08-08T18:56:51Z)
• Gated Fusion Network for Degraded Image Super Resolution [78.67168802945069]
  We propose a dual-branch convolutional neural network to extract base features and recovered features separately. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process.
  arXiv  Detail & Related papers  (2020-03-02T13:28:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.