From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removal

• URL: http://arxiv.org/abs/2308.03867v2
• Date: Fri, 18 Aug 2023 14:46:04 GMT
• Title: From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removal
• Authors: Yun Guo, Xueyao Xiao, Yi Chang, Shumin Deng, Luxin Yan
• Abstract summary: We construct a Large-scale High-quality Paired real rain benchmark (LHP-Rain), including 3000 video sequences with 1 million high-resolution (1920*1080) frame pairs. The advantages of the proposed dataset are three-fold: rain with higher-diversity and larger-scale, image with higher-resolution and higher-quality ground-truth.
• Score: 28.029107707930063
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning-based image deraining methods have made great progress. However, the lack of large-scale high-quality paired training samples is the main bottleneck to hamper the real image deraining (RID). To address this dilemma and advance RID, we construct a Large-scale High-quality Paired real rain benchmark (LHP-Rain), including 3000 video sequences with 1 million high-resolution (1920*1080) frame pairs. The advantages of the proposed dataset over the existing ones are three-fold: rain with higher-diversity and larger-scale, image with higher-resolution and higher-quality ground-truth. Specifically, the real rains in LHP-Rain not only contain the classical rain streak/veiling/occlusion in the sky, but also the \textbf{splashing on the ground} overlooked by deraining community. Moreover, we propose a novel robust low-rank tensor recovery model to generate the GT with better separating the static background from the dynamic rain. In addition, we design a simple transformer-based single image deraining baseline, which simultaneously utilize the self-attention and cross-layer attention within the image and rain layer with discriminative feature representation. Extensive experiments verify the superiority of the proposed dataset and deraining method over state-of-the-art.

Related papers

• RainyScape: Unsupervised Rainy Scene Reconstruction using Decoupled Neural Rendering [50.14860376758962]
  We propose RainyScape, an unsupervised framework for reconstructing clean scenes from a collection of multi-view rainy images. Based on the spectral bias property of neural networks, we first optimize the neural rendering pipeline to obtain a low-frequency scene representation. We jointly optimize the two modules, driven by the proposed adaptive direction-sensitive gradient-based reconstruction loss.
  arXiv  Detail & Related papers  (2024-04-17T14:07:22Z)
• Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streaks [17.00078021737863]
  In the real world, image degradations caused by rain often exhibit a combination of rain streaks and raindrops, thereby increasing the challenges of recovering the underlying clean image. This paper aims to present an efficient and flexible mechanism to learn and model degradation relationships in a global view.
  arXiv  Detail & Related papers  (2023-08-27T16:33:11Z)
• A Two-Stage Real Image Deraining Method for GT-RAIN Challenge CVPR 2023 Workshop UG$^{\textbf{2}}$+ Track 3 [15.370704973282848]
  We propose an efficient two-stage framework to reconstruct a clear image from rainy frames. A transformer-based single image deraining network Uformer is implemented to pre-train on large real rain dataset. Our overall framework is elaborately designed and able to handle both heavy rainy and foggy sequences.
  arXiv  Detail & Related papers  (2023-05-13T18:30:27Z)
• Single Image Deraining via Feature-based Deep Convolutional Neural Network [13.39233717329633]
  A single image deraining algorithm based on the combination of data-driven and model-based approaches is proposed. Experiments show that the proposed algorithm significantly outperforms state-of-the-art methods in terms of both qualitative and quantitative measures.
  arXiv  Detail & Related papers  (2023-05-03T13:12:51Z)
• Toward Real-world Single Image Deraining: A New Benchmark and Beyond [79.5893880599847]
  Single image deraining (SID) in real scenarios attracts increasing attention in recent years. Previous real datasets suffer from low-resolution images, homogeneous rain streaks, limited background variation, and even misalignment of image pairs. We establish a new high-quality dataset named RealRain-1k, consisting of $1,120$ high-resolution paired clean and rainy images with low- and high-density rain streaks, respectively.
  arXiv  Detail & Related papers  (2022-06-11T12:26:59Z)
• RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining [49.99207211126791]
  We specifically build a novel deep architecture, called rain convolutional dictionary network (RCDNet) RCDNet embeds the intrinsic priors of rain streaks and has clear interpretability. By end-to-end training such an interpretable network, all involved rain kernels and proximal operators can be automatically extracted.
  arXiv  Detail & Related papers  (2021-07-14T16:08:11Z)
• SDNet: mutil-branch for single image deraining using swin [14.574622548559269]
  We introduce Swin-transformer into the field of image deraining for the first time. Specifically, we improve the basic module of Swin-transformer and design a three-branch model to implement single-image rain removal. Our proposed method has performance and inference speed advantages over the current mainstream single-image rain streaks removal models.
  arXiv  Detail & Related papers  (2021-05-31T16:06:02Z)
• From Rain Generation to Rain Removal [67.71728610434698]
  We build a full Bayesian generative model for rainy image where the rain layer is parameterized as a generator. We employ the variational inference framework to approximate the expected statistical distribution of rainy image. Comprehensive experiments substantiate that the proposed model can faithfully extract the complex rain distribution.
  arXiv  Detail & Related papers  (2020-08-08T18:56:51Z)
• Structural Residual Learning for Single Image Rain Removal [48.87977695398587]
  This study proposes a new network architecture by enforcing the output residual of the network possess intrinsic rain structures. Such a structural residual setting guarantees the rain layer extracted by the network finely comply with the prior knowledge of general rain streaks.
  arXiv  Detail & Related papers  (2020-05-19T05:52:13Z)
• Multi-Scale Progressive Fusion Network for Single Image Deraining [84.0466298828417]
  Rain streaks in the air appear in various blurring degrees and resolutions due to different distances from their positions to the camera. Similar rain patterns are visible in a rain image as well as its multi-scale (or multi-resolution) versions. In this work, we explore the multi-scale collaborative representation for rain streaks from the perspective of input image scales and hierarchical deep features.
  arXiv  Detail & Related papers  (2020-03-24T17:22:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.