Harnessing Joint Rain-/Detail-aware Representations to Eliminate Intricate Rains

• URL: http://arxiv.org/abs/2404.12091v1
• Date: Thu, 18 Apr 2024 11:20:53 GMT
• Title: Harnessing Joint Rain-/Detail-aware Representations to Eliminate Intricate Rains
• Authors: Wu Ran, Peirong Ma, Zhiquan He, Hao Ren, Hong Lu,
• Abstract summary: We develop a Context-based Instance-level Modulation mechanism adept at efficiently modulating CNN- or Transformer-based models. We also devise a rain-/detail-aware contrastive learning strategy to help extract joint rain-/detail-aware representations. By integrating CoI-M with the rain-/detail-aware Contrastive learning, we develop CoIC, an innovative and potent algorithm tailored for training models on mixed datasets.
• Score: 9.6606245317525
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in image deraining have focused on training powerful models on mixed multiple datasets comprising diverse rain types and backgrounds. However, this approach tends to overlook the inherent differences among rainy images, leading to suboptimal results. To overcome this limitation, we focus on addressing various rainy images by delving into meaningful representations that encapsulate both the rain and background components. Leveraging these representations as instructive guidance, we put forth a Context-based Instance-level Modulation (CoI-M) mechanism adept at efficiently modulating CNN- or Transformer-based models. Furthermore, we devise a rain-/detail-aware contrastive learning strategy to help extract joint rain-/detail-aware representations. By integrating CoI-M with the rain-/detail-aware Contrastive learning, we develop CoIC, an innovative and potent algorithm tailored for training models on mixed datasets. Moreover, CoIC offers insight into modeling relationships of datasets, quantitatively assessing the impact of rain and details on restoration, and unveiling distinct behaviors of models given diverse inputs. Extensive experiments validate the efficacy of CoIC in boosting the deraining ability of CNN and Transformer models. CoIC also enhances the deraining prowess remarkably when real-world dataset is included.

Related papers

• TRG-Net: An Interpretable and Controllable Rain Generator [61.2760968459789]
  This study proposes a novel deep learning based rain generator, which fully takes the physical generation mechanism underlying rains into consideration. Its significance lies in that the generator not only elaborately design essential elements of the rain to simulate expected rains, but also finely adapt to complicated and diverse practical rainy images. Our unpaired generation experiments demonstrate that the rain generated by the proposed rain generator is not only of higher quality, but also more effective for deraining and downstream tasks.
  arXiv  Detail & Related papers  (2024-03-15T03:27:39Z)
• Contrastive Learning Based Recursive Dynamic Multi-Scale Network for Image Deraining [47.764883957379745]
  Rain streaks significantly decrease the visibility of captured images. Existing deep learning-based image deraining methods employ manually crafted networks and learn a straightforward projection from rainy images to clear images. We propose a contrastive learning-based image deraining method that investigates the correlation between rainy and clear images.
  arXiv  Detail & Related papers  (2023-05-29T13:51:41Z)
• Semi-MoreGAN: A New Semi-supervised Generative Adversarial Network for Mixture of Rain Removal [18.04268933542476]
  We propose a new SEMI-supervised Mixture Of rain REmoval Generative Adversarial Network (Semi-MoreGAN) Semi-MoreGAN consists of four key modules: (I) a novel attentional depth prediction network to provide precise depth estimation; (ii) a context feature prediction network composed of several well-designed detailed residual blocks to produce detailed image context features; (iii) a pyramid depth-guided non-local network to effectively integrate the image context with the depth information, and produce the final rain-free images; and (iv) a comprehensive semi-supervised loss function to make the model not limited
  arXiv  Detail & Related papers  (2022-04-28T11:35:26Z)
• Structure-Preserving Deraining with Residue Channel Prior Guidance [33.41254475191555]
  Single image deraining is important for many high-level computer vision tasks. We propose a Structure-Preserving Deraining Network (SPDNet) with RCP guidance. SPDNet directly generates high-quality rain-free images with clear and accurate structures under RCP guidance.
  arXiv  Detail & Related papers  (2021-08-20T09:09:56Z)
• Towards a Unified Approach to Single Image Deraining and Dehazing [16.383099109400156]
  We develop a new physical model for the rain effect and show that the well-known atmosphere scattering model (ASM) for the haze effect naturally emerges as its homogeneous continuous limit. We also propose a Densely Scale-Connected Attentive Network (DSCAN) that is suitable for both deraining and dehazing tasks.
  arXiv  Detail & Related papers  (2021-03-26T01:35:43Z)
• Semi-Supervised Video Deraining with Dynamic Rain Generator [59.71640025072209]
  This paper proposes a new semi-supervised video deraining method, in which a dynamic rain generator is employed to fit the rain layer. Specifically, such dynamic generator consists of one emission model and one transition model to simultaneously encode the spatially physical structure and temporally continuous changes of rain streaks. Various prior formats are designed for the labeled synthetic and unlabeled real data, so as to fully exploit the common knowledge underlying them.
  arXiv  Detail & Related papers  (2021-03-14T14:28:57Z)
• From Rain Generation to Rain Removal [67.71728610434698]
  We build a full Bayesian generative model for rainy image where the rain layer is parameterized as a generator. We employ the variational inference framework to approximate the expected statistical distribution of rainy image. Comprehensive experiments substantiate that the proposed model can faithfully extract the complex rain distribution.
  arXiv  Detail & Related papers  (2020-08-08T18:56:51Z)
• Structural Residual Learning for Single Image Rain Removal [48.87977695398587]
  This study proposes a new network architecture by enforcing the output residual of the network possess intrinsic rain structures. Such a structural residual setting guarantees the rain layer extracted by the network finely comply with the prior knowledge of general rain streaks.
  arXiv  Detail & Related papers  (2020-05-19T05:52:13Z)
• Multi-Scale Progressive Fusion Network for Single Image Deraining [84.0466298828417]
  Rain streaks in the air appear in various blurring degrees and resolutions due to different distances from their positions to the camera. Similar rain patterns are visible in a rain image as well as its multi-scale (or multi-resolution) versions. In this work, we explore the multi-scale collaborative representation for rain streaks from the perspective of input image scales and hierarchical deep features.
  arXiv  Detail & Related papers  (2020-03-24T17:22:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.