SAPNet: Segmentation-Aware Progressive Network for Perceptual Contrastive Deraining

• URL: http://arxiv.org/abs/2111.08892v1
• Date: Wed, 17 Nov 2021 03:57:11 GMT
• Title: SAPNet: Segmentation-Aware Progressive Network for Perceptual Contrastive Deraining
• Authors: Shen Zheng, Changjie Lu, Yuxiong Wu and Gaurav Gupta
• Abstract summary: We present a segmentation-aware progressive network (SAPNet) based upon contrastive learning for single image deraining. Our model surpasses top-performing methods and aids object detection and semantic segmentation with considerable efficacy.
• Score: 2.615176171489612
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning algorithms have recently achieved promising deraining performances on both the natural and synthetic rainy datasets. As an essential low-level pre-processing stage, a deraining network should clear the rain streaks and preserve the fine semantic details. However, most existing methods only consider low-level image restoration. That limits their performances at high-level tasks requiring precise semantic information. To address this issue, in this paper, we present a segmentation-aware progressive network (SAPNet) based upon contrastive learning for single image deraining. We start our method with a lightweight derain network formed with progressive dilated units (PDU). The PDU can significantly expand the receptive field and characterize multi-scale rain streaks without the heavy computation on multi-scale images. A fundamental aspect of this work is an unsupervised background segmentation (UBS) network initialized with ImageNet and Gaussian weights. The UBS can faithfully preserve an image's semantic information and improve the generalization ability to unseen photos. Furthermore, we introduce a perceptual contrastive loss (PCL) and a learned perceptual image similarity loss (LPISL) to regulate model learning. By exploiting the rainy image and groundtruth as the negative and the positive sample in the VGG-16 latent space, we bridge the fine semantic details between the derained image and the groundtruth in a fully constrained manner. Comprehensive experiments on synthetic and real-world rainy images show our model surpasses top-performing methods and aids object detection and semantic segmentation with considerable efficacy. A Pytorch Implementation is available at https://github.com/ShenZheng2000/SAPNet-for-image-deraining.

Related papers

• Dynamic Association Learning of Self-Attention and Convolution in Image Restoration [56.49098856632478]
  CNNs and Self attention have achieved great success in multimedia applications for dynamic association learning of self-attention and convolution in image restoration. This paper proposes an association learning method to utilize the advantages and suppress their shortcomings, so as to achieve high-quality and efficient inpainting.
  arXiv  Detail & Related papers  (2023-11-09T05:11:24Z)
• Contrastive Learning Based Recursive Dynamic Multi-Scale Network for Image Deraining [47.764883957379745]
  Rain streaks significantly decrease the visibility of captured images. Existing deep learning-based image deraining methods employ manually crafted networks and learn a straightforward projection from rainy images to clear images. We propose a contrastive learning-based image deraining method that investigates the correlation between rainy and clear images.
  arXiv  Detail & Related papers  (2023-05-29T13:51:41Z)
• Single Image Deraining via Feature-based Deep Convolutional Neural Network [13.39233717329633]
  A single image deraining algorithm based on the combination of data-driven and model-based approaches is proposed. Experiments show that the proposed algorithm significantly outperforms state-of-the-art methods in terms of both qualitative and quantitative measures.
  arXiv  Detail & Related papers  (2023-05-03T13:12:51Z)
• Semi-DRDNet Semi-supervised Detail-recovery Image Deraining Network via Unpaired Contrastive Learning [59.22620253308322]
  We propose a semi-supervised detail-recovery image deraining network (termed as Semi-DRDNet) As a semi-supervised learning paradigm, Semi-DRDNet operates smoothly on both synthetic and real-world rainy data in terms of deraining robustness and detail accuracy.
  arXiv  Detail & Related papers  (2022-04-06T12:35:27Z)
• Structure-Preserving Deraining with Residue Channel Prior Guidance [33.41254475191555]
  Single image deraining is important for many high-level computer vision tasks. We propose a Structure-Preserving Deraining Network (SPDNet) with RCP guidance. SPDNet directly generates high-quality rain-free images with clear and accurate structures under RCP guidance.
  arXiv  Detail & Related papers  (2021-08-20T09:09:56Z)
• RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining [49.99207211126791]
  We specifically build a novel deep architecture, called rain convolutional dictionary network (RCDNet) RCDNet embeds the intrinsic priors of rain streaks and has clear interpretability. By end-to-end training such an interpretable network, all involved rain kernels and proximal operators can be automatically extracted.
  arXiv  Detail & Related papers  (2021-07-14T16:08:11Z)
• Beyond Monocular Deraining: Parallel Stereo Deraining Network Via Semantic Prior [103.49307603952144]
  Most existing de-rain algorithms use only one single input image and aim to recover a clean image. We present a Paired Rain Removal Network (PRRNet), which exploits both stereo images and semantic information. Experiments on both monocular and the newly proposed stereo rainy datasets demonstrate that the proposed method achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-05-09T04:15:10Z)
• A Model-driven Deep Neural Network for Single Image Rain Removal [52.787356046951494]
  We propose a model-driven deep neural network for the task, with fully interpretable network structures. Based on the convolutional dictionary learning mechanism for representing rain, we propose a novel single image deraining model. All the rain kernels and operators can be automatically extracted, faithfully characterizing the features of both rain and clean background layers.
  arXiv  Detail & Related papers  (2020-05-04T09:13:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.