MBA-RainGAN: Multi-branch Attention Generative Adversarial Network for Mixture of Rain Removal from Single Images

• URL: http://arxiv.org/abs/2005.10582v2
• Date: Sat, 23 May 2020 00:21:00 GMT
• Title: MBA-RainGAN: Multi-branch Attention Generative Adversarial Network for Mixture of Rain Removal from Single Images
• Authors: Yiyang Shen, Yidan Feng, Sen Deng, Dong Liang, Jing Qin, Haoran Xie, Mingqiang Wei
• Abstract summary: Rain severely hampers the visibility of scene objects when images are captured through glass in heavily rainy days. We observe three intriguing phenomenons that, 1) rain is a mixture of raindrops, rain streaks and rainy haze; 2) the depth from the camera determines the degrees of object visibility; and 3) raindrops on the glass randomly affect the object visibility of the whole image space.
• Score: 24.60495609529114
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Rain severely hampers the visibility of scene objects when images are captured through glass in heavily rainy days. We observe three intriguing phenomenons that, 1) rain is a mixture of raindrops, rain streaks and rainy haze; 2) the depth from the camera determines the degrees of object visibility, where objects nearby and faraway are visually blocked by rain streaks and rainy haze, respectively; and 3) raindrops on the glass randomly affect the object visibility of the whole image space. We for the first time consider that, the overall visibility of objects is determined by the mixture of rain (MOR). However, existing solutions and established datasets lack full consideration of the MOR. In this work, we first formulate a new rain imaging model; by then, we enrich the popular RainCityscapes by considering raindrops, named RainCityscapes++. Furthermore, we propose a multi-branch attention generative adversarial network (termed an MBA-RainGAN) to fully remove the MOR. The experiment shows clear visual and numerical improvements of our approach over the state-of-the-arts on RainCityscapes++. The code and dataset will be available.

Related papers

• NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative Learning for Dynamic Driving Scenes [49.92839157944134]
  In nighttime driving scenes, insufficient and uneven lighting shrouds the scenes in darkness, resulting degradation of image quality and visibility. We develop an image de-raining framework tailored for rainy nighttime driving scenes. It aims to remove rain artifacts, enrich scene representation, and restore useful information.
  arXiv  Detail & Related papers  (2024-02-28T09:02:33Z)
• Joint Depth Estimation and Mixture of Rain Removal From a Single Image [24.009353523566162]
  We propose an effective image deraining paradigm for Mixture of rain REmoval, called DEMore-Net. This study explores normalization approaches in image deraining tasks and introduces a new Hybrid Normalization Block (HNB) to enhance the deraining performance of DEMore-Net.
  arXiv  Detail & Related papers  (2023-03-31T02:05:45Z)
• Not Just Streaks: Towards Ground Truth for Single Image Deraining [42.15398478201746]
  We propose a large-scale dataset of real-world rainy and clean image pairs. We propose a deep neural network that reconstructs the underlying scene by minimizing a rain-robust loss between rainy and clean images.
  arXiv  Detail & Related papers  (2022-06-22T00:10:06Z)
• Semi-MoreGAN: A New Semi-supervised Generative Adversarial Network for Mixture of Rain Removal [18.04268933542476]
  We propose a new SEMI-supervised Mixture Of rain REmoval Generative Adversarial Network (Semi-MoreGAN) Semi-MoreGAN consists of four key modules: (I) a novel attentional depth prediction network to provide precise depth estimation; (ii) a context feature prediction network composed of several well-designed detailed residual blocks to produce detailed image context features; (iii) a pyramid depth-guided non-local network to effectively integrate the image context with the depth information, and produce the final rain-free images; and (iv) a comprehensive semi-supervised loss function to make the model not limited
  arXiv  Detail & Related papers  (2022-04-28T11:35:26Z)
• Semi-DRDNet Semi-supervised Detail-recovery Image Deraining Network via Unpaired Contrastive Learning [59.22620253308322]
  We propose a semi-supervised detail-recovery image deraining network (termed as Semi-DRDNet) As a semi-supervised learning paradigm, Semi-DRDNet operates smoothly on both synthetic and real-world rainy data in terms of deraining robustness and detail accuracy.
  arXiv  Detail & Related papers  (2022-04-06T12:35:27Z)
• Towards Robust Rain Removal Against Adversarial Attacks: A Comprehensive Benchmark Analysis and Beyond [85.06231315901505]
  Rain removal aims to remove rain streaks from images/videos and reduce the disruptive effects caused by rain. This paper makes the first attempt to conduct a comprehensive study on the robustness of deep learning-based rain removal methods against adversarial attacks.
  arXiv  Detail & Related papers  (2022-03-31T10:22:24Z)
• M2GAN: A Multi-Stage Self-Attention Network for Image Rain Removal on Autonomous Vehicles [8.642603456626391]
  We propose a new multi-stage multi-task recurrent generative adversarial network (M2GAN) to deal with challenging problems of raindrops hitting the car's windshield. M2GAN is considered the first method to deal with challenging problems of real-world rains under unconstrained environments such as autonomous vehicles.
  arXiv  Detail & Related papers  (2021-10-12T16:58:33Z)
• UnfairGAN: An Enhanced Generative Adversarial Network for Raindrop Removal from A Single Image [8.642603456626391]
  UnfairGAN is an enhanced generative adversarial network that can utilize prior high-level information, such as edges and rain estimation, to boost deraining performance. We show that our proposed method is superior to other state-of-the-art approaches of deraining raindrops regarding quantitative metrics and visual quality.
  arXiv  Detail & Related papers  (2021-10-11T18:02:43Z)
• Beyond Monocular Deraining: Parallel Stereo Deraining Network Via Semantic Prior [103.49307603952144]
  Most existing de-rain algorithms use only one single input image and aim to recover a clean image. We present a Paired Rain Removal Network (PRRNet), which exploits both stereo images and semantic information. Experiments on both monocular and the newly proposed stereo rainy datasets demonstrate that the proposed method achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-05-09T04:15:10Z)
• Dual Attention-in-Attention Model for Joint Rain Streak and Raindrop Removal [103.4067418083549]
  We propose a Dual Attention-in-Attention Model (DAiAM) which includes two DAMs for removing both rain streaks and raindrops simultaneously. The proposed method not only is capable of removing rain streaks and raindrops simultaneously, but also achieves the state-of-the-art performance on both tasks.
  arXiv  Detail & Related papers  (2021-03-12T03:00:33Z)
• Structural Residual Learning for Single Image Rain Removal [48.87977695398587]
  This study proposes a new network architecture by enforcing the output residual of the network possess intrinsic rain structures. Such a structural residual setting guarantees the rain layer extracted by the network finely comply with the prior knowledge of general rain streaks.
  arXiv  Detail & Related papers  (2020-05-19T05:52:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.