Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation

• URL: http://arxiv.org/abs/2103.13660v1
• Date: Thu, 25 Mar 2021 08:21:43 GMT
• Title: Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation
• Authors: Yuntong Ye, Yi Chang, Hanyu Zhou, Luxin Yan
• Abstract summary: We argue that the rain generation and removal are the two sides of the same coin and should be tightly coupled. We propose a bidirectional disentangled translation network, in which each unidirectional network contains two loops of joint rain generation and removal. Experiments on synthetic and real-world rain datasets show the superiority of proposed method compared to state-of-the-arts.
• Score: 12.639320247831181
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing deep learning-based image deraining methods have achieved promising performance for synthetic rainy images, typically rely on the pairs of sharp images and simulated rainy counterparts. However, these methods suffer from significant performance drop when facing the real rain, because of the huge gap between the simplified synthetic rain and the complex real rain. In this work, we argue that the rain generation and removal are the two sides of the same coin and should be tightly coupled. To close the loop, we propose to jointly learn real rain generation and removal procedure within a unified disentangled image translation framework. Specifically, we propose a bidirectional disentangled translation network, in which each unidirectional network contains two loops of joint rain generation and removal for both the real and synthetic rain image, respectively. Meanwhile, we enforce the disentanglement strategy by decomposing the rainy image into a clean background and rain layer (rain removal), in order to better preserve the identity background via both the cycle-consistency loss and adversarial loss, and ease the rain layer translating between the real and synthetic rainy image. A counterpart composition with the entanglement strategy is symmetrically applied for rain generation. Extensive experiments on synthetic and real-world rain datasets show the superiority of proposed method compared to state-of-the-arts.

Related papers

• TRG-Net: An Interpretable and Controllable Rain Generator [61.2760968459789]
  This study proposes a novel deep learning based rain generator, which fully takes the physical generation mechanism underlying rains into consideration. Its significance lies in that the generator not only elaborately design essential elements of the rain to simulate expected rains, but also finely adapt to complicated and diverse practical rainy images. Our unpaired generation experiments demonstrate that the rain generated by the proposed rain generator is not only of higher quality, but also more effective for deraining and downstream tasks.
  arXiv  Detail & Related papers  (2024-03-15T03:27:39Z)
• TPSeNCE: Towards Artifact-Free Realistic Rain Generation for Deraining and Object Detection in Rain [23.050711662981655]
  We propose an unpaired image-to-image translation framework for generating realistic rainy images. We first introduce a Triangular Probability Similarity constraint to guide the generated images toward clear and rainy images in the discriminator manifold. Experiments demonstrate realistic rain generation with minimal artifacts and distortions, which benefits image deraining and object detection in rain.
  arXiv  Detail & Related papers  (2023-11-01T17:08:26Z)
• Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streaks [17.00078021737863]
  In the real world, image degradations caused by rain often exhibit a combination of rain streaks and raindrops, thereby increasing the challenges of recovering the underlying clean image. This paper aims to present an efficient and flexible mechanism to learn and model degradation relationships in a global view.
  arXiv  Detail & Related papers  (2023-08-27T16:33:11Z)
• Dual Degradation Representation for Joint Deraining and Low-Light Enhancement in the Dark [57.85378202032541]
  Rain in the dark poses a significant challenge to deploying real-world applications such as autonomous driving, surveillance systems, and night photography. Existing low-light enhancement or deraining methods struggle to brighten low-light conditions and remove rain simultaneously. We introduce an end-to-end model called L$2$RIRNet, designed to manage both low-light enhancement and deraining in real-world settings.
  arXiv  Detail & Related papers  (2023-05-06T10:17:42Z)
• Unsupervised Deraining: Where Asymmetric Contrastive Learning Meets Self-similarity [33.188738443097336]
  In this paper, we explore the intrinsic intra-similarity within each layer and inter-exclusiveness between two layers. We propose an unsupervised non-local contrastive learning (NLCL) deraining method.
  arXiv  Detail & Related papers  (2022-11-02T02:47:20Z)
• Semi-DRDNet Semi-supervised Detail-recovery Image Deraining Network via Unpaired Contrastive Learning [59.22620253308322]
  We propose a semi-supervised detail-recovery image deraining network (termed as Semi-DRDNet) As a semi-supervised learning paradigm, Semi-DRDNet operates smoothly on both synthetic and real-world rainy data in terms of deraining robustness and detail accuracy.
  arXiv  Detail & Related papers  (2022-04-06T12:35:27Z)
• Deep Single Image Deraining using An Asymetric Cycle Generative and Adversarial Framework [16.59494337699748]
  We propose a novel Asymetric Cycle Generative and Adrial framework (ACGF) for single image deraining. ACGF trains on both synthetic and real rainy images while simultaneously capturing both rain streaks and fog features. Experiments on benchmark rain-fog and rain datasets show that ACGF outperforms state-of-the-art deraining methods.
  arXiv  Detail & Related papers  (2022-02-19T16:14:10Z)
• Dual Attention-in-Attention Model for Joint Rain Streak and Raindrop Removal [103.4067418083549]
  We propose a Dual Attention-in-Attention Model (DAiAM) which includes two DAMs for removing both rain streaks and raindrops simultaneously. The proposed method not only is capable of removing rain streaks and raindrops simultaneously, but also achieves the state-of-the-art performance on both tasks.
  arXiv  Detail & Related papers  (2021-03-12T03:00:33Z)
• Structural Residual Learning for Single Image Rain Removal [48.87977695398587]
  This study proposes a new network architecture by enforcing the output residual of the network possess intrinsic rain structures. Such a structural residual setting guarantees the rain layer extracted by the network finely comply with the prior knowledge of general rain streaks.
  arXiv  Detail & Related papers  (2020-05-19T05:52:13Z)
• Semi-DerainGAN: A New Semi-supervised Single Image Deraining Network [45.78251508028359]
  We propose a new semi-supervised GAN-based deraining network termed Semi-DerainGAN. It can use both synthetic and real rainy images in a uniform network using two supervised and unsupervised processes. To deliver better deraining results, we design a paired discriminator for distinguishing the real pairs from fake pairs.
  arXiv  Detail & Related papers  (2020-01-23T07:01:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.