Unsupervised Deraining: Where Contrastive Learning Meets Self-similarity
- URL: http://arxiv.org/abs/2203.11509v1
- Date: Tue, 22 Mar 2022 07:37:08 GMT
- Title: Unsupervised Deraining: Where Contrastive Learning Meets Self-similarity
- Authors: Ye Yuntong, Yu Changfeng, Chang Yi, Zhu Lin, Zhao Xile, Yan Luxin and
Tian Yonghong
- Abstract summary: We propose a novel non-local contrastive learning (NLCL) method for unsupervised image deraining.
The proposed method obtains state-of-the-art performance in real deraining.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Image deraining is a typical low-level image restoration task, which aims at
decomposing the rainy image into two distinguishable layers: the clean image
layer and the rain layer. Most of the existing learning-based deraining methods
are supervisedly trained on synthetic rainy-clean pairs. The domain gap between
the synthetic and real rains makes them less generalized to different real
rainy scenes. Moreover, the existing methods mainly utilize the property of the
two layers independently, while few of them have considered the mutually
exclusive relationship between the two layers. In this work, we propose a novel
non-local contrastive learning (NLCL) method for unsupervised image deraining.
Consequently, we not only utilize the intrinsic self-similarity property within
samples but also the mutually exclusive property between the two layers, so as
to better differ the rain layer from the clean image. Specifically, the
non-local self-similarity image layer patches as the positives are pulled
together and similar rain layer patches as the negatives are pushed away. Thus
the similar positive/negative samples that are close in the original space
benefit us to enrich more discriminative representation. Apart from the
self-similarity sampling strategy, we analyze how to choose an appropriate
feature encoder in NLCL. Extensive experiments on different real rainy datasets
demonstrate that the proposed method obtains state-of-the-art performance in
real deraining.
Related papers
- TPSeNCE: Towards Artifact-Free Realistic Rain Generation for Deraining
and Object Detection in Rain [23.050711662981655]
We propose an unpaired image-to-image translation framework for generating realistic rainy images.
We first introduce a Triangular Probability Similarity constraint to guide the generated images toward clear and rainy images in the discriminator manifold.
Experiments demonstrate realistic rain generation with minimal artifacts and distortions, which benefits image deraining and object detection in rain.
arXiv Detail & Related papers (2023-11-01T17:08:26Z) - Contrastive Learning Based Recursive Dynamic Multi-Scale Network for
Image Deraining [47.764883957379745]
Rain streaks significantly decrease the visibility of captured images.
Existing deep learning-based image deraining methods employ manually crafted networks and learn a straightforward projection from rainy images to clear images.
We propose a contrastive learning-based image deraining method that investigates the correlation between rainy and clear images.
arXiv Detail & Related papers (2023-05-29T13:51:41Z) - Unsupervised Deraining: Where Asymmetric Contrastive Learning Meets
Self-similarity [33.188738443097336]
In this paper, we explore the intrinsic intra-similarity within each layer and inter-exclusiveness between two layers.
We propose an unsupervised non-local contrastive learning (NLCL) deraining method.
arXiv Detail & Related papers (2022-11-02T02:47:20Z) - Deep Single Image Deraining using An Asymetric Cycle Generative and
Adversarial Framework [16.59494337699748]
We propose a novel Asymetric Cycle Generative and Adrial framework (ACGF) for single image deraining.
ACGF trains on both synthetic and real rainy images while simultaneously capturing both rain streaks and fog features.
Experiments on benchmark rain-fog and rain datasets show that ACGF outperforms state-of-the-art deraining methods.
arXiv Detail & Related papers (2022-02-19T16:14:10Z) - Both Style and Fog Matter: Cumulative Domain Adaptation for Semantic
Foggy Scene Understanding [63.99301797430936]
We propose a new pipeline to cumulatively adapt style, fog and the dual-factor (style and fog)
Specifically, we devise a unified framework to disentangle the style factor and the fog factor separately, and then the dual-factor from images in different domains.
Our method achieves the state-of-the-art performance on three benchmarks and shows generalization ability in rainy and snowy scenes.
arXiv Detail & Related papers (2021-12-01T13:21:20Z) - Unpaired Adversarial Learning for Single Image Deraining with Rain-Space
Contrastive Constraints [61.40893559933964]
We develop an effective unpaired SID method which explores mutual properties of the unpaired exemplars by a contrastive learning manner in a GAN framework, named as CDR-GAN.
Our method performs favorably against existing unpaired deraining approaches on both synthetic and real-world datasets, even outperforms several fully-supervised or semi-supervised models.
arXiv Detail & Related papers (2021-09-07T10:00:45Z) - Closing the Loop: Joint Rain Generation and Removal via Disentangled
Image Translation [12.639320247831181]
We argue that the rain generation and removal are the two sides of the same coin and should be tightly coupled.
We propose a bidirectional disentangled translation network, in which each unidirectional network contains two loops of joint rain generation and removal.
Experiments on synthetic and real-world rain datasets show the superiority of proposed method compared to state-of-the-arts.
arXiv Detail & Related papers (2021-03-25T08:21:43Z) - Learning to See Through Obstructions with Layered Decomposition [117.77024641706451]
We present a learning-based approach for removing unwanted obstructions from moving images.
Our method leverages motion differences between the background and obstructing elements to recover both layers.
We show that the proposed approach learned from synthetically generated data performs well to real images.
arXiv Detail & Related papers (2020-08-11T17:59:31Z) - Structural Residual Learning for Single Image Rain Removal [48.87977695398587]
This study proposes a new network architecture by enforcing the output residual of the network possess intrinsic rain structures.
Such a structural residual setting guarantees the rain layer extracted by the network finely comply with the prior knowledge of general rain streaks.
arXiv Detail & Related papers (2020-05-19T05:52:13Z) - Multi-Scale Progressive Fusion Network for Single Image Deraining [84.0466298828417]
Rain streaks in the air appear in various blurring degrees and resolutions due to different distances from their positions to the camera.
Similar rain patterns are visible in a rain image as well as its multi-scale (or multi-resolution) versions.
In this work, we explore the multi-scale collaborative representation for rain streaks from the perspective of input image scales and hierarchical deep features.
arXiv Detail & Related papers (2020-03-24T17:22:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.