Towards Ground Truth for Single Image Deraining
- URL: http://arxiv.org/abs/2206.10779v1
- Date: Wed, 22 Jun 2022 00:10:06 GMT
- Title: Towards Ground Truth for Single Image Deraining
- Authors: Yunhao Ba, Howard Zhang, Ethan Yang, Akira Suzuki, Arnold Pfahnl,
Chethan Chinder Chandrappa, Celso de Melo, Suya You, Stefano Soatto, Alex
Wong, Achuta Kadambi
- Abstract summary: We propose a large-scale dataset of real-world rainy and clean image pairs.
We propose a deep neural network that reconstructs the underlying scene by minimizing a rain-invariant loss between rainy and clean images.
Our model can outperform the state-of-the-art deraining methods on real rainy images under various conditions.
- Score: 45.50400293855075
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We propose a large-scale dataset of real-world rainy and clean image pairs
and a method to remove degradations, induced by rain streaks and rain
accumulation, from the image. As there exists no real-world dataset for
deraining, current state-of-the-art methods rely on synthetic data and thus are
limited by the sim2real domain gap; moreover, rigorous evaluation remains a
challenge due to the absence of a real paired dataset. We fill this gap by
collecting the first real paired deraining dataset through meticulous control
of non-rain variations. Our dataset enables paired training and quantitative
evaluation for diverse real-world rain phenomena (e.g. rain streaks and rain
accumulation). To learn a representation invariant to rain phenomena, we
propose a deep neural network that reconstructs the underlying scene by
minimizing a rain-invariant loss between rainy and clean images. Extensive
experiments demonstrate that the proposed dataset benefits existing derainers,
and our model can outperform the state-of-the-art deraining methods on real
rainy images under various conditions.
Related papers
- TRG-Net: An Interpretable and Controllable Rain Generator [61.2760968459789]
This study proposes a novel deep learning based rain generator, which fully takes the physical generation mechanism underlying rains into consideration.
Its significance lies in that the generator not only elaborately design essential elements of the rain to simulate expected rains, but also finely adapt to complicated and diverse practical rainy images.
Our unpaired generation experiments demonstrate that the rain generated by the proposed rain generator is not only of higher quality, but also more effective for deraining and downstream tasks.
arXiv Detail & Related papers (2024-03-15T03:27:39Z) - TPSeNCE: Towards Artifact-Free Realistic Rain Generation for Deraining
and Object Detection in Rain [23.050711662981655]
We propose an unpaired image-to-image translation framework for generating realistic rainy images.
We first introduce a Triangular Probability Similarity constraint to guide the generated images toward clear and rainy images in the discriminator manifold.
Experiments demonstrate realistic rain generation with minimal artifacts and distortions, which benefits image deraining and object detection in rain.
arXiv Detail & Related papers (2023-11-01T17:08:26Z) - Rethinking Real-world Image Deraining via An Unpaired Degradation-Conditioned Diffusion Model [51.49854435403139]
We propose RainDiff, the first real-world image deraining paradigm based on diffusion models.
We introduce a stable and non-adversarial unpaired cycle-consistent architecture that can be trained, end-to-end, with only unpaired data for supervision.
We also propose a degradation-conditioned diffusion model that refines the desired output via a diffusive generative process conditioned by learned priors of multiple rain degradations.
arXiv Detail & Related papers (2023-01-23T13:34:01Z) - Toward Real-world Single Image Deraining: A New Benchmark and Beyond [79.5893880599847]
Single image deraining (SID) in real scenarios attracts increasing attention in recent years.
Previous real datasets suffer from low-resolution images, homogeneous rain streaks, limited background variation, and even misalignment of image pairs.
We establish a new high-quality dataset named RealRain-1k, consisting of $1,120$ high-resolution paired clean and rainy images with low- and high-density rain streaks, respectively.
arXiv Detail & Related papers (2022-06-11T12:26:59Z) - Semi-MoreGAN: A New Semi-supervised Generative Adversarial Network for
Mixture of Rain Removal [18.04268933542476]
We propose a new SEMI-supervised Mixture Of rain REmoval Generative Adversarial Network (Semi-MoreGAN)
Semi-MoreGAN consists of four key modules: (I) a novel attentional depth prediction network to provide precise depth estimation; (ii) a context feature prediction network composed of several well-designed detailed residual blocks to produce detailed image context features; (iii) a pyramid depth-guided non-local network to effectively integrate the image context with the depth information, and produce the final rain-free images; and (iv) a comprehensive semi-supervised loss function to make the model not limited
arXiv Detail & Related papers (2022-04-28T11:35:26Z) - From Rain Generation to Rain Removal [67.71728610434698]
We build a full Bayesian generative model for rainy image where the rain layer is parameterized as a generator.
We employ the variational inference framework to approximate the expected statistical distribution of rainy image.
Comprehensive experiments substantiate that the proposed model can faithfully extract the complex rain distribution.
arXiv Detail & Related papers (2020-08-08T18:56:51Z) - Structural Residual Learning for Single Image Rain Removal [48.87977695398587]
This study proposes a new network architecture by enforcing the output residual of the network possess intrinsic rain structures.
Such a structural residual setting guarantees the rain layer extracted by the network finely comply with the prior knowledge of general rain streaks.
arXiv Detail & Related papers (2020-05-19T05:52:13Z) - Rainy screens: Collecting rainy datasets, indoors [19.71705192452036]
We present a simple method for generating diverse rainy images from existing clear ground-truth images.
This setup allows us to leverage the diversity of existing datasets with auxiliary task ground-truth data.
We generate rainy images with real adherent droplets and rain streaks based on Cityscapes and BDD, and train a de-raining model.
arXiv Detail & Related papers (2020-03-10T13:57:37Z) - Semi-DerainGAN: A New Semi-supervised Single Image Deraining Network [45.78251508028359]
We propose a new semi-supervised GAN-based deraining network termed Semi-DerainGAN.
It can use both synthetic and real rainy images in a uniform network using two supervised and unsupervised processes.
To deliver better deraining results, we design a paired discriminator for distinguishing the real pairs from fake pairs.
arXiv Detail & Related papers (2020-01-23T07:01:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.