A Prototype Unit for Image De-raining using Time-Lapse Data

• URL: http://arxiv.org/abs/2412.19459v1
• Date: Fri, 27 Dec 2024 05:04:56 GMT
• Title: A Prototype Unit for Image De-raining using Time-Lapse Data
• Authors: Jaehoon Cho, Minjung Yoo, Jini Yang, Sunok Kim,
• Abstract summary: We address the challenge of single-image de-raining, a task that involves recovering rain-free background information from a single rain image. We introduce a novel solution: the Rain Streak Prototype Unit (RsPU) The RsPU efficiently encodes rain streak-relevant features as real-time prototypes derived from time-lapse data, eliminating the need for excessive memory resources.
• Score: 9.37072441362836
• License:
• Abstract: We address the challenge of single-image de-raining, a task that involves recovering rain-free background information from a single rain image. While recent advancements have utilized real-world time-lapse data for training, enabling the estimation of consistent backgrounds and realistic rain streaks, these methods often suffer from computational and memory consumption, limiting their applicability in real-world scenarios. In this paper, we introduce a novel solution: the Rain Streak Prototype Unit (RsPU). The RsPU efficiently encodes rain streak-relevant features as real-time prototypes derived from time-lapse data, eliminating the need for excessive memory resources. Our de-raining network combines encoder-decoder networks with the RsPU, allowing us to learn and encapsulate diverse rain streak-relevant features as concise prototypes, employing an attention-based approach. To ensure the effectiveness of our approach, we propose a feature prototype loss encompassing cohesion and divergence components. This loss function captures both the compactness and diversity aspects of the prototypical rain streak features within the RsPU. Our method evaluates various de-raining benchmarks, accompanied by comprehensive ablation studies. We show that it can achieve competitive results in various rain images compared to state-of-the-art methods.

Related papers

• Contrastive Learning Based Recursive Dynamic Multi-Scale Network for Image Deraining [47.764883957379745]
  Rain streaks significantly decrease the visibility of captured images. Existing deep learning-based image deraining methods employ manually crafted networks and learn a straightforward projection from rainy images to clear images. We propose a contrastive learning-based image deraining method that investigates the correlation between rainy and clear images.
  arXiv  Detail & Related papers  (2023-05-29T13:51:41Z)
• Single Image Deraining via Feature-based Deep Convolutional Neural Network [13.39233717329633]
  A single image deraining algorithm based on the combination of data-driven and model-based approaches is proposed. Experiments show that the proposed algorithm significantly outperforms state-of-the-art methods in terms of both qualitative and quantitative measures.
  arXiv  Detail & Related papers  (2023-05-03T13:12:51Z)
• Uncertainty-Aware Cascaded Dilation Filtering for High-Efficiency Deraining [25.669665033163497]
  Deraining is a significant and fundamental computer vision task, aiming to remove the rain streaks and accumulations in an image or video captured under a rainy day. Existing deraining methods usually make assumptions of the rain model, which compels them to employ complex optimization or iterative refinement for high recovery quality. We propose a simple yet efficient deraining method by formulating deraining as a predictive filtering problem without complex rain model assumptions.
  arXiv  Detail & Related papers  (2022-01-07T08:31:57Z)
• Memory-guided Image De-raining Using Time-Lapse Data [83.12497916664904]
  We address the problem of single image de-raining, that is, the task of recovering clean and rain-free background scenes from a single image obscured by a rainy artifact. We propose a novel network architecture based on a memory network that explicitly helps to capture long-term rain streak information in the time-lapse data.
  arXiv  Detail & Related papers  (2022-01-06T01:36:59Z)
• Structure-Preserving Deraining with Residue Channel Prior Guidance [33.41254475191555]
  Single image deraining is important for many high-level computer vision tasks. We propose a Structure-Preserving Deraining Network (SPDNet) with RCP guidance. SPDNet directly generates high-quality rain-free images with clear and accurate structures under RCP guidance.
  arXiv  Detail & Related papers  (2021-08-20T09:09:56Z)
• RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining [49.99207211126791]
  We specifically build a novel deep architecture, called rain convolutional dictionary network (RCDNet) RCDNet embeds the intrinsic priors of rain streaks and has clear interpretability. By end-to-end training such an interpretable network, all involved rain kernels and proximal operators can be automatically extracted.
  arXiv  Detail & Related papers  (2021-07-14T16:08:11Z)
• EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining [29.11278650245332]
  Single-image deraining is rather challenging due to the unknown rain model. In this paper, we propose a model-free deraining method, i.e., EfficientDeRain. It is able to process a rainy image within 10ms (i.e., around 6ms on average), over 80 times faster than the state-of-the-art method.
  arXiv  Detail & Related papers  (2020-09-19T14:32:50Z)
• From Rain Generation to Rain Removal [67.71728610434698]
  We build a full Bayesian generative model for rainy image where the rain layer is parameterized as a generator. We employ the variational inference framework to approximate the expected statistical distribution of rainy image. Comprehensive experiments substantiate that the proposed model can faithfully extract the complex rain distribution.
  arXiv  Detail & Related papers  (2020-08-08T18:56:51Z)
• Structural Residual Learning for Single Image Rain Removal [48.87977695398587]
  This study proposes a new network architecture by enforcing the output residual of the network possess intrinsic rain structures. Such a structural residual setting guarantees the rain layer extracted by the network finely comply with the prior knowledge of general rain streaks.
  arXiv  Detail & Related papers  (2020-05-19T05:52:13Z)
• Multi-Scale Progressive Fusion Network for Single Image Deraining [84.0466298828417]
  Rain streaks in the air appear in various blurring degrees and resolutions due to different distances from their positions to the camera. Similar rain patterns are visible in a rain image as well as its multi-scale (or multi-resolution) versions. In this work, we explore the multi-scale collaborative representation for rain streaks from the perspective of input image scales and hierarchical deep features.
  arXiv  Detail & Related papers  (2020-03-24T17:22:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.