NightRain: Nighttime Video Deraining via Adaptive-Rain-Removal and Adaptive-Correction

• URL: http://arxiv.org/abs/2401.00729v2
• Date: Wed, 10 Jan 2024 06:31:17 GMT
• Title: NightRain: Nighttime Video Deraining via Adaptive-Rain-Removal and Adaptive-Correction
• Abstract summary: We introduce NightRain, a novel nighttime video deraining method with adaptive-rain-removal and adaptive-correction. Our method achieves a PSNR of 26.73dB, surpassing existing nighttime video deraining methods by a substantial margin of 13.7%.
• Score: 18.89549992743636
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Existing deep-learning-based methods for nighttime video deraining rely on synthetic data due to the absence of real-world paired data. However, the intricacies of the real world, particularly with the presence of light effects and low-light regions affected by noise, create significant domain gaps, hampering synthetic-trained models in removing rain streaks properly and leading to over-saturation and color shifts. Motivated by this, we introduce NightRain, a novel nighttime video deraining method with adaptive-rain-removal and adaptive-correction. Our adaptive-rain-removal uses unlabeled rain videos to enable our model to derain real-world rain videos, particularly in regions affected by complex light effects. The idea is to allow our model to obtain rain-free regions based on the confidence scores. Once rain-free regions and the corresponding regions from our input are obtained, we can have region-based paired real data. These paired data are used to train our model using a teacher-student framework, allowing the model to iteratively learn from less challenging regions to more challenging regions. Our adaptive-correction aims to rectify errors in our model's predictions, such as over-saturation and color shifts. The idea is to learn from clear night input training videos based on the differences or distance between those input videos and their corresponding predictions. Our model learns from these differences, compelling our model to correct the errors. From extensive experiments, our method demonstrates state-of-the-art performance. It achieves a PSNR of 26.73dB, surpassing existing nighttime video deraining methods by a substantial margin of 13.7%.

Related papers

• UENR-600K: A Large-Scale Physically Grounded Dataset for Nighttime Video Deraining [58.726887410862055]
  Nighttime video deraining is challenging because raindrops interact with artificial lighting.<n>UENR-600K is a large-scale, physically grounded dataset containing 600,000 1080p frame pairs.<n>We utilize Unreal Engine to simulate rain as 3D particles within virtual environments.
  arXiv  Detail & Related papers  (2026-04-06T04:01:46Z)
• Zero-Shot Video Deraining with Video Diffusion Models [11.578999728002065]
  We introduce the first zero-shot video deraining method for complex dynamic scenes that does not require synthetic data or model fine-tuning.<n>Our approach is validated through extensive experiments on real-world rain datasets.
  arXiv  Detail & Related papers  (2025-11-23T17:06:22Z)
• Semi-Supervised State-Space Model with Dynamic Stacking Filter for Real-World Video Deraining [73.5575992346396]
  We propose a dual-branch-temporal state-space model to enhance rain streak removal in video sequences.<n>To improve multi-frame feature fusion, we derive a dynamic filter stacking, which adaptively approximates statistical filters for pixel-wise feature refinement.<n>To further explore the capacity of deraining models in supporting other vision-based tasks in rainy environments, we introduce a novel real-world benchmark.
  arXiv  Detail & Related papers  (2025-05-22T15:50:00Z)
• Controllable Weather Synthesis and Removal with Video Diffusion Models [61.56193902622901]
  WeatherWeaver is a video diffusion model that synthesizes diverse weather effects directly into any input video. Our model provides precise control over weather effect intensity and supports blending various weather types, ensuring both realism and adaptability.
  arXiv  Detail & Related papers  (2025-05-01T17:59:57Z)
• SpikeDerain: Unveiling Clear Videos from Rainy Sequences Using Color Spike Streams [49.34425133546994]
  Restoring clear frames from rainy videos presents a significant challenge due to the rapid motion of rain streaks. Traditional frame-based visual sensors, which capture scene content synchronously, struggle to capture the fast-moving details of rain accurately. We propose a Color Spike Stream Deraining Network (SpikeDerain), capable of reconstructing spike streams of dynamic scenes and accurately removing rain streaks.
  arXiv  Detail & Related papers  (2025-03-26T08:28:28Z)
• RainMamba: Enhanced Locality Learning with State Space Models for Video Deraining [14.025870185802463]
  We present an improved SSMs-based video deraining network (RainMamba) with a novel Hilbert mechanism to better capture sequence-level local information. We also introduce a difference-guided dynamic contrastive locality learning strategy to enhance the patch-level self-similarity learning ability of the proposed network.
  arXiv  Detail & Related papers  (2024-07-31T17:48:22Z)
• Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal [53.15046196592023]
  We introduce test-time adaptation into adverse weather removal in videos. We propose the first framework that integrates test-time adaptation into the iterative diffusion reverse process.
  arXiv  Detail & Related papers  (2024-03-12T14:21:30Z)
• NiteDR: Nighttime Image De-Raining with Cross-View Sensor Cooperative Learning for Dynamic Driving Scenes [49.92839157944134]
  In nighttime driving scenes, insufficient and uneven lighting shrouds the scenes in darkness, resulting degradation of image quality and visibility. We develop an image de-raining framework tailored for rainy nighttime driving scenes. It aims to remove rain artifacts, enrich scene representation, and restore useful information.
  arXiv  Detail & Related papers  (2024-02-28T09:02:33Z)
• Towards General and Fast Video Derain via Knowledge Distillation [10.614356931086267]
  We propose a Rain Review-based General video derain Network via knowledge distillation (named RRGNet) We design a frame grouping-based encoder-decoder network that makes full use of the temporal information of the video. To consolidate the network's ability to derain, we design a rain review module to play back data from old tasks for the current model.
  arXiv  Detail & Related papers  (2023-08-10T05:27:43Z)
• Dual Degradation Representation for Joint Deraining and Low-Light Enhancement in the Dark [57.85378202032541]
  Rain in the dark poses a significant challenge to deploying real-world applications such as autonomous driving, surveillance systems, and night photography. Existing low-light enhancement or deraining methods struggle to brighten low-light conditions and remove rain simultaneously. We introduce an end-to-end model called L$2$RIRNet, designed to manage both low-light enhancement and deraining in real-world settings.
  arXiv  Detail & Related papers  (2023-05-06T10:17:42Z)
• Rethinking Real-world Image Deraining via An Unpaired Degradation-Conditioned Diffusion Model [51.49854435403139]
  We propose RainDiff, the first real-world image deraining paradigm based on diffusion models. We introduce a stable and non-adversarial unpaired cycle-consistent architecture that can be trained, end-to-end, with only unpaired data for supervision. We also propose a degradation-conditioned diffusion model that refines the desired output via a diffusive generative process conditioned by learned priors of multiple rain degradations.
  arXiv  Detail & Related papers  (2023-01-23T13:34:01Z)
• Not Just Streaks: Towards Ground Truth for Single Image Deraining [42.15398478201746]
  We propose a large-scale dataset of real-world rainy and clean image pairs. We propose a deep neural network that reconstructs the underlying scene by minimizing a rain-robust loss between rainy and clean images.
  arXiv  Detail & Related papers  (2022-06-22T00:10:06Z)
• RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs [79.00855490550367]
  We show that NeRF can produce photorealistic renderings of unseen viewpoints when many input views are available. We address this by regularizing the geometry and appearance of patches rendered from unobserved viewpoints. Our model outperforms not only other methods that optimize over a single scene, but also conditional models that are extensively pre-trained on large multi-view datasets.
  arXiv  Detail & Related papers  (2021-12-01T18:59:46Z)
• Semi-Supervised Video Deraining with Dynamic Rain Generator [59.71640025072209]
  This paper proposes a new semi-supervised video deraining method, in which a dynamic rain generator is employed to fit the rain layer. Specifically, such dynamic generator consists of one emission model and one transition model to simultaneously encode the spatially physical structure and temporally continuous changes of rain streaks. Various prior formats are designed for the labeled synthetic and unlabeled real data, so as to fully exploit the common knowledge underlying them.
  arXiv  Detail & Related papers  (2021-03-14T14:28:57Z)
• Rain Streak Removal in a Video to Improve Visibility by TAWL Algorithm [12.056495277232118]
  We propose a novel method by combining three novel extracted features focusing on temporal appearance, wide shape and relative location of the rain streak. The proposed TAWL method adaptively uses features from different resolutions and frame rates to remove rain in the real-time. The experiments have been conducted using video sequences with both real rains and synthetic rains to compare the performance of the proposed method against the relevant state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-10T05:07:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.