RainDancer: RGB-Event Video Deraining with Rain-Oriented Spiking Dynamics

• URL: http://arxiv.org/abs/2607.13802v1
• Date: Wed, 15 Jul 2026 13:08:21 GMT
• Title: RainDancer: RGB-Event Video Deraining with Rain-Oriented Spiking Dynamics
• Abstract summary: Video deraining aims to recover clean visual content from rainy videos for reliable perception under adverse weather.<n>Existing methods mainly rely on RGB sequences and temporal redundancy, but RGB-only restoration remains ambiguous in dynamic rainy scenes.<n>We propose RainDancer, a progressive RGB-Event video deraining framework based on a decompose-before-interact paradigm.
• Score: 93.19893734404417
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video deraining aims to recover clean visual content from rainy videos for reliable perception under adverse weather. Existing methods mainly rely on RGB sequences and temporal redundancy, but RGB-only restoration remains ambiguous in dynamic rainy scenes, where rain streaks, textures, boundaries, motion, and occlusions may share similar visual patterns. Event cameras provide complementary motion-sensitive cues with high temporal resolution, but event streams also contain sensor noise and background-triggered responses, so direct RGB-Event fusion may introduce cross-modal interference. To address this issue, we propose RainDancer, a progressive RGB-Event video deraining framework based on a decompose-before-interact paradigm. The core idea is to separate rain and background components within each modality before cross-modal interaction. In the RGB branch, frame features are progressively decomposed into rain and background representations. In the event branch, a rain-oriented spiking neural network module captures sparse and bursty event dynamics associated with rain motion. Component-level fusion is then performed between semantically aligned representations for structure preservation and rain suppression. We further introduce event-domain supervision to regularize sparse event reconstruction, structural consistency, and gradient orientation. Experiments on synthetic and real RGB-Event video deraining datasets demonstrate superior quantitative performance, visual quality, and downstream perception robustness. Code is available at https://github.com/AE86-plus/RainDancer.

Related papers

• ENCORE: Event-Assisted Complementary Motion Refinement for Learned Video Compression [16.49973800168176]
  Event-Assisted Complementary Motion Refinement framework for learned video compression.<n> ENCORE achieves up to 20.80% PSNR-RGB and 22.14% MS-SSIM-RGB BD-rate savings.<n> experiments on BS-ERGB, HQ-EVFI, and CED demonstrate consistent gains across datasets and GOP lengths.
  arXiv  Detail & Related papers  (2026-07-30T11:10:13Z)
• Spatio-Temporal Difference Guided Motion Deblurring with the Complementary Vision Sensor [12.418360718687026]
  Motion blur arises when rapid scene changes occur during the exposure period, collapsing rich intra-exposure motion into a single RGB frame.<n>We propose Spatio-Temporal Difference Guided Deblur Net (STGDNet) to restore structure and color details lost in blurry RGB inputs.<n>Our method outperforms current RGB or event-based approaches in both synthetic CVS dataset and real-world evaluations.
  arXiv  Detail & Related papers  (2026-04-12T09:54:14Z)
• SpikeDerain: Unveiling Clear Videos from Rainy Sequences Using Color Spike Streams [49.34425133546994]
  Restoring clear frames from rainy videos presents a significant challenge due to the rapid motion of rain streaks.<n>Traditional frame-based visual sensors, which capture scene content synchronously, struggle to capture the fast-moving details of rain accurately.<n>We propose a Color Spike Stream Deraining Network (SpikeDerain), capable of reconstructing spike streams of dynamic scenes and accurately removing rain streaks.
  arXiv  Detail & Related papers  (2025-03-26T08:28:28Z)
• RainMamba: Enhanced Locality Learning with State Space Models for Video Deraining [14.025870185802463]
  We present an improved SSMs-based video deraining network (RainMamba) with a novel Hilbert mechanism to better capture sequence-level local information. We also introduce a difference-guided dynamic contrastive locality learning strategy to enhance the patch-level self-similarity learning ability of the proposed network.
  arXiv  Detail & Related papers  (2024-07-31T17:48:22Z)
• RainyScape: Unsupervised Rainy Scene Reconstruction using Decoupled Neural Rendering [50.14860376758962]
  We propose RainyScape, an unsupervised framework for reconstructing clean scenes from a collection of multi-view rainy images. Based on the spectral bias property of neural networks, we first optimize the neural rendering pipeline to obtain a low-frequency scene representation. We jointly optimize the two modules, driven by the proposed adaptive direction-sensitive gradient-based reconstruction loss.
  arXiv  Detail & Related papers  (2024-04-17T14:07:22Z)
• EGVD: Event-Guided Video Deraining [57.59935209162314]
  We propose an end-to-end learning-based network to unlock the potential of the event camera for video deraining. We build a real-world dataset consisting of rainy videos and temporally synchronized event streams.
  arXiv  Detail & Related papers  (2023-09-29T13:47:53Z)
• Feature-Aligned Video Raindrop Removal with Temporal Constraints [68.49161092870224]
  Raindrop removal is challenging for both single image and video. Unlike rain streaks, adherent raindrops tend to cover the same area in several frames. Our method employs a two-stage video-based raindrop removal method.
  arXiv  Detail & Related papers  (2022-05-29T05:42:14Z)
• RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining [49.99207211126791]
  We specifically build a novel deep architecture, called rain convolutional dictionary network (RCDNet) RCDNet embeds the intrinsic priors of rain streaks and has clear interpretability. By end-to-end training such an interpretable network, all involved rain kernels and proximal operators can be automatically extracted.
  arXiv  Detail & Related papers  (2021-07-14T16:08:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.