Towards General and Fast Video Derain via Knowledge Distillation
- URL: http://arxiv.org/abs/2308.05346v1
- Date: Thu, 10 Aug 2023 05:27:43 GMT
- Title: Towards General and Fast Video Derain via Knowledge Distillation
- Authors: Defang Cai, Pan Mu, Sixian Chan, Zhanpeng Shao, Cong Bai
- Abstract summary: We propose a Rain Review-based General video derain Network via knowledge distillation (named RRGNet)
We design a frame grouping-based encoder-decoder network that makes full use of the temporal information of the video.
To consolidate the network's ability to derain, we design a rain review module to play back data from old tasks for the current model.
- Score: 10.614356931086267
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: As a common natural weather condition, rain can obscure video frames and thus
affect the performance of the visual system, so video derain receives a lot of
attention. In natural environments, rain has a wide variety of streak types,
which increases the difficulty of the rain removal task. In this paper, we
propose a Rain Review-based General video derain Network via knowledge
distillation (named RRGNet) that handles different rain streak types with one
pre-training weight. Specifically, we design a frame grouping-based
encoder-decoder network that makes full use of the temporal information of the
video. Further, we use the old task model to guide the current model in
learning new rain streak types while avoiding forgetting. To consolidate the
network's ability to derain, we design a rain review module to play back data
from old tasks for the current model. The experimental results show that our
developed general method achieves the best results in terms of running speed
and derain effect.
Related papers
- RainMamba: Enhanced Locality Learning with State Space Models for Video Deraining [14.025870185802463]
We present an improved SSMs-based video deraining network (RainMamba) with a novel Hilbert mechanism to better capture sequence-level local information.
We also introduce a difference-guided dynamic contrastive locality learning strategy to enhance the patch-level self-similarity learning ability of the proposed network.
arXiv Detail & Related papers (2024-07-31T17:48:22Z) - TRG-Net: An Interpretable and Controllable Rain Generator [61.2760968459789]
This study proposes a novel deep learning based rain generator, which fully takes the physical generation mechanism underlying rains into consideration.
Its significance lies in that the generator not only elaborately design essential elements of the rain to simulate expected rains, but also finely adapt to complicated and diverse practical rainy images.
Our unpaired generation experiments demonstrate that the rain generated by the proposed rain generator is not only of higher quality, but also more effective for deraining and downstream tasks.
arXiv Detail & Related papers (2024-03-15T03:27:39Z) - Feature-Aligned Video Raindrop Removal with Temporal Constraints [68.49161092870224]
Raindrop removal is challenging for both single image and video.
Unlike rain streaks, adherent raindrops tend to cover the same area in several frames.
Our method employs a two-stage video-based raindrop removal method.
arXiv Detail & Related papers (2022-05-29T05:42:14Z) - Semi-DRDNet Semi-supervised Detail-recovery Image Deraining Network via
Unpaired Contrastive Learning [59.22620253308322]
We propose a semi-supervised detail-recovery image deraining network (termed as Semi-DRDNet)
As a semi-supervised learning paradigm, Semi-DRDNet operates smoothly on both synthetic and real-world rainy data in terms of deraining robustness and detail accuracy.
arXiv Detail & Related papers (2022-04-06T12:35:27Z) - UnfairGAN: An Enhanced Generative Adversarial Network for Raindrop
Removal from A Single Image [8.642603456626391]
UnfairGAN is an enhanced generative adversarial network that can utilize prior high-level information, such as edges and rain estimation, to boost deraining performance.
We show that our proposed method is superior to other state-of-the-art approaches of deraining raindrops regarding quantitative metrics and visual quality.
arXiv Detail & Related papers (2021-10-11T18:02:43Z) - RCDNet: An Interpretable Rain Convolutional Dictionary Network for
Single Image Deraining [49.99207211126791]
We specifically build a novel deep architecture, called rain convolutional dictionary network (RCDNet)
RCDNet embeds the intrinsic priors of rain streaks and has clear interpretability.
By end-to-end training such an interpretable network, all involved rain kernels and proximal operators can be automatically extracted.
arXiv Detail & Related papers (2021-07-14T16:08:11Z) - Beyond Monocular Deraining: Parallel Stereo Deraining Network Via
Semantic Prior [103.49307603952144]
Most existing de-rain algorithms use only one single input image and aim to recover a clean image.
We present a Paired Rain Removal Network (PRRNet), which exploits both stereo images and semantic information.
Experiments on both monocular and the newly proposed stereo rainy datasets demonstrate that the proposed method achieves the state-of-the-art performance.
arXiv Detail & Related papers (2021-05-09T04:15:10Z) - Rain Streak Removal in a Video to Improve Visibility by TAWL Algorithm [12.056495277232118]
We propose a novel method by combining three novel extracted features focusing on temporal appearance, wide shape and relative location of the rain streak.
The proposed TAWL method adaptively uses features from different resolutions and frame rates to remove rain in the real-time.
The experiments have been conducted using video sequences with both real rains and synthetic rains to compare the performance of the proposed method against the relevant state-of-the-art methods.
arXiv Detail & Related papers (2020-07-10T05:07:59Z) - Structural Residual Learning for Single Image Rain Removal [48.87977695398587]
This study proposes a new network architecture by enforcing the output residual of the network possess intrinsic rain structures.
Such a structural residual setting guarantees the rain layer extracted by the network finely comply with the prior knowledge of general rain streaks.
arXiv Detail & Related papers (2020-05-19T05:52:13Z) - Physical Model Guided Deep Image Deraining [10.14977592107907]
Single image deraining is an urgent task because the degraded rainy image makes many computer vision systems fail to work.
We propose a novel network based on physical model guided learning for single image deraining.
arXiv Detail & Related papers (2020-03-30T07:08:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.