Enhanced Spatio-Temporal Interaction Learning for Video Deraining: A Faster and Better Framework

• URL: http://arxiv.org/abs/2103.12318v1
• Date: Tue, 23 Mar 2021 05:19:35 GMT
• Title: Enhanced Spatio-Temporal Interaction Learning for Video Deraining: A Faster and Better Framework
• Authors: Kaihao Zhang, Dongxu Li, Wenhan Luo, Wen-Yan Lin, Fang Zhao, Wenqi Ren, Wei Liu, Hongdong Li
• Abstract summary: Video deraining is an important task in computer vision as the unwanted rain hampers the visibility of videos and deteriorates the robustness of most outdoor vision systems. We present a new end-to-end deraining framework, named Enhanced Spatio-Temporal Interaction Network (ESTINet) ESTINet considerably boosts current state-of-the-art video deraining quality and speed.
• Score: 93.37833982180538
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video deraining is an important task in computer vision as the unwanted rain hampers the visibility of videos and deteriorates the robustness of most outdoor vision systems. Despite the significant success which has been achieved for video deraining recently, two major challenges remain: 1) how to exploit the vast information among continuous frames to extract powerful spatio-temporal features across both the spatial and temporal domains, and 2) how to restore high-quality derained videos with a high-speed approach. In this paper, we present a new end-to-end video deraining framework, named Enhanced Spatio-Temporal Interaction Network (ESTINet), which considerably boosts current state-of-the-art video deraining quality and speed. The ESTINet takes the advantage of deep residual networks and convolutional long short-term memory, which can capture the spatial features and temporal correlations among continuing frames at the cost of very little computational source. Extensive experiments on three public datasets show that the proposed ESTINet can achieve faster speed than the competitors, while maintaining better performance than the state-of-the-art methods.

Related papers

• Online Video Quality Enhancement with Spatial-Temporal Look-up Tables [42.07242907586958]
  Low latency rates are crucial for online video-based applications, such as video conferencing and cloud gaming. Existing quality enhancement methods are limited by slow inference speed and the requirement for temporal information contained in future frames. We propose STLVQE, specifically designed to address the rarely studied online video quality enhancement (Online-VQE) problem.
  arXiv  Detail & Related papers  (2023-11-22T06:49:44Z)
• ReBotNet: Fast Real-time Video Enhancement [59.08038313427057]
  Most restoration networks are slow, have high computational bottleneck, and can't be used for real-time video enhancement. In this work, we design an efficient and fast framework to perform real-time enhancement for practical use-cases like live video calls and video streams. To evaluate our method, we emulate two new datasets that real-world video call and streaming scenarios, and show extensive results on multiple datasets where ReBotNet outperforms existing approaches with lower computations, reduced memory requirements, and faster inference time.
  arXiv  Detail & Related papers  (2023-03-23T17:58:05Z)
• Video Dehazing via a Multi-Range Temporal Alignment Network with Physical Prior [117.6741444489174]
  Video dehazing aims to recover haze-free frames with high visibility and contrast. This paper presents a novel framework to explore the physical haze priors and aggregate temporal information. We construct the first large-scale outdoor video dehazing benchmark dataset.
  arXiv  Detail & Related papers  (2023-03-17T03:44:17Z)
• Video Salient Object Detection via Contrastive Features and Attention Modules [106.33219760012048]
  We propose a network with attention modules to learn contrastive features for video salient object detection. A co-attention formulation is utilized to combine the low-level and high-level features. We show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-03T17:40:32Z)
• An Efficient Recurrent Adversarial Framework for Unsupervised Real-Time Video Enhancement [132.60976158877608]
  We propose an efficient adversarial video enhancement framework that learns directly from unpaired video examples. In particular, our framework introduces new recurrent cells that consist of interleaved local and global modules for implicit integration of spatial and temporal information. The proposed design allows our recurrent cells to efficiently propagate-temporal-information across frames and reduces the need for high complexity networks.
  arXiv  Detail & Related papers  (2020-12-24T00:03:29Z)
• Fast Video Salient Object Detection via Spatiotemporal Knowledge Distillation [20.196945571479002]
  We present a lightweight network tailored for video salient object detection. Specifically, we combine a saliency guidance embedding structure and spatial knowledge distillation to refine the spatial features. In the temporal aspect, we propose a temporal knowledge distillation strategy, which allows the network to learn the robust temporal features.
  arXiv  Detail & Related papers  (2020-10-20T04:48:36Z)
• Exploring Rich and Efficient Spatial Temporal Interactions for Real Time Video Salient Object Detection [87.32774157186412]
  Main stream methods formulate their video saliency mainly from two independent venues, i.e., the spatial and temporal branches. In this paper, we propose atemporal network to achieve such improvement in a full interactive fashion. Our method is easy to implement yet effective, achieving high quality video saliency detection in real-time speed with 50 FPS.
  arXiv  Detail & Related papers  (2020-08-07T03:24:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.