Unidirectional Video Denoising by Mimicking Backward Recurrent Modules with Look-ahead Forward Ones

• URL: http://arxiv.org/abs/2204.05532v1
• Date: Tue, 12 Apr 2022 05:33:15 GMT
• Title: Unidirectional Video Denoising by Mimicking Backward Recurrent Modules with Look-ahead Forward Ones
• Authors: Junyi Li, Xiaohe Wu, Zhenxin Niu, and Wangmeng Zuo
• Abstract summary: Bidirectional recurrent networks (BiRNN) have exhibited appealing performance in several video restoration tasks. BiRNN is intrinsically offline because it uses backward recurrent modules to propagate from the last to current frames. We present a novel recurrent network consisting of forward and look-ahead recurrent modules for unidirectional video denoising.
• Score: 72.68740880786312
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While significant progress has been made in deep video denoising, it remains very challenging for exploiting historical and future frames. Bidirectional recurrent networks (BiRNN) have exhibited appealing performance in several video restoration tasks. However, BiRNN is intrinsically offline because it uses backward recurrent modules to propagate from the last to current frames, which causes high latency and large memory consumption. To address the offline issue of BiRNN, we present a novel recurrent network consisting of forward and look-ahead recurrent modules for unidirectional video denoising. Particularly, look-ahead module is an elaborate forward module for leveraging information from near-future frames. When denoising the current frame, the hidden features by forward and look-ahead recurrent modules are combined, thereby making it feasible to exploit both historical and near-future frames. Due to the scene motion between non-neighboring frames, border pixels missing may occur when warping look-ahead feature from near-future frame to current frame, which can be largely alleviated by incorporating forward warping and border enlargement. Experiments show that our method achieves state-of-the-art performance with constant latency and memory consumption. The source code and pre-trained models will be publicly available.

Related papers

• A Practical Gated Recurrent Transformer Network Incorporating Multiple Fusions for Video Denoising [1.5044085747326295]
  State-of-the-art (SOTA) video denoising methods employ multi-frame simultaneous denoising mechanisms. We propose a multi-fusion gated recurrent Transformer network (GRTN) that achieves SOTA denoising performance with only a single-frame delay.
  arXiv  Detail & Related papers  (2024-09-10T15:55:53Z)
• LaneTCA: Enhancing Video Lane Detection with Temporal Context Aggregation [87.71768494466959]
  LaneTCA bridges the individual video frames and explore how to effectively aggregate the temporal context. We develop an accumulative attention module and an adjacent attention module to abstract the long-term and short-term temporal context. The two modules are meticulously designed based on the transformer architecture.
  arXiv  Detail & Related papers  (2024-08-25T14:46:29Z)
• Recurrent Self-Supervised Video Denoising with Denser Receptive Field [33.3711070590966]
  Self-supervised video denoising has seen decent progress through the use of blind spot networks. Previous self-supervised video denoising methods suffer from significant information loss and texture destruction in either the whole reference frame or neighbor frames. We propose RDRF for self-supervised video denoising, which fully exploits both the reference and neighbor frames with a denser receptive field.
  arXiv  Detail & Related papers  (2023-08-07T14:09:08Z)
• Towards Smooth Video Composition [59.134911550142455]
  Video generation requires consistent and persistent frames with dynamic content over time. This work investigates modeling the temporal relations for composing video with arbitrary length, from a few frames to even infinite, using generative adversarial networks (GANs) We show that the alias-free operation for single image generation, together with adequately pre-learned knowledge, brings a smooth frame transition without compromising the per-frame quality.
  arXiv  Detail & Related papers  (2022-12-14T18:54:13Z)
• Real-time Streaming Video Denoising with Bidirectional Buffers [48.57108807146537]
  Real-time denoising algorithms are typically adopted on the user device to remove the noise involved during the shooting and transmission of video streams. Recent multi-output inference works propagate the bidirectional temporal feature with a parallel or recurrent framework. We propose a Bidirectional Streaming Video Denoising framework, to achieve high-fidelity real-time denoising for streaming videos with both past and future temporal receptive fields.
  arXiv  Detail & Related papers  (2022-07-14T14:01:03Z)
• Pay Attention to Hidden States for Video Deblurring: Ping-Pong Recurrent Neural Networks and Selective Non-Local Attention [58.49075799159015]
  We propose 2 modules to supplement the RNN architecture for video deblurring. First, we design Ping-Pong RNN that acts on updating the hidden states by referring to the features from the current and the previous time steps alternately. Second, we use a Selective Non-Local Attention(SNLA) module to additionally refine the hidden state by aligning it with the positional information from the input frame feature.
  arXiv  Detail & Related papers  (2022-03-30T05:21:05Z)
• Memory-Augmented Non-Local Attention for Video Super-Resolution [61.55700315062226]
  We propose a novel video super-resolution method that aims at generating high-fidelity high-resolution (HR) videos from low-resolution (LR) ones. Previous methods predominantly leverage temporal neighbor frames to assist the super-resolution of the current frame. In contrast, we devise a cross-frame non-local attention mechanism that allows video super-resolution without frame alignment.
  arXiv  Detail & Related papers  (2021-08-25T05:12:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.