ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring

• URL: http://arxiv.org/abs/2103.04260v1
• Date: Sun, 7 Mar 2021 04:33:13 GMT
• Title: ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring
• Authors: Dongxu Li, Chenchen Xu, Kaihao Zhang, Xin Yu, Yiran Zhong, Wenqi Ren, Hanna Suominen, Hongdong Li
• Abstract summary: Video deblurring models exploit consecutive frames to remove blurs from camera shakes and object motions. We propose a novel implicit method to learn spatial correspondence among blurry frames in the feature space. Our proposed method is evaluated on the widely-adopted DVD dataset, along with a newly collected High-Frame-Rate (1000 fps) dataset for Video Deblurring.
• Score: 92.40655035360729
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video deblurring models exploit consecutive frames to remove blurs from camera shakes and object motions. In order to utilize neighboring sharp patches, typical methods rely mainly on homography or optical flows to spatially align neighboring blurry frames. However, such explicit approaches are less effective in the presence of fast motions with large pixel displacements. In this work, we propose a novel implicit method to learn spatial correspondence among blurry frames in the feature space. To construct distant pixel correspondences, our model builds a correlation volume pyramid among all the pixel-pairs between neighboring frames. To enhance the features of the reference frame, we design a correlative aggregation module that maximizes the pixel-pair correlations with its neighbors based on the volume pyramid. Finally, we feed the aggregated features into a reconstruction module to obtain the restored frame. We design a generative adversarial paradigm to optimize the model progressively. Our proposed method is evaluated on the widely-adopted DVD dataset, along with a newly collected High-Frame-Rate (1000 fps) Dataset for Video Deblurring (HFR-DVD). Quantitative and qualitative experiments show that our model performs favorably on both datasets against previous state-of-the-art methods, confirming the benefit of modeling all-range spatial correspondence for video deblurring.

Related papers

• Disentangled Motion Modeling for Video Frame Interpolation [40.83962594702387]
  Video frame (VFI) aims to synthesize intermediate frames in between existing frames to enhance visual smoothness and quality. We introduce disentangled Motion Modeling (MoMo), a diffusion-based approach for VFI that enhances visual quality by focusing on intermediate motion modeling.
  arXiv  Detail & Related papers  (2024-06-25T03:50:20Z)
• Motion-Aware Video Frame Interpolation [49.49668436390514]
  We introduce a Motion-Aware Video Frame Interpolation (MA-VFI) network, which directly estimates intermediate optical flow from consecutive frames. It not only extracts global semantic relationships and spatial details from input frames with different receptive fields, but also effectively reduces the required computational cost and complexity.
  arXiv  Detail & Related papers  (2024-02-05T11:00:14Z)
• PDWN: Pyramid Deformable Warping Network for Video Interpolation [11.62213584807003]
  We propose a light but effective model, called Pyramid Deformable Warping Network (PDWN) PDWN uses a pyramid structure to generate DConv offsets of the unknown middle frame with respect to the known frames through coarse-to-fine successive refinements. Our method achieves better or on-par accuracy compared to state-of-the-art models on multiple datasets.
  arXiv  Detail & Related papers  (2021-04-04T02:08:57Z)
• Motion-blurred Video Interpolation and Extrapolation [72.3254384191509]
  We present a novel framework for deblurring, interpolating and extrapolating sharp frames from a motion-blurred video in an end-to-end manner. To ensure temporal coherence across predicted frames and address potential temporal ambiguity, we propose a simple, yet effective flow-based rule.
  arXiv  Detail & Related papers  (2021-03-04T12:18:25Z)
• FineNet: Frame Interpolation and Enhancement for Face Video Deblurring [18.49184807837449]
  The aim of this work is to deblur face videos. We propose a method that tackles this problem from two directions: (1) enhancing the blurry frames, and (2) treating the blurry frames as missing values and estimate them by objective. Experiments on three real and synthetically generated video datasets show that our method outperforms the previous state-of-the-art methods by a large margin in terms of both quantitative and qualitative results.
  arXiv  Detail & Related papers  (2021-03-01T09:47:16Z)
• FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation [97.99012124785177]
  FLAVR is a flexible and efficient architecture that uses 3D space-time convolutions to enable end-to-end learning and inference for video framesupervised. We demonstrate that FLAVR can serve as a useful self- pretext task for action recognition, optical flow estimation, and motion magnification.
  arXiv  Detail & Related papers  (2020-12-15T18:59:30Z)
• ALANET: Adaptive Latent Attention Network forJoint Video Deblurring and Interpolation [38.52446103418748]
  We introduce a novel architecture, Adaptive Latent Attention Network (ALANET), which synthesizes sharp high frame-rate videos. We employ combination of self-attention and cross-attention module between consecutive frames in the latent space to generate optimized representation for each frame. Our method performs favorably against various state-of-the-art approaches, even though we tackle a much more difficult problem.
  arXiv  Detail & Related papers  (2020-08-31T21:11:53Z)
• All at Once: Temporally Adaptive Multi-Frame Interpolation with Advanced Motion Modeling [52.425236515695914]
  State-of-the-art methods are iterative solutions interpolating one frame at the time. This work introduces a true multi-frame interpolator. It utilizes a pyramidal style network in the temporal domain to complete the multi-frame task in one-shot.
  arXiv  Detail & Related papers  (2020-07-23T02:34:39Z)
• Blurry Video Frame Interpolation [57.77512131536132]
  We propose a blurry video frame method to reduce blur motion and up-convert frame rate simultaneously. Specifically, we develop a pyramid module to cyclically synthesize clear intermediate frames. Our method performs favorably against state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-02-27T17:00:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.