ALANET: Adaptive Latent Attention Network forJoint Video Deblurring and
Interpolation
- URL: http://arxiv.org/abs/2009.01005v1
- Date: Mon, 31 Aug 2020 21:11:53 GMT
- Title: ALANET: Adaptive Latent Attention Network forJoint Video Deblurring and
Interpolation
- Authors: Akash Gupta, Abhishek Aich, Amit K. Roy-Chowdhury
- Abstract summary: We introduce a novel architecture, Adaptive Latent Attention Network (ALANET), which synthesizes sharp high frame-rate videos.
We employ combination of self-attention and cross-attention module between consecutive frames in the latent space to generate optimized representation for each frame.
Our method performs favorably against various state-of-the-art approaches, even though we tackle a much more difficult problem.
- Score: 38.52446103418748
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Existing works address the problem of generating high frame-rate sharp videos
by separately learning the frame deblurring and frame interpolation modules.
Most of these approaches have a strong prior assumption that all the input
frames are blurry whereas in a real-world setting, the quality of frames
varies. Moreover, such approaches are trained to perform either of the two
tasks - deblurring or interpolation - in isolation, while many practical
situations call for both. Different from these works, we address a more
realistic problem of high frame-rate sharp video synthesis with no prior
assumption that input is always blurry. We introduce a novel architecture,
Adaptive Latent Attention Network (ALANET), which synthesizes sharp high
frame-rate videos with no prior knowledge of input frames being blurry or not,
thereby performing the task of both deblurring and interpolation. We
hypothesize that information from the latent representation of the consecutive
frames can be utilized to generate optimized representations for both frame
deblurring and frame interpolation. Specifically, we employ combination of
self-attention and cross-attention module between consecutive frames in the
latent space to generate optimized representation for each frame. The optimized
representation learnt using these attention modules help the model to generate
and interpolate sharp frames. Extensive experiments on standard datasets
demonstrate that our method performs favorably against various state-of-the-art
approaches, even though we tackle a much more difficult problem.
Related papers
- Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation [93.18163456287164]
This paper proposes a novel text-guided video-to-video translation framework to adapt image models to videos.
Our framework achieves global style and local texture temporal consistency at a low cost.
arXiv Detail & Related papers (2023-06-13T17:52:23Z) - TTVFI: Learning Trajectory-Aware Transformer for Video Frame
Interpolation [50.49396123016185]
Video frame (VFI) aims to synthesize an intermediate frame between two consecutive frames.
We propose a novel Trajectory-aware Transformer for Video Frame Interpolation (TTVFI)
Our method outperforms other state-of-the-art methods in four widely-used VFI benchmarks.
arXiv Detail & Related papers (2022-07-19T03:37:49Z) - Video Frame Interpolation without Temporal Priors [91.04877640089053]
Video frame aims to synthesize non-exist intermediate frames in a video sequence.
The temporal priors of videos, i.e. frames per second (FPS) and frame exposure time, may vary from different camera sensors.
We devise a novel optical flow refinement strategy for better synthesizing results.
arXiv Detail & Related papers (2021-12-02T12:13:56Z) - TimeLens: Event-based Video Frame Interpolation [54.28139783383213]
We introduce Time Lens, a novel indicates equal contribution method that leverages the advantages of both synthesis-based and flow-based approaches.
We show an up to 5.21 dB improvement in terms of PSNR over state-of-the-art frame-based and event-based methods.
arXiv Detail & Related papers (2021-06-14T10:33:47Z) - ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring [92.40655035360729]
Video deblurring models exploit consecutive frames to remove blurs from camera shakes and object motions.
We propose a novel implicit method to learn spatial correspondence among blurry frames in the feature space.
Our proposed method is evaluated on the widely-adopted DVD dataset, along with a newly collected High-Frame-Rate (1000 fps) dataset for Video Deblurring.
arXiv Detail & Related papers (2021-03-07T04:33:13Z) - FineNet: Frame Interpolation and Enhancement for Face Video Deblurring [18.49184807837449]
The aim of this work is to deblur face videos.
We propose a method that tackles this problem from two directions: (1) enhancing the blurry frames, and (2) treating the blurry frames as missing values and estimate them by objective.
Experiments on three real and synthetically generated video datasets show that our method outperforms the previous state-of-the-art methods by a large margin in terms of both quantitative and qualitative results.
arXiv Detail & Related papers (2021-03-01T09:47:16Z) - Deep Sketch-guided Cartoon Video Inbetweening [24.00033622396297]
We propose a framework to produce cartoon videos by fetching the color information from two inputs while following the animated motion guided by a user sketch.
By explicitly considering the correspondence between frames and the sketch, we can achieve higher quality results than other image synthesis methods.
arXiv Detail & Related papers (2020-08-10T14:22:04Z) - All at Once: Temporally Adaptive Multi-Frame Interpolation with Advanced
Motion Modeling [52.425236515695914]
State-of-the-art methods are iterative solutions interpolating one frame at the time.
This work introduces a true multi-frame interpolator.
It utilizes a pyramidal style network in the temporal domain to complete the multi-frame task in one-shot.
arXiv Detail & Related papers (2020-07-23T02:34:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.