Is There Tradeoff between Spatial and Temporal in Video Super-Resolution?

• URL: http://arxiv.org/abs/2003.06141v1
• Date: Fri, 13 Mar 2020 07:49:05 GMT
• Title: Is There Tradeoff between Spatial and Temporal in Video Super-Resolution?
• Authors: Haochen Zhang and Dong Liu and Zhiwei Xiong
• Abstract summary: Advanced algorithms have been proposed to exploit the temporal correlation between low-resolution (LR) video frames, and/or to super-resolve a frame with multiple LR frames. These methods pursue higher quality of super-resolved frames, where the quality is usually measured frame by frame in e.g. PSNR. It is a natural requirement to improve both frame-wise fidelity and between-frame consistency, which are termed spatial quality and temporal quality, respectively.
• Score: 50.70768797616581
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances of deep learning lead to great success of image and video super-resolution (SR) methods that are based on convolutional neural networks (CNN). For video SR, advanced algorithms have been proposed to exploit the temporal correlation between low-resolution (LR) video frames, and/or to super-resolve a frame with multiple LR frames. These methods pursue higher quality of super-resolved frames, where the quality is usually measured frame by frame in e.g. PSNR. However, frame-wise quality may not reveal the consistency between frames. If an algorithm is applied to each frame independently (which is the case of most previous methods), the algorithm may cause temporal inconsistency, which can be observed as flickering. It is a natural requirement to improve both frame-wise fidelity and between-frame consistency, which are termed spatial quality and temporal quality, respectively. Then we may ask, is a method optimized for spatial quality also optimized for temporal quality? Can we optimize the two quality metrics jointly?

Related papers

• RIGID: Recurrent GAN Inversion and Editing of Real Face Videos [73.97520691413006]
  GAN inversion is indispensable for applying the powerful editability of GAN to real images. Existing methods invert video frames individually often leading to undesired inconsistent results over time. We propose a unified recurrent framework, named textbfRecurrent vtextbfIdeo textbfGAN textbfInversion and etextbfDiting (RIGID) Our framework learns the inherent coherence between input frames in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-08-11T12:17:24Z)
• Video Frame Interpolation without Temporal Priors [91.04877640089053]
  Video frame aims to synthesize non-exist intermediate frames in a video sequence. The temporal priors of videos, i.e. frames per second (FPS) and frame exposure time, may vary from different camera sensors. We devise a novel optical flow refinement strategy for better synthesizing results.
  arXiv  Detail & Related papers  (2021-12-02T12:13:56Z)
• FastRIFE: Optimization of Real-Time Intermediate Flow Estimation for Video Frame Interpolation [0.0]
  This paper proposes the FastRIFE algorithm, which is some speed improvement of the RIFE (Real-Time Intermediate Flow Estimation) model. All source codes are available at https://gitlab.com/malwinq/interpolation-of-images-for-slow-motion-videos.
  arXiv  Detail & Related papers  (2021-05-27T22:31:40Z)
• Zooming SlowMo: An Efficient One-Stage Framework for Space-Time Video Super-Resolution [100.11355888909102]
  Space-time video super-resolution aims at generating a high-resolution (HR) slow-motion video from a low-resolution (LR) and low frame rate (LFR) video sequence. We present a one-stage space-time video super-resolution framework, which can directly reconstruct an HR slow-motion video sequence from an input LR and LFR video.
  arXiv  Detail & Related papers  (2021-04-15T17:59:23Z)
• Efficient Space-time Video Super Resolution using Low-Resolution Flow and Mask Upsampling [12.856102293479486]
  This paper aims to generate High-resolution Slow-motion videos from Low Resolution and Low Frame rate videos. A simplistic solution is the sequential running of Video Super Resolution and Video Frame models. Our model is lightweight and performs better than current state-of-the-art models in REDS STSR validation set.
  arXiv  Detail & Related papers  (2021-04-12T19:11:57Z)
• Efficient Semantic Video Segmentation with Per-frame Inference [117.97423110566963]
  In this work, we process efficient semantic video segmentation in a per-frame fashion during the inference process. We employ compact models for real-time execution. To narrow the performance gap between compact models and large models, new knowledge distillation methods are designed.
  arXiv  Detail & Related papers  (2020-02-26T12:24:32Z)
• Video Face Super-Resolution with Motion-Adaptive Feedback Cell [90.73821618795512]
  Video super-resolution (VSR) methods have recently achieved a remarkable success due to the development of deep convolutional neural networks (CNN) In this paper, we propose a Motion-Adaptive Feedback Cell (MAFC), a simple but effective block, which can efficiently capture the motion compensation and feed it back to the network in an adaptive way.
  arXiv  Detail & Related papers  (2020-02-15T13:14:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.