Benchmark Dataset and Effective Inter-Frame Alignment for Real-World Video Super-Resolution

• URL: http://arxiv.org/abs/2212.05342v1
• Date: Sat, 10 Dec 2022 17:41:46 GMT
• Title: Benchmark Dataset and Effective Inter-Frame Alignment for Real-World Video Super-Resolution
• Authors: Ruohao Wang, Xiaohui Liu, Zhilu Zhang, Xiaohe Wu, Chun-Mei Feng, Lei Zhang, Wangmeng Zuo
• Abstract summary: Video super-resolution (VSR) aiming to reconstruct a high-resolution (HR) video from its low-resolution (LR) counterpart has made tremendous progress in recent years. It remains challenging to deploy existing VSR methods to real-world data with complex degradations. EAVSR takes the proposed multi-layer adaptive spatial transform network (MultiAdaSTN) to refine the offsets provided by the pre-trained optical flow estimation network.
• Score: 65.20905703823965
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video super-resolution (VSR) aiming to reconstruct a high-resolution (HR) video from its low-resolution (LR) counterpart has made tremendous progress in recent years. However, it remains challenging to deploy existing VSR methods to real-world data with complex degradations. On the one hand, there are few well-aligned real-world VSR datasets, especially with large super-resolution scale factors, which limits the development of real-world VSR tasks. On the other hand, alignment algorithms in existing VSR methods perform poorly for real-world videos, leading to unsatisfactory results. As an attempt to address the aforementioned issues, we build a real-world 4 VSR dataset, namely MVSR4$\times$, where low- and high-resolution videos are captured with different focal length lenses of a smartphone, respectively. Moreover, we propose an effective alignment method for real-world VSR, namely EAVSR. EAVSR takes the proposed multi-layer adaptive spatial transform network (MultiAdaSTN) to refine the offsets provided by the pre-trained optical flow estimation network. Experimental results on RealVSR and MVSR4$\times$ datasets show the effectiveness and practicality of our method, and we achieve state-of-the-art performance in real-world VSR task. The dataset and code will be publicly available.

Related papers

• Enhanced Super-Resolution Training via Mimicked Alignment for Real-World Scenes [51.92255321684027]
  We propose a novel plug-and-play module designed to mitigate misalignment issues by aligning LR inputs with HR images during training. Specifically, our approach involves mimicking a novel LR sample that aligns with HR while preserving the characteristics of the original LR samples. We comprehensively evaluate our method on synthetic and real-world datasets, demonstrating its effectiveness across a spectrum of SR models.
  arXiv  Detail & Related papers  (2024-10-07T18:18:54Z)
• Towards Realistic Data Generation for Real-World Super-Resolution [58.88039242455039]
  RealDGen is an unsupervised learning data generation framework designed for real-world super-resolution. We develop content and degradation extraction strategies, which are integrated into a novel content-degradation decoupled diffusion model. Experiments demonstrate that RealDGen excels in generating large-scale, high-quality paired data that mirrors real-world degradations.
  arXiv  Detail & Related papers  (2024-06-11T13:34:57Z)
• Learning Dual-Level Deformable Implicit Representation for Real-World Scale Arbitrary Super-Resolution [81.74583887661794]
  We build a new real-world super-resolution benchmark with both integer and non-integer scaling factors. We propose a Dual-level Deformable Implicit Representation (DDIR) to solve real-world scale arbitrary super-resolution. Our trained model achieves state-of-the-art performance on the RealArbiSR and RealSR benchmarks for real-world scale arbitrary super-resolution.
  arXiv  Detail & Related papers  (2024-03-16T13:44:42Z)
• Expanding Synthetic Real-World Degradations for Blind Video Super Resolution [3.474523163017713]
  Video super-resolution (VSR) techniques have drastically improved over the last few years and shown impressive performance on synthetic data. However, their performance on real-world video data suffers because of the complexity of real-world degradations and misaligned video frames. In this paper, we propose real-world degradations on synthetic training datasets.
  arXiv  Detail & Related papers  (2023-05-04T08:58:31Z)
• Boosting Video Super Resolution with Patch-Based Temporal Redundancy Optimization [46.833568886576074]
  We discuss the influence of the temporal redundancy in the patches with stationary objects and background. We develop two simple yet effective plug and play methods to improve the performance of existing local and non-local propagation-based VSR algorithms.
  arXiv  Detail & Related papers  (2022-07-18T15:11:18Z)
• Real-Time Super-Resolution System of 4K-Video Based on Deep Learning [6.182364004551161]
  Video-resolution (VSR) technology excels in low-quality video computation, avoiding unpleasant blur effect caused by occupation-based algorithms. This paper explores the possibility of real-time VS system and designs an efficient generic VSR network, termed EGVSR. Compared with TecoGAN, the most advanced VSR network at present, we achieve 84% reduction of density and 7.92x performance speedups.
  arXiv  Detail & Related papers  (2021-07-12T10:35:05Z)
• DynaVSR: Dynamic Adaptive Blind Video Super-Resolution [60.154204107453914]
  DynaVSR is a novel meta-learning-based framework for real-world video SR. We train a multi-frame downscaling module with various types of synthetic blur kernels, which is seamlessly combined with a video SR network for input-aware adaptation. Experimental results show that DynaVSR consistently improves the performance of the state-of-the-art video SR models by a large margin.
  arXiv  Detail & Related papers  (2020-11-09T15:07:32Z)
• MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolution [63.02785017714131]
  Video super-resolution (VSR) aims to utilize multiple low-resolution frames to generate a high-resolution prediction for each frame. Inter- and intra-frames are the key sources for exploiting temporal and spatial information. We build an effective multi-correspondence aggregation network (MuCAN) for VSR.
  arXiv  Detail & Related papers  (2020-07-23T05:41:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.