Self-Supervised Adaptation for Video Super-Resolution
- URL: http://arxiv.org/abs/2103.10081v1
- Date: Thu, 18 Mar 2021 08:30:24 GMT
- Title: Self-Supervised Adaptation for Video Super-Resolution
- Authors: Jinsu Yoo and Tae Hyun Kim
- Abstract summary: Single-image super-resolution (SISR) networks can adapt their network parameters to specific input images.
We present a new learning algorithm that allows conventional video super-resolution (VSR) networks to adapt their parameters to test video frames.
- Score: 7.26562478548988
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent single-image super-resolution (SISR) networks, which can adapt their
network parameters to specific input images, have shown promising results by
exploiting the information available within the input data as well as large
external datasets. However, the extension of these self-supervised SISR
approaches to video handling has yet to be studied. Thus, we present a new
learning algorithm that allows conventional video super-resolution (VSR)
networks to adapt their parameters to test video frames without using the
ground-truth datasets. By utilizing many self-similar patches across space and
time, we improve the performance of fully pre-trained VSR networks and produce
temporally consistent video frames. Moreover, we present a test-time knowledge
distillation technique that accelerates the adaptation speed with less hardware
resources. In our experiments, we demonstrate that our novel learning algorithm
can fine-tune state-of-the-art VSR networks and substantially elevate
performance on numerous benchmark datasets.
Related papers
- Cascaded Temporal Updating Network for Efficient Video Super-Resolution [47.63267159007611]
Key components in recurrent-based VSR networks significantly impact model efficiency.
We propose a cascaded temporal updating network (CTUN) for efficient VSR.
CTUN achieves a favorable trade-off between efficiency and performance compared to existing methods.
arXiv Detail & Related papers (2024-08-26T12:59:32Z) - Neighbourhood Representative Sampling for Efficient End-to-end Video
Quality Assessment [60.57703721744873]
The increased resolution of real-world videos presents a dilemma between efficiency and accuracy for deep Video Quality Assessment (VQA)
In this work, we propose a unified scheme, spatial-temporal grid mini-cube sampling (St-GMS) to get a novel type of sample, named fragments.
With fragments and FANet, the proposed efficient end-to-end FAST-VQA and FasterVQA achieve significantly better performance than existing approaches on all VQA benchmarks.
arXiv Detail & Related papers (2022-10-11T11:38:07Z) - Sliding Window Recurrent Network for Efficient Video Super-Resolution [0.0]
Video super-resolution (VSR) is the task of restoring high-resolution frames from a sequence of low-resolution inputs.
We propose a textitSliding Window based Recurrent Network (SWRN) which can be real-time inference while still achieving superior performance.
Our experiment on REDS dataset shows that the proposed method can be well adapted to mobile devices and produce visually pleasant results.
arXiv Detail & Related papers (2022-08-24T15:23:44Z) - Boosting Video Super Resolution with Patch-Based Temporal Redundancy
Optimization [46.833568886576074]
We discuss the influence of the temporal redundancy in the patches with stationary objects and background.
We develop two simple yet effective plug and play methods to improve the performance of existing local and non-local propagation-based VSR algorithms.
arXiv Detail & Related papers (2022-07-18T15:11:18Z) - Ada-VSR: Adaptive Video Super-Resolution with Meta-Learning [56.676110454594344]
VideoSuperResolution (Ada-SR) uses external as well as internal, information through meta-transfer learning and internal learning, respectively.
Model trained using our approach can quickly adapt to a specific video condition with only a few gradient updates, which reduces the inference time significantly.
arXiv Detail & Related papers (2021-08-05T19:59:26Z) - Real-Time Super-Resolution System of 4K-Video Based on Deep Learning [6.182364004551161]
Video-resolution (VSR) technology excels in low-quality video computation, avoiding unpleasant blur effect caused by occupation-based algorithms.
This paper explores the possibility of real-time VS system and designs an efficient generic VSR network, termed EGVSR.
Compared with TecoGAN, the most advanced VSR network at present, we achieve 84% reduction of density and 7.92x performance speedups.
arXiv Detail & Related papers (2021-07-12T10:35:05Z) - DynaVSR: Dynamic Adaptive Blind Video Super-Resolution [60.154204107453914]
DynaVSR is a novel meta-learning-based framework for real-world video SR.
We train a multi-frame downscaling module with various types of synthetic blur kernels, which is seamlessly combined with a video SR network for input-aware adaptation.
Experimental results show that DynaVSR consistently improves the performance of the state-of-the-art video SR models by a large margin.
arXiv Detail & Related papers (2020-11-09T15:07:32Z) - Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video
Super-Resolution [95.26202278535543]
A simple solution is to split it into two sub-tasks: video frame (VFI) and video super-resolution (VSR)
temporalsynthesis and spatial super-resolution are intra-related in this task.
We propose a one-stage space-time video super-resolution framework, which directly synthesizes an HR slow-motion video from an LFR, LR video.
arXiv Detail & Related papers (2020-02-26T16:59:48Z) - Fast Adaptation to Super-Resolution Networks via Meta-Learning [24.637337634643885]
In this work, we observe the opportunity for further improvement of the performance of SISR without changing the architecture of conventional SR networks.
In the training stage, we train the network via meta-learning; thus, the network can quickly adapt to any input image at test time.
We demonstrate that the proposed model-agnostic approach consistently improves the performance of conventional SR networks on various benchmark SR datasets.
arXiv Detail & Related papers (2020-01-09T09:59:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.