Ada-VSR: Adaptive Video Super-Resolution with Meta-Learning

• URL: http://arxiv.org/abs/2108.02832v1
• Date: Thu, 5 Aug 2021 19:59:26 GMT
• Title: Ada-VSR: Adaptive Video Super-Resolution with Meta-Learning
• Authors: Akash Gupta, Padmaja Jonnalagedda, Bir Bhanu, Amit K. Roy-Chowdhury
• Abstract summary: VideoSuperResolution (Ada-SR) uses external as well as internal, information through meta-transfer learning and internal learning, respectively. Model trained using our approach can quickly adapt to a specific video condition with only a few gradient updates, which reduces the inference time significantly.
• Score: 56.676110454594344
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Most of the existing works in supervised spatio-temporal video super-resolution (STVSR) heavily rely on a large-scale external dataset consisting of paired low-resolution low-frame rate (LR-LFR)and high-resolution high-frame-rate (HR-HFR) videos. Despite their remarkable performance, these methods make a prior assumption that the low-resolution video is obtained by down-scaling the high-resolution video using a known degradation kernel, which does not hold in practical settings. Another problem with these methods is that they cannot exploit instance-specific internal information of video at testing time. Recently, deep internal learning approaches have gained attention due to their ability to utilize the instance-specific statistics of a video. However, these methods have a large inference time as they require thousands of gradient updates to learn the intrinsic structure of the data. In this work, we presentAdaptiveVideoSuper-Resolution (Ada-VSR) which leverages external, as well as internal, information through meta-transfer learning and internal learning, respectively. Specifically, meta-learning is employed to obtain adaptive parameters, using a large-scale external dataset, that can adapt quickly to the novel condition (degradation model) of the given test video during the internal learning task, thereby exploiting external and internal information of a video for super-resolution. The model trained using our approach can quickly adapt to a specific video condition with only a few gradient updates, which reduces the inference time significantly. Extensive experiments on standard datasets demonstrate that our method performs favorably against various state-of-the-art approaches.

Related papers

• Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Prompts [57.01985221057047]
  This paper introduces a novel method that learnstemporal prompt embeddings for weakly supervised video anomaly detection and localization (WSVADL) based on pre-trained vision-language models (VLMs) Our method achieves state-of-theart performance on three public benchmarks for the WSVADL task.
  arXiv  Detail & Related papers  (2024-08-12T03:31:29Z)
• Video Dynamics Prior: An Internal Learning Approach for Robust Video Enhancements [83.5820690348833]
  We present a framework for low-level vision tasks that does not require any external training data corpus. Our approach learns neural modules by optimizing over a corrupted sequence, leveraging the weights of the coherence-temporal test and statistics internal statistics.
  arXiv  Detail & Related papers  (2023-12-13T01:57:11Z)
• Enabling Weakly-Supervised Temporal Action Localization from On-Device Learning of the Video Stream [5.215681853828831]
  We propose an efficient video learning approach to learn from a long, untrimmed streaming video. To the best of our knowledge, we are the first attempt to directly learn from the on-device, long video stream.
  arXiv  Detail & Related papers  (2022-08-25T13:41:03Z)
• Combining Contrastive and Supervised Learning for Video Super-Resolution Detection [0.0]
  We propose a new upscaled-resolution-detection method based on learning of visual representations using contrastive and cross-entropy losses. Our method effectively detects upscaling even in compressed videos and outperforms the state-of-the-art alternatives.
  arXiv  Detail & Related papers  (2022-05-20T18:58:13Z)
• Self-Supervised Deep Blind Video Super-Resolution [46.410705294831374]
  We propose a self-supervised learning method to solve the blind video SR problem. We generate auxiliary paired data from original LR videos according to the image formation of video SR. Experiments show that our method performs favorably against state-of-the-art ones on benchmarks and real-world videos.
  arXiv  Detail & Related papers  (2022-01-19T05:18:44Z)
• Video Salient Object Detection via Contrastive Features and Attention Modules [106.33219760012048]
  We propose a network with attention modules to learn contrastive features for video salient object detection. A co-attention formulation is utilized to combine the low-level and high-level features. We show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-03T17:40:32Z)
• Self-Supervised Adaptation for Video Super-Resolution [7.26562478548988]
  Single-image super-resolution (SISR) networks can adapt their network parameters to specific input images. We present a new learning algorithm that allows conventional video super-resolution (VSR) networks to adapt their parameters to test video frames.
  arXiv  Detail & Related papers  (2021-03-18T08:30:24Z)
• Incremental Learning from Low-labelled Stream Data in Open-Set Video Face Recognition [0.0]
  We propose a novel incremental learning approach which combines a deep features encoder with an Open-Set Dynamic Ensembles of SVM. Our method can use unsupervised operational data to enhance recognition. Results show a benefit of up to 15% F1-score increase respect to non-adaptive state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-12-17T13:28:13Z)
• Temporal Context Aggregation for Video Retrieval with Contrastive Learning [81.12514007044456]
  We propose TCA, a video representation learning framework that incorporates long-range temporal information between frame-level features. The proposed method shows a significant performance advantage (17% mAP on FIVR-200K) over state-of-the-art methods with video-level features.
  arXiv  Detail & Related papers  (2020-08-04T05:24:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.