AnimeSR: Learning Real-World Super-Resolution Models for Animation Videos

• URL: http://arxiv.org/abs/2206.07038v1
• Date: Tue, 14 Jun 2022 17:57:11 GMT
• Title: AnimeSR: Learning Real-World Super-Resolution Models for Animation Videos
• Authors: Yanze Wu, Xintao Wang, Gen Li, Ying Shan
• Abstract summary: This paper studies the problem of real-world video super-resolution (VSR) for animation videos, and reveals three key improvements for practical animation VSR. We propose to learn such basic operators from real low-quality animation videos, and incorporate the learned ones into the degradation generation pipeline. Our method, AnimeSR, is capable of restoring real-world low-quality animation videos effectively and efficiently, achieving superior performance to previous state-of-the-art methods.
• Score: 23.71771590274543
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper studies the problem of real-world video super-resolution (VSR) for animation videos, and reveals three key improvements for practical animation VSR. First, recent real-world super-resolution approaches typically rely on degradation simulation using basic operators without any learning capability, such as blur, noise, and compression. In this work, we propose to learn such basic operators from real low-quality animation videos, and incorporate the learned ones into the degradation generation pipeline. Such neural-network-based basic operators could help to better capture the distribution of real degradations. Second, a large-scale high-quality animation video dataset, AVC, is built to facilitate comprehensive training and evaluations for animation VSR. Third, we further investigate an efficient multi-scale network structure. It takes advantage of the efficiency of unidirectional recurrent networks and the effectiveness of sliding-window-based methods. Thanks to the above delicate designs, our method, AnimeSR, is capable of restoring real-world low-quality animation videos effectively and efficiently, achieving superior performance to previous state-of-the-art methods.

Related papers

• Free Video-LLM: Prompt-guided Visual Perception for Efficient Training-free Video LLMs [56.040198387038025]
  We present a novel prompt-guided visual perception framework (abbreviated as Free Video-LLM) for efficient inference of training-free video LLMs. Our method effectively reduces the number of visual tokens while maintaining high performance across multiple video question-answering benchmarks.
  arXiv  Detail & Related papers  (2024-10-14T12:35:12Z)
• An Animation-based Augmentation Approach for Action Recognition from Discontinuous Video [11.293897932762809]
  Action recognition, an essential component of computer vision, plays a pivotal role in multiple applications. CNNs suffer performance declines when trained with discontinuous video frames, which is a frequent scenario in real-world settings. To overcome this issue, we introduce the 4A pipeline, which employs a series of sophisticated techniques.
  arXiv  Detail & Related papers  (2024-04-10T04:59:51Z)
• VJT: A Video Transformer on Joint Tasks of Deblurring, Low-light Enhancement and Denoising [45.349350685858276]
  Video restoration task aims to recover high-quality videos from low-quality observations. Video often faces different types of degradation, such as blur, low light, and noise. We propose an efficient end-to-end video transformer approach for the joint task of video deblurring, low-light enhancement, and denoising.
  arXiv  Detail & Related papers  (2024-01-26T10:27:56Z)
• Sample Less, Learn More: Efficient Action Recognition via Frame Feature Restoration [59.6021678234829]
  We propose a novel method to restore the intermediate features for two sparsely sampled and adjacent video frames. With the integration of our method, the efficiency of three commonly used baselines has been improved by over 50%, with a mere 0.5% reduction in recognition accuracy.
  arXiv  Detail & Related papers  (2023-07-27T13:52:42Z)
• Learning Data-Driven Vector-Quantized Degradation Model for Animation Video Super-Resolution [59.71387128485845]
  We explore the characteristics of animation videos and leverage the rich priors in real-world animation data for a more practical animation VSR model. We propose a multi-scale Vector-Quantized Degradation model for animation video Super-Resolution (VQD-SR) to decompose the local details from global structures. A rich-content Real Animation Low-quality (RAL) video dataset is collected for extracting the priors.
  arXiv  Detail & Related papers  (2023-03-17T08:11:14Z)
• Investigating Tradeoffs in Real-World Video Super-Resolution [90.81396836308085]
  Real-world video super-resolution (VSR) models are often trained with diverse degradations to improve generalizability. To alleviate the first tradeoff, we propose a degradation scheme that reduces up to 40% of training time without sacrificing performance. To facilitate fair comparisons, we propose the new VideoLQ dataset, which contains a large variety of real-world low-quality video sequences.
  arXiv  Detail & Related papers  (2021-11-24T18:58:21Z)
• Real-Time Video Super-Resolution by Joint Local Inference and Global Parameter Estimation [0.0]
  We present a novel approach to synthesizing training data by simulating two digital-camera image-capture processes at different scales. Our method produces image-pairs in which both images have properties of natural images. We present an efficient CNN architecture, which enables real-time application of video SR on low-power edge-devices.
  arXiv  Detail & Related papers  (2021-05-06T16:35:09Z)
• 3D Human Pose, Shape and Texture from Low-Resolution Images and Videos [107.36352212367179]
  We propose RSC-Net, which consists of a Resolution-aware network, a Self-supervision loss, and a Contrastive learning scheme. The proposed method is able to learn 3D body pose and shape across different resolutions with one single model. We extend the RSC-Net to handle low-resolution videos and apply it to reconstruct textured 3D pedestrians from low-resolution input.
  arXiv  Detail & Related papers  (2021-03-11T06:52:12Z)
• Deep Unrolled Network for Video Super-Resolution [0.45880283710344055]
  Video super-resolution (VSR) aims to reconstruct a sequence of high-resolution (HR) images from their corresponding low-resolution (LR) versions. Traditionally, solving a VSR problem has been based on iterative algorithms that exploit prior knowledge on image formation and assumptions on the motion. Deep learning (DL) algorithms can efficiently learn spatial patterns from large collections of images. We propose a new VSR neural network based on unrolled optimization techniques and discuss its performance.
  arXiv  Detail & Related papers  (2021-02-23T14:35:09Z)
• Video Face Super-Resolution with Motion-Adaptive Feedback Cell [90.73821618795512]
  Video super-resolution (VSR) methods have recently achieved a remarkable success due to the development of deep convolutional neural networks (CNN) In this paper, we propose a Motion-Adaptive Feedback Cell (MAFC), a simple but effective block, which can efficiently capture the motion compensation and feed it back to the network in an adaptive way.
  arXiv  Detail & Related papers  (2020-02-15T13:14:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.