AIM 2024 Challenge on Efficient Video Super-Resolution for AV1 Compressed Content
- URL: http://arxiv.org/abs/2409.17256v1
- Date: Wed, 25 Sep 2024 18:12:19 GMT
- Title: AIM 2024 Challenge on Efficient Video Super-Resolution for AV1 Compressed Content
- Authors: Marcos V Conde, Zhijun Lei, Wen Li, Christos Bampis, Ioannis Katsavounidis, Radu Timofte,
- Abstract summary: Video super-resolution (VSR) is a critical task for enhancing low-bitrate and low-resolution videos, particularly in streaming applications.
In this work, we compile different methods to address these challenges, the solutions are end-to-end real-time video super-resolution frameworks.
The proposed solutions tackle video up-scaling for two applications: 540p to 4K (x4) as a general case, and 360p to 1080p (x3) more tailored towards mobile devices.
- Score: 56.552444900457395
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Video super-resolution (VSR) is a critical task for enhancing low-bitrate and low-resolution videos, particularly in streaming applications. While numerous solutions have been developed, they often suffer from high computational demands, resulting in low frame rates (FPS) and poor power efficiency, especially on mobile platforms. In this work, we compile different methods to address these challenges, the solutions are end-to-end real-time video super-resolution frameworks optimized for both high performance and low runtime. We also introduce a new test set of high-quality 4K videos to further validate the approaches. The proposed solutions tackle video up-scaling for two applications: 540p to 4K (x4) as a general case, and 360p to 1080p (x3) more tailored towards mobile devices. In both tracks, the solutions have a reduced number of parameters and operations (MACs), allow high FPS, and improve VMAF and PSNR over interpolation baselines. This report gauges some of the most efficient video super-resolution methods to date.
Related papers
- Adaptive Caching for Faster Video Generation with Diffusion Transformers [52.73348147077075]
Diffusion Transformers (DiTs) rely on larger models and heavier attention mechanisms, resulting in slower inference speeds.
We introduce a training-free method to accelerate video DiTs, termed Adaptive Caching (AdaCache)
We also introduce a Motion Regularization (MoReg) scheme to utilize video information within AdaCache, controlling the compute allocation based on motion content.
arXiv Detail & Related papers (2024-11-04T18:59:44Z) - AsConvSR: Fast and Lightweight Super-Resolution Network with Assembled
Convolutions [32.85522513271578]
We propose a fast and lightweight super-resolution network to achieve real-time performance.
By analyzing the applications of divide-and-conquer in super-resolution, we propose assembled convolutions which can adapt convolution kernels according to the input features.
Our method also wins the first place in NTIRE 2023 Real-Time Super-Resolution - Track 1.
arXiv Detail & Related papers (2023-05-05T09:33:34Z) - Towards High-Quality and Efficient Video Super-Resolution via
Spatial-Temporal Data Overfitting [27.302681897961588]
Deep convolutional neural networks (DNNs) are widely used in various fields of computer vision.
We propose a novel method for high-quality and efficient video resolution upscaling tasks.
We deploy our models on an off-the-shelf mobile phone, and experimental results show that our method achieves real-time video super-resolution with high video quality.
arXiv Detail & Related papers (2023-03-15T02:40:02Z) - QuickSRNet: Plain Single-Image Super-Resolution Architecture for Faster
Inference on Mobile Platforms [36.962828335199596]
QuickSRNet is an efficient super-resolution architecture for real-time applications on mobile platforms.
Our proposed architecture produces 1080p outputs via 2x upscaling in 2.2 ms on a modern smartphone.
arXiv Detail & Related papers (2023-03-08T02:19:54Z) - Power Efficient Video Super-Resolution on Mobile NPUs with Deep
Learning, Mobile AI & AIM 2022 challenge: Report [97.01510729548531]
We propose a real-time video super-resolution solution for mobile NPUs optimized for low energy consumption.
Models were evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit.
All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt / 30 FPS] power consumption.
arXiv Detail & Related papers (2022-11-07T22:33:19Z) - Memory-Augmented Non-Local Attention for Video Super-Resolution [61.55700315062226]
We propose a novel video super-resolution method that aims at generating high-fidelity high-resolution (HR) videos from low-resolution (LR) ones.
Previous methods predominantly leverage temporal neighbor frames to assist the super-resolution of the current frame.
In contrast, we devise a cross-frame non-local attention mechanism that allows video super-resolution without frame alignment.
arXiv Detail & Related papers (2021-08-25T05:12:14Z) - Real-Time Video Super-Resolution on Smartphones with Deep Learning,
Mobile AI 2021 Challenge: Report [135.69469815238193]
Video super-resolution has become one of the most important mobile-related problems due to the rise of video communication and streaming services.
To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based video super-resolution solutions.
The proposed solutions are fully compatible with any mobile GPU and can upscale videos to HD resolution at up to 80 FPS while demonstrating high fidelity results.
arXiv Detail & Related papers (2021-05-17T13:40:50Z) - COMISR: Compression-Informed Video Super-Resolution [76.94152284740858]
Most videos on the web or mobile devices are compressed, and the compression can be severe when the bandwidth is limited.
We propose a new compression-informed video super-resolution model to restore high-resolution content without introducing artifacts caused by compression.
arXiv Detail & Related papers (2021-05-04T01:24:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.