Power Efficient Video Super-Resolution on Mobile NPUs with Deep
Learning, Mobile AI & AIM 2022 challenge: Report
- URL: http://arxiv.org/abs/2211.05256v1
- Date: Mon, 7 Nov 2022 22:33:19 GMT
- Title: Power Efficient Video Super-Resolution on Mobile NPUs with Deep
Learning, Mobile AI & AIM 2022 challenge: Report
- Authors: Andrey Ignatov and Radu Timofte and Cheng-Ming Chiang and Hsien-Kai
Kuo and Yu-Syuan Xu and Man-Yu Lee and Allen Lu and Chia-Ming Cheng and
Chih-Cheng Chen and Jia-Ying Yong and Hong-Han Shuai and Wen-Huang Cheng and
Zhuang Jia and Tianyu Xu and Yijian Zhang and Long Bao and Heng Sun and
Diankai Zhang and Si Gao and Shaoli Liu and Biao Wu and Xiaofeng Zhang and
Chengjian Zheng and Kaidi Lu and Ning Wang and Xiao Sun and HaoDong Wu and
Xuncheng Liu and Weizhan Zhang and Caixia Yan and Haipeng Du and Qinghua
Zheng and Qi Wang and Wangdu Chen and Ran Duan and Ran Duan and Mengdi Sun
and Dan Zhu and Guannan Chen and Hojin Cho and Steve Kim and Shijie Yue and
Chenghua Li and Zhengyang Zhuge and Wei Chen and Wenxu Wang and Yufeng Zhou
and Xiaochen Cai and Hengxing Cai and Kele Xu and Li Liu and Zehua Cheng and
Wenyi Lian and Wenjing Lian
- Abstract summary: We propose a real-time video super-resolution solution for mobile NPUs optimized for low energy consumption.
Models were evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit.
All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt / 30 FPS] power consumption.
- Score: 97.01510729548531
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Video super-resolution is one of the most popular tasks on mobile devices,
being widely used for an automatic improvement of low-bitrate and
low-resolution video streams. While numerous solutions have been proposed for
this problem, they are usually quite computationally demanding, demonstrating
low FPS rates and power efficiency on mobile devices. In this Mobile AI
challenge, we address this problem and propose the participants to design an
end-to-end real-time video super-resolution solution for mobile NPUs optimized
for low energy consumption. The participants were provided with the REDS
training dataset containing video sequences for a 4X video upscaling task. The
runtime and power efficiency of all models was evaluated on the powerful
MediaTek Dimensity 9000 platform with a dedicated AI processing unit capable of
accelerating floating-point and quantized neural networks. All proposed
solutions are fully compatible with the above NPU, demonstrating an up to 500
FPS rate and 0.2 [Watt / 30 FPS] power consumption. A detailed description of
all models developed in the challenge is provided in this paper.
Related papers
- AIM 2024 Challenge on Efficient Video Super-Resolution for AV1 Compressed Content [56.552444900457395]
Video super-resolution (VSR) is a critical task for enhancing low-bitrate and low-resolution videos, particularly in streaming applications.
In this work, we compile different methods to address these challenges, the solutions are end-to-end real-time video super-resolution frameworks.
The proposed solutions tackle video up-scaling for two applications: 540p to 4K (x4) as a general case, and 360p to 1080p (x3) more tailored towards mobile devices.
arXiv Detail & Related papers (2024-09-25T18:12:19Z) - Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs,
Mobile AI & AIM 2022 challenge: Report [144.41960648643564]
In this paper, we propose the participants to design an efficient quantized image super-resolution solution.
The solution can demonstrate a real-time performance on mobile NPUs.
The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU.
arXiv Detail & Related papers (2022-11-07T22:27:58Z) - ELSR: Extreme Low-Power Super Resolution Network For Mobile Devices [4.759823735082844]
We propose Extreme Low-Power Super Resolution (ELSR) network which only consumes a small amount of energy in mobile devices.
Pretraining and finetuning methods are applied to boost the performance of the extremely tiny model.
We achieve a competitive score of 90.9 with PSNR 27.34 dB and power 0.09 W/30FPS on the target MediaTek Dimensity 9000 plantform.
arXiv Detail & Related papers (2022-08-31T02:32:50Z) - Real-Time Video Super-Resolution on Smartphones with Deep Learning,
Mobile AI 2021 Challenge: Report [135.69469815238193]
Video super-resolution has become one of the most important mobile-related problems due to the rise of video communication and streaming services.
To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based video super-resolution solutions.
The proposed solutions are fully compatible with any mobile GPU and can upscale videos to HD resolution at up to 80 FPS while demonstrating high fidelity results.
arXiv Detail & Related papers (2021-05-17T13:40:50Z) - Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI
2021 Challenge: Report [67.86837649834636]
We introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image super-resolution solution.
The proposed solutions are fully compatible with all major mobile AI accelerators and are capable of reconstructing Full HD images under 40-60 ms.
arXiv Detail & Related papers (2021-05-17T13:34:15Z) - Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI
2021 Challenge: Report [64.09439666916465]
We introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image denoising solution.
The proposed solutions are fully compatible with any mobile GPU and are capable of processing 480p resolution images under 40-80 ms.
arXiv Detail & Related papers (2021-05-17T13:27:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.