Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report

• URL: http://arxiv.org/abs/2105.08826v1
• Date: Mon, 17 May 2021 13:40:50 GMT
• Title: Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report
• Authors: Andrey Ignatov, Andres Romero, Heewon Kim, Radu Timofte, Chiu Man Ho, Zibo Meng, Kyoung Mu Lee, Yuxiang Chen, Yutong Wang, Zeyu Long, Chenhao Wang, Yifei Chen, Boshen Xu, Shuhang Gu, Lixin Duan, Wen Li, Wang Bofei, Zhang Diankai, Zheng Chengjian, Liu Shaoli, Gao Si, Zhang Xiaofeng, Lu Kaidi, Xu Tianyu, Zheng Hui, Xinbo Gao, Xiumei Wang, Jiaming Guo, Xueyi Zhou, Hao Jia, Youliang Yan
• Abstract summary: Video super-resolution has become one of the most important mobile-related problems due to the rise of video communication and streaming services. To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based video super-resolution solutions. The proposed solutions are fully compatible with any mobile GPU and can upscale videos to HD resolution at up to 80 FPS while demonstrating high fidelity results.
• Score: 135.69469815238193
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video super-resolution has recently become one of the most important mobile-related problems due to the rise of video communication and streaming services. While many solutions have been proposed for this task, the majority of them are too computationally expensive to run on portable devices with limited hardware resources. To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based video super-resolution solutions that can achieve a real-time performance on mobile GPUs. The participants were provided with the REDS dataset and trained their models to do an efficient 4X video upscaling. The runtime of all models was evaluated on the OPPO Find X2 smartphone with the Snapdragon 865 SoC capable of accelerating floating-point networks on its Adreno GPU. The proposed solutions are fully compatible with any mobile GPU and can upscale videos to HD resolution at up to 80 FPS while demonstrating high fidelity results. A detailed description of all models developed in the challenge is provided in this paper.

Related papers

• Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 challenge: Report [97.01510729548531]
  We propose a real-time video super-resolution solution for mobile NPUs optimized for low energy consumption. Models were evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt / 30 FPS] power consumption.
  arXiv  Detail & Related papers  (2022-11-07T22:33:19Z)
• Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report [144.41960648643564]
  In this paper, we propose the participants to design an efficient quantized image super-resolution solution. The solution can demonstrate a real-time performance on mobile NPUs. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU.
  arXiv  Detail & Related papers  (2022-11-07T22:27:58Z)
• Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report [105.32612705754605]
  We introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based depth estimation solution. The proposed solutions can generate VGA resolution depth maps at up to 10 FPS on the Raspberry Pi 4 while achieving high fidelity results.
  arXiv  Detail & Related papers  (2021-05-17T13:49:57Z)
• Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report [67.86837649834636]
  We introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image super-resolution solution. The proposed solutions are fully compatible with all major mobile AI accelerators and are capable of reconstructing Full HD images under 40-60 ms.
  arXiv  Detail & Related papers  (2021-05-17T13:34:15Z)
• Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report [64.09439666916465]
  We introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image denoising solution. The proposed solutions are fully compatible with any mobile GPU and are capable of processing 480p resolution images under 40-80 ms.
  arXiv  Detail & Related papers  (2021-05-17T13:27:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.