Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report

• URL: http://arxiv.org/abs/2211.03885v1
• Date: Mon, 7 Nov 2022 22:13:10 GMT
• Title: Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report
• Authors: Andrey Ignatov and Radu Timofte and Shuai Liu and Chaoyu Feng and Furui Bai and Xiaotao Wang and Lei Lei and Ziyao Yi and Yan Xiang and Zibin Liu and Shaoqing Li and Keming Shi and Dehui Kong and Ke Xu and Minsu Kwon and Yaqi Wu and Jiesi Zheng and Zhihao Fan and Xun Wu and Feng Zhang and Albert No and Minhyeok Cho and Zewen Chen and Xiaze Zhang and Ran Li and Juan Wang and Zhiming Wang and Marcos V. Conde and Ui-Jin Choi and Georgy Perevozchikov and Egor Ershov and Zheng Hui and Mengchuan Dong and Xin Lou and Wei Zhou and Cong Pang and Haina Qin and Mingxuan Cai
• Abstract summary: This challenge aims to develop an efficient end-to-end AI-based image signal processing pipeline. The models were evaluated on the Snapdragon's 8 Gen 1 GPU that provides excellent acceleration results for the majority of common deep learning ops. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos in less than 20-50 milliseconds while achieving high fidelity results.
• Score: 59.831324427712815
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale Fujifilm UltraISP dataset consisting of thousands of paired photos captured with a normal mobile camera sensor and a professional 102MP medium-format FujiFilm GFX100 camera. The runtime of the resulting models was evaluated on the Snapdragon's 8 Gen 1 GPU that provides excellent acceleration results for the majority of common deep learning ops. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos in less than 20-50 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.

Related papers

• MicroISP: Processing 32MP Photos on Mobile Devices with Deep Learning [114.66037224769005]
  We present a novel MicroISP model designed specifically for edge devices. The proposed solution is capable of processing up to 32MP photos on recent smartphones using the standard mobile ML libraries. The architecture of the model is flexible, allowing to adjust its complexity to devices of different computational power.
  arXiv  Detail & Related papers  (2022-11-08T17:40:50Z)
• Realistic Bokeh Effect Rendering on Mobile GPUs, Mobile AI & AIM 2022 challenge: Report [75.79829464552311]
  This challenge was to develop an efficient end-to-end AI-based rendering approach that can run on modern smartphone models. The resulting model was evaluated on the Kirin 9000's Mali GPU that provides excellent acceleration results for the majority of common deep learning ops.
  arXiv  Detail & Related papers  (2022-11-07T22:42:02Z)
• Fast and Accurate Quantized Camera Scene Detection on Smartphones, Mobile AI 2021 Challenge: Report [65.91472671013302]
  We introduce the first Mobile AI challenge, where the target is to develop quantized deep learning-based camera scene classification solutions. The proposed solutions are fully compatible with all major mobile AI accelerators and can demonstrate more than 100-200 FPS on the majority of recent smartphone platforms.
  arXiv  Detail & Related papers  (2021-05-17T13:55:38Z)
• Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report [64.09439666916465]
  We introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image denoising solution. The proposed solutions are fully compatible with any mobile GPU and are capable of processing 480p resolution images under 40-80 ms.
  arXiv  Detail & Related papers  (2021-05-17T13:27:56Z)
• Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report [49.643297263102845]
  This challenge aims to develop an end-to-end deep learning-based image signal processing pipeline. The proposed solutions are capable of processing Full HD photos under 60-100 milliseconds while achieving high fidelity results.
  arXiv  Detail & Related papers  (2021-05-17T13:20:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.