Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution

• URL: http://arxiv.org/abs/2207.12577v1
• Date: Mon, 25 Jul 2022 23:59:19 GMT
• Title: Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution
• Authors: Yushu Wu, Yifan Gong, Pu Zhao, Yanyu Li, Zheng Zhan, Wei Niu, Hao Tang, Minghai Qin, Bin Ren, and Yanzhi Wang
• Abstract summary: We propose a compiler-aware SR neural architecture search (NAS) framework that conducts depth search and per-layer width search with adaptive SR blocks. We achieve real-time SR inference for implementing 720p resolution with competitive SR performance on GPU/DSP of mobile platforms.
• Score: 48.13296296287587
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning-based super-resolution (SR) has gained tremendous popularity in recent years because of its high image quality performance and wide application scenarios. However, prior methods typically suffer from large amounts of computations and huge power consumption, causing difficulties for real-time inference, especially on resource-limited platforms such as mobile devices. To mitigate this, we propose a compiler-aware SR neural architecture search (NAS) framework that conducts depth search and per-layer width search with adaptive SR blocks. The inference speed is directly taken into the optimization along with the SR loss to derive SR models with high image quality while satisfying the real-time inference requirement. Instead of measuring the speed on mobile devices at each iteration during the search process, a speed model incorporated with compiler optimizations is leveraged to predict the inference latency of the SR block with various width configurations for faster convergence. With the proposed framework, we achieve real-time SR inference for implementing 720p resolution with competitive SR performance (in terms of PSNR and SSIM) on GPU/DSP of mobile platforms (Samsung Galaxy S21).

Related papers

• Frequency-Assisted Mamba for Remote Sensing Image Super-Resolution [49.902047563260496]
  We develop the first attempt to integrate the Vision State Space Model (Mamba) for remote sensing image (RSI) super-resolution. To achieve better SR reconstruction, building upon Mamba, we devise a Frequency-assisted Mamba framework, dubbed FMSR. Our FMSR features a multi-level fusion architecture equipped with the Frequency Selection Module (FSM), Vision State Space Module (VSSM), and Hybrid Gate Module (HGM)
  arXiv  Detail & Related papers  (2024-05-08T11:09:24Z)
• Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search [64.80878113422824]
  We propose an automatic search framework that derives sparse super-resolution (SR) models with high image quality while satisfying the real-time inference requirement. With the proposed framework, we are the first to achieve real-time SR inference (with only tens of milliseconds per frame) for implementing 720p resolution with competitive image quality.
  arXiv  Detail & Related papers  (2021-08-18T06:47:31Z)
• SplitSR: An End-to-End Approach to Super-Resolution on Mobile Devices [7.72178128781302]
  We demonstrate state-of-the-art latency and accuracy for on-device super-resolution using a novel hybrid architecture called SplitSR. SplitSR has a hybrid design consisting of standard convolutional blocks and lightweight residual blocks. We deploy our model onto a smartphone in an app called ZoomSR to demonstrate the first-ever instance of on-device, deep learning-based SR.
  arXiv  Detail & Related papers  (2021-01-20T06:47:41Z)
• On-Device Text Image Super Resolution [0.0]
  We present a novel deep neural network that reconstructs sharper character edges and thus boosts OCR confidence. The proposed architecture not only achieves significant improvement in PSNR over bicubic upsampling but also runs with an average inference time of 11.7 ms per image. We also achieve an OCR accuracy of 75.89% on the ICDAR 2015 TextSR dataset, where ground truth has an accuracy of 78.10%.
  arXiv  Detail & Related papers  (2020-11-20T07:49:48Z)
• DynaVSR: Dynamic Adaptive Blind Video Super-Resolution [60.154204107453914]
  DynaVSR is a novel meta-learning-based framework for real-world video SR. We train a multi-frame downscaling module with various types of synthetic blur kernels, which is seamlessly combined with a video SR network for input-aware adaptation. Experimental results show that DynaVSR consistently improves the performance of the state-of-the-art video SR models by a large margin.
  arXiv  Detail & Related papers  (2020-11-09T15:07:32Z)
• Real Image Super Resolution Via Heterogeneous Model Ensemble using GP-NAS [63.48801313087118]
  We propose a new method for image superresolution using deep residual network with dense skip connections. The proposed method won the first place in all three tracks of the AIM 2020 Real Image Super-Resolution Challenge.
  arXiv  Detail & Related papers  (2020-09-02T22:33:23Z)
• Journey Towards Tiny Perceptual Super-Resolution [23.30464519074935]
  We propose a neural architecture search (NAS) approach that integrates NAS and generative adversarial networks (GANs) with recent advances in perceptual SR. Our tiny perceptual SR (TPSR) models outperform SRGAN and EnhanceNet on both full-reference perceptual metric (LPIPS) and distortion metric (PSNR)
  arXiv  Detail & Related papers  (2020-07-08T18:24:40Z)
• Lightweight image super-resolution with enhanced CNN [82.36883027158308]
  Deep convolutional neural networks (CNNs) with strong expressive ability have achieved impressive performances on single image super-resolution (SISR) We propose a lightweight enhanced SR CNN (LESRCNN) with three successive sub-blocks, an information extraction and enhancement block (IEEB), a reconstruction block (RB) and an information refinement block (IRB) IEEB extracts hierarchical low-resolution (LR) features and aggregates the obtained features step-by-step to increase the memory ability of the shallow layers on deep layers for SISR. RB converts low-frequency features into high-frequency features by fusing global
  arXiv  Detail & Related papers  (2020-07-08T18:03:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.