Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search

• URL: http://arxiv.org/abs/2108.08910v1
• Date: Wed, 18 Aug 2021 06:47:31 GMT
• Title: Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search
• Authors: Zheng Zhan, Yifan Gong, Pu Zhao, Geng Yuan, Wei Niu, Yushu Wu, Tianyun Zhang, Malith Jayaweera, David Kaeli, Bin Ren, Xue Lin, Yanzhi Wang
• Abstract summary: We propose an automatic search framework that derives sparse super-resolution (SR) models with high image quality while satisfying the real-time inference requirement. With the proposed framework, we are the first to achieve real-time SR inference (with only tens of milliseconds per frame) for implementing 720p resolution with competitive image quality.
• Score: 64.80878113422824
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Though recent years have witnessed remarkable progress in single image super-resolution (SISR) tasks with the prosperous development of deep neural networks (DNNs), the deep learning methods are confronted with the computation and memory consumption issues in practice, especially for resource-limited platforms such as mobile devices. To overcome the challenge and facilitate the real-time deployment of SISR tasks on mobile, we combine neural architecture search with pruning search and propose an automatic search framework that derives sparse super-resolution (SR) models with high image quality while satisfying the real-time inference requirement. To decrease the search cost, we leverage the weight sharing strategy by introducing a supernet and decouple the search problem into three stages, including supernet construction, compiler-aware architecture and pruning search, and compiler-aware pruning ratio search. With the proposed framework, we are the first to achieve real-time SR inference (with only tens of milliseconds per frame) for implementing 720p resolution with competitive image quality (in terms of PSNR and SSIM) on mobile platforms (Samsung Galaxy S20).

Related papers

• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Real-Time Image Segmentation via Hybrid Convolutional-Transformer Architecture Search [49.81353382211113]
  We address the challenge of integrating multi-head self-attention into high resolution representation CNNs efficiently. We develop a multi-target multi-branch supernet method, which fully utilizes the advantages of high-resolution features. We present a series of model via Hybrid Convolutional-Transformer Architecture Search (HyCTAS) method that searched for the best hybrid combination of light-weight convolution layers and memory-efficient self-attention layers.
  arXiv  Detail & Related papers  (2024-03-15T15:47:54Z)
• SYENet: A Simple Yet Effective Network for Multiple Low-Level Vision Tasks with Real-time Performance on Mobile Device [6.548475407783714]
  We propose a novel network, SYENet, to handle multiple low-level vision tasks on mobile devices in a real-time manner. The proposed method proves its superior performance with the best PSNR as compared with other networks in real-time applications.
  arXiv  Detail & Related papers  (2023-08-16T04:03:59Z)
• Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution [48.13296296287587]
  We propose a compiler-aware SR neural architecture search (NAS) framework that conducts depth search and per-layer width search with adaptive SR blocks. We achieve real-time SR inference for implementing 720p resolution with competitive SR performance on GPU/DSP of mobile platforms.
  arXiv  Detail & Related papers  (2022-07-25T23:59:19Z)
• Extremely Lightweight Quantization Robust Real-Time Single-Image Super Resolution for Mobile Devices [0.0]
  Single-Image Super Resolution (SISR) is a classical computer vision problem and it has been studied for over decades. Recent work on SISR focuses solutions with deep learning methodologies and achieves state-of-the-art results. We propose a hardware (Synaptics Dolphin NPU) aware, extremely lightweight quantization robust real-time super resolution network (XLSR)
  arXiv  Detail & Related papers  (2021-05-21T11:29:48Z)
• Lightweight Image Super-Resolution with Hierarchical and Differentiable Neural Architecture Search [38.83764580480486]
  Single Image Super-Resolution (SISR) tasks have achieved significant performance with deep neural networks. We propose a novel differentiable Neural Architecture Search (NAS) approach on both the cell-level and network-level to search for lightweight SISR models.
  arXiv  Detail & Related papers  (2021-05-09T13:30:16Z)
• Learning Frequency-aware Dynamic Network for Efficient Super-Resolution [56.98668484450857]
  This paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain. In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden. Experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures.
  arXiv  Detail & Related papers  (2021-03-15T12:54:26Z)
• MS-RANAS: Multi-Scale Resource-Aware Neural Architecture Search [94.80212602202518]
  We propose Multi-Scale Resource-Aware Neural Architecture Search (MS-RANAS) We employ a one-shot architecture search approach in order to obtain a reduced search cost. We achieve state-of-the-art results in terms of accuracy-speed trade-off.
  arXiv  Detail & Related papers  (2020-09-29T11:56:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.