Collapsible Linear Blocks for Super-Efficient Super Resolution

• URL: http://arxiv.org/abs/2103.09404v1
• Date: Wed, 17 Mar 2021 02:16:31 GMT
• Title: Collapsible Linear Blocks for Super-Efficient Super Resolution
• Authors: Kartikeya Bhardwaj, Milos Milosavljevic, Alex Chalfin, Naveen Suda, Liam O'Neil, Dibakar Gope, Lingchuan Meng, Ramon Matas, Danny Loh
• Abstract summary: Single Image Super Resolution (SISR) has become an important computer vision problem. We propose SESR, a new class of Super-Efficient Super Resolution networks. Detailed experiments across six benchmark datasets demonstrate that SESR achieves similar or better image quality.
• Score: 3.5554418329811557
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the advent of smart devices that support 4K and 8K resolution, Single Image Super Resolution (SISR) has become an important computer vision problem. However, most super resolution deep networks are computationally very expensive. In this paper, we propose SESR, a new class of Super-Efficient Super Resolution networks that significantly improve image quality and reduce computational complexity. Detailed experiments across six benchmark datasets demonstrate that SESR achieves similar or better image quality than state-of-the-art models while requiring 2x to 330x fewer Multiply-Accumulate (MAC) operations. As a result, SESR can be used on constrained hardware to perform x2 (1080p to 4K) and x4 SISR (1080p to 8K). Towards this, we simulate hardware performance numbers for a commercial mobile Neural Processing Unit (NPU) for 1080p to 4K (x2) and 1080p to 8K (x4) SISR. Our results highlight the challenges faced by super resolution on AI accelerators and demonstrate that SESR is significantly faster than existing models. Overall, SESR establishes a new Pareto frontier on the quality (PSNR)-computation relationship for the super resolution task.

Related papers

• RTSR: A Real-Time Super-Resolution Model for AV1 Compressed Content [10.569678424799616]
  Super-resolution (SR) is a key technique for improving the visual quality of video content. To support real-time playback, it is important to implement fast SR models while preserving reconstruction quality. This paper proposes a low-complexity SR method, RTSR, designed to enhance the visual quality of compressed video content.
  arXiv  Detail & Related papers  (2024-11-20T14:36:06Z)
• InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD [129.9919468062788]
  InternLM-XComposer2-4KHD is a groundbreaking exploration into elevating LVLM resolution capabilities up to 4K HD (3840 x 1600) and beyond. This research advances the patch division paradigm by introducing a novel extension: dynamic resolution with automatic patch configuration. Our research demonstrates that scaling training resolution up to 4K HD leads to consistent performance enhancements without hitting the ceiling of potential improvements.
  arXiv  Detail & Related papers  (2024-04-09T17:59:32Z)
• HIRI-ViT: Scaling Vision Transformer with High Resolution Inputs [102.4965532024391]
  hybrid deep models of Vision Transformer (ViT) and Convolution Neural Network (CNN) have emerged as a powerful class of backbones for vision tasks. We present a new hybrid backbone with HIgh-Resolution Inputs (namely HIRI-ViT), that upgrades prevalent four-stage ViT to five-stage ViT tailored for high-resolution inputs. HiRI-ViT achieves to-date the best published Top-1 accuracy of 84.3% on ImageNet with 448$times$448 inputs, which absolutely improves 83.4% of iFormer-S by 0.9% with 224$times
  arXiv  Detail & Related papers  (2024-03-18T17:34:29Z)
• PTSR: Patch Translator for Image Super-Resolution [16.243363392717434]
  We propose a patch translator for image super-resolution (PTSR) to address this problem. The proposed PTSR is a transformer-based GAN network with no convolution operation. We introduce a novel patch translator module for regenerating the improved patches utilising multi-head attention.
  arXiv  Detail & Related papers  (2023-10-20T01:45:00Z)
• AsConvSR: Fast and Lightweight Super-Resolution Network with Assembled Convolutions [32.85522513271578]
  We propose a fast and lightweight super-resolution network to achieve real-time performance. By analyzing the applications of divide-and-conquer in super-resolution, we propose assembled convolutions which can adapt convolution kernels according to the input features. Our method also wins the first place in NTIRE 2023 Real-Time Super-Resolution - Track 1.
  arXiv  Detail & Related papers  (2023-05-05T09:33:34Z)
• Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report [144.41960648643564]
  In this paper, we propose the participants to design an efficient quantized image super-resolution solution. The solution can demonstrate a real-time performance on mobile NPUs. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU.
  arXiv  Detail & Related papers  (2022-11-07T22:27:58Z)
• Hybrid Pixel-Unshuffled Network for Lightweight Image Super-Resolution [64.54162195322246]
  Convolutional neural network (CNN) has achieved great success on image super-resolution (SR) Most deep CNN-based SR models take massive computations to obtain high performance. We propose a novel Hybrid Pixel-Unshuffled Network (HPUN) by introducing an efficient and effective downsampling module into the SR task.
  arXiv  Detail & Related papers  (2022-03-16T20:10:41Z)
• Super-Efficient Super Resolution for Fast Adversarial Defense at the Edge [4.2698418800007865]
  A new technique has emerged for mitigating attacks on image classification Deep Neural Networks (DNNs) Super resolution incurs a heavy computational cost. We show that Super-Efficient Super Resolution (SESR) achieves similar or better image quality than prior art while requiring 2x to 330x fewer Multiply-Accumulate (MAC) operations.
  arXiv  Detail & Related papers  (2021-12-29T00:35:41Z)
• Generating Superpixels for High-resolution Images with Decoupled Patch Calibration [82.21559299694555]
  Patch Networks (PCNet) is designed to efficiently and accurately implement high-resolution superpixel segmentation. DPC takes a local patch from the high-resolution images and dynamically generates a binary mask to impose the network to focus on region boundaries. In particular, DPC takes a local patch from the high-resolution images and dynamically generates a binary mask to impose the network to focus on region boundaries.
  arXiv  Detail & Related papers  (2021-08-19T10:33:05Z)
• Extremely Lightweight Quantization Robust Real-Time Single-Image Super Resolution for Mobile Devices [0.0]
  Single-Image Super Resolution (SISR) is a classical computer vision problem and it has been studied for over decades. Recent work on SISR focuses solutions with deep learning methodologies and achieves state-of-the-art results. We propose a hardware (Synaptics Dolphin NPU) aware, extremely lightweight quantization robust real-time super resolution network (XLSR)
  arXiv  Detail & Related papers  (2021-05-21T11:29:48Z)
• AIM 2020 Challenge on Video Extreme Super-Resolution: Methods and Results [96.74919503142014]
  This paper reviews the video extreme super-resolution challenge associated with the AIM 2020 workshop at ECCV 2020. Track 1 is set up to gauge the state-of-the-art for such a demanding task, where fidelity to the ground truth is measured by PSNR and SSIM. Track 2 therefore aims at generating visually pleasing results, which are ranked according to human perception, evaluated by a user study.
  arXiv  Detail & Related papers  (2020-09-14T09:36:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.