BSRA: Block-based Super Resolution Accelerator with Hardware Efficient Pixel Attention

• URL: http://arxiv.org/abs/2205.00777v1
• Date: Mon, 2 May 2022 09:56:29 GMT
• Title: BSRA: Block-based Super Resolution Accelerator with Hardware Efficient Pixel Attention
• Authors: Dun-Hao Yang, and Tian-Sheuan Chang
• Abstract summary: This paper proposes a super resolution hardware accelerator with hardware efficient pixel attention. The final implementation can support full HD image reconstruction at 30 frames per second with TSMC 40nm CMOS process.
• Score: 0.10547353841674209
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Increasingly, convolution neural network (CNN) based super resolution models have been proposed for better reconstruction results, but their large model size and complicated structure inhibit their real-time hardware implementation. Current hardware designs are limited to a plain network and suffer from lower quality and high memory bandwidth requirements. This paper proposes a super resolution hardware accelerator with hardware efficient pixel attention that just needs 25.9K parameters and simple structure but achieves 0.38dB better reconstruction images than the widely used FSRCNN. The accelerator adopts full model block wise convolution for full model layer fusion to reduce external memory access to model input and output only. In addition, CNN and pixel attention are well supported by PE arrays with distributed weights. The final implementation can support full HD image reconstruction at 30 frames per second with TSMC 40nm CMOS process.

Related papers

• Hierarchical Patch Diffusion Models for High-Resolution Video Generation [50.42746357450949]
  We develop deep context fusion, which propagates context information from low-scale to high-scale patches in a hierarchical manner. We also propose adaptive computation, which allocates more network capacity and computation towards coarse image details. The resulting model sets a new state-of-the-art FVD score of 66.32 and Inception Score of 87.68 in class-conditional video generation.
  arXiv  Detail & Related papers  (2024-06-12T01:12:53Z)
• Image super-resolution via dynamic network [19.404066956727885]
  We present a dynamic network for image super-resolution (DSRNet) It contains a residual enhancement block, wide enhancement block, feature refinement block and construction block. Our method is more competitive in terms of performance and recovering time of image super-resolution and complexity.
  arXiv  Detail & Related papers  (2023-10-16T13:56:56Z)
• ACNPU: A 4.75TOPS/W 1080P@30FPS Super Resolution Accelerator with Decoupled Asymmetric Convolution [0.0502254944841629]
  Deep learning-driven superresolution (SR) outperforms traditional techniques but also faces the challenge of high complexity and memory bandwidth. This paper proposes an energy-efficient SR accelerator, ACNPU, to tackle this challenge. The ACNPU enhances image quality by 0.34dB with a 27-layer model, but needs 36% less complexity than FSRCNN.
  arXiv  Detail & Related papers  (2023-08-30T07:23:32Z)
• ShuffleMixer: An Efficient ConvNet for Image Super-Resolution [88.86376017828773]
  We propose ShuffleMixer, for lightweight image super-resolution that explores large convolution and channel split-shuffle operation. Specifically, we develop a large depth-wise convolution and two projection layers based on channel splitting and shuffling as the basic component to mix features efficiently. Experimental results demonstrate that the proposed ShuffleMixer is about 6x smaller than the state-of-the-art methods in terms of model parameters and FLOPs.
  arXiv  Detail & Related papers  (2022-05-30T15:26:52Z)
• EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction [67.11722682878722]
  This work presents EfficientViT, a new family of high-resolution vision models with novel multi-scale linear attention. Our multi-scale linear attention achieves the global receptive field and multi-scale learning. EfficientViT delivers remarkable performance gains over previous state-of-the-art models.
  arXiv  Detail & Related papers  (2022-05-29T20:07:23Z)
• Hybrid Pixel-Unshuffled Network for Lightweight Image Super-Resolution [64.54162195322246]
  Convolutional neural network (CNN) has achieved great success on image super-resolution (SR) Most deep CNN-based SR models take massive computations to obtain high performance. We propose a novel Hybrid Pixel-Unshuffled Network (HPUN) by introducing an efficient and effective downsampling module into the SR task.
  arXiv  Detail & Related papers  (2022-03-16T20:10:41Z)
• Super-Efficient Super Resolution for Fast Adversarial Defense at the Edge [4.2698418800007865]
  A new technique has emerged for mitigating attacks on image classification Deep Neural Networks (DNNs) Super resolution incurs a heavy computational cost. We show that Super-Efficient Super Resolution (SESR) achieves similar or better image quality than prior art while requiring 2x to 330x fewer Multiply-Accumulate (MAC) operations.
  arXiv  Detail & Related papers  (2021-12-29T00:35:41Z)
• Projected GANs Converge Faster [50.23237734403834]
  Generative Adversarial Networks (GANs) produce high-quality images but are challenging to train. We make significant headway on these issues by projecting generated and real samples into a fixed, pretrained feature space. Our Projected GAN improves image quality, sample efficiency, and convergence speed.
  arXiv  Detail & Related papers  (2021-11-01T15:11:01Z)
• Generating Superpixels for High-resolution Images with Decoupled Patch Calibration [82.21559299694555]
  Patch Networks (PCNet) is designed to efficiently and accurately implement high-resolution superpixel segmentation. DPC takes a local patch from the high-resolution images and dynamically generates a binary mask to impose the network to focus on region boundaries. In particular, DPC takes a local patch from the high-resolution images and dynamically generates a binary mask to impose the network to focus on region boundaries.
  arXiv  Detail & Related papers  (2021-08-19T10:33:05Z)
• Extremely Lightweight Quantization Robust Real-Time Single-Image Super Resolution for Mobile Devices [0.0]
  Single-Image Super Resolution (SISR) is a classical computer vision problem and it has been studied for over decades. Recent work on SISR focuses solutions with deep learning methodologies and achieves state-of-the-art results. We propose a hardware (Synaptics Dolphin NPU) aware, extremely lightweight quantization robust real-time super resolution network (XLSR)
  arXiv  Detail & Related papers  (2021-05-21T11:29:48Z)
• Towards Fast and Light-Weight Restoration of Dark Images [26.779714419544085]
  We show that we can enhance a full resolution, 2848 x 4256, extremely dark single-image in the ballpark of 3 seconds even on a CPU. We achieve this with 2 - 7x fewer model parameters, 2 - 3x lower memory utilization, 5 - 20x speed up and yet maintain a competitive image reconstruction quality.
  arXiv  Detail & Related papers  (2020-11-28T13:53:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.