Generative Adversarial Super-Resolution at the Edge with Knowledge Distillation

• URL: http://arxiv.org/abs/2209.03355v2
• Date: Tue, 9 May 2023 15:14:50 GMT
• Title: Generative Adversarial Super-Resolution at the Edge with Knowledge Distillation
• Authors: Simone Angarano, Francesco Salvetti, Mauro Martini, Marcello Chiaberge
• Abstract summary: Single-Image Super-Resolution can support robotic tasks in environments where a reliable visual stream is required. We propose an efficient Generative Adversarial Network model for real-time Super-Resolution, called EdgeSRGAN.
• Score: 1.3764085113103222
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Single-Image Super-Resolution can support robotic tasks in environments where a reliable visual stream is required to monitor the mission, handle teleoperation or study relevant visual details. In this work, we propose an efficient Generative Adversarial Network model for real-time Super-Resolution, called EdgeSRGAN (code available at https://github.com/PIC4SeR/EdgeSRGAN). We adopt a tailored architecture of the original SRGAN and model quantization to boost the execution on CPU and Edge TPU devices, achieving up to 200 fps inference. We further optimize our model by distilling its knowledge to a smaller version of the network and obtain remarkable improvements compared to the standard training approach. Our experiments show that our fast and lightweight model preserves considerably satisfying image quality compared to heavier state-of-the-art models. Finally, we conduct experiments on image transmission with bandwidth degradation to highlight the advantages of the proposed system for mobile robotic applications.

Related papers

• Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration. We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• VmambaIR: Visual State Space Model for Image Restoration [36.11385876754612]
  We propose VmambaIR, which introduces State Space Models (SSMs) with linear complexity into comprehensive image restoration tasks. VmambaIR achieves state-of-the-art (SOTA) performance with much fewer computational resources and parameters.
  arXiv  Detail & Related papers  (2024-03-18T02:38:55Z)
• E$^{2}$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation [69.72194342962615]
  We introduce and address a novel research direction: can the process of distilling GANs from diffusion models be made significantly more efficient? First, we construct a base GAN model with generalized features, adaptable to different concepts through fine-tuning, eliminating the need for training from scratch. Second, we identify crucial layers within the base GAN model and employ Low-Rank Adaptation (LoRA) with a simple yet effective rank search process, rather than fine-tuning the entire base model. Third, we investigate the minimal amount of data necessary for fine-tuning, further reducing the overall training time.
  arXiv  Detail & Related papers  (2024-01-11T18:59:14Z)
• Swift Parameter-free Attention Network for Efficient Super-Resolution [8.365929625909509]
  Single Image Super-Resolution is a crucial task in low-level computer vision. We propose the Swift. parameter-free Attention Network (SPAN), which balances parameter count, inference speed, and image quality. We evaluate SPAN on multiple benchmarks, showing that it outperforms existing efficient super-resolution models in terms of both image quality and inference speed.
  arXiv  Detail & Related papers  (2023-11-21T18:30:40Z)
• A Light-weight Deep Learning Model for Remote Sensing Image Classification [70.66164876551674]
  We present a high-performance and light-weight deep learning model for Remote Sensing Image Classification (RSIC) By conducting extensive experiments on the NWPU-RESISC45 benchmark, our proposed teacher-student models outperforms the state-of-the-art systems.
  arXiv  Detail & Related papers  (2023-02-25T09:02:01Z)
• TcGAN: Semantic-Aware and Structure-Preserved GANs with Individual Vision Transformer for Fast Arbitrary One-Shot Image Generation [11.207512995742999]
  One-shot image generation (OSG) with generative adversarial networks that learn from the internal patches of a given image has attracted world wide attention. We propose a novel structure-preserved method TcGAN with individual vision transformer to overcome the shortcomings of the existing one-shot image generation methods.
  arXiv  Detail & Related papers  (2023-02-16T03:05:59Z)
• Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model [97.9548609175831]
  We resort to plain vision transformers with about 100 million parameters and make the first attempt to propose large vision models customized for remote sensing tasks. Specifically, to handle the large image size and objects of various orientations in RS images, we propose a new rotated varied-size window attention. Experiments on detection tasks demonstrate the superiority of our model over all state-of-the-art models, achieving 81.16% mAP on the DOTA-V1.0 dataset.
  arXiv  Detail & Related papers  (2022-08-08T09:08:40Z)
• Activating More Pixels in Image Super-Resolution Transformer [53.87533738125943]
  Transformer-based methods have shown impressive performance in low-level vision tasks, such as image super-resolution. We propose a novel Hybrid Attention Transformer (HAT) to activate more input pixels for better reconstruction. Our overall method significantly outperforms the state-of-the-art methods by more than 1dB.
  arXiv  Detail & Related papers  (2022-05-09T17:36:58Z)
• FMD-cGAN: Fast Motion Deblurring using Conditional Generative Adversarial Networks [26.878173373199786]
  We present a Fast Motion Deblurring-Conditional Generative Adversarial Network (FMD-cGAN) that helps in blind motion deblurring of a single image. FMD-cGAN delivers impressive structural similarity and visual appearance after deblurring an image.
  arXiv  Detail & Related papers  (2021-11-30T14:30:44Z)
• Projected GANs Converge Faster [50.23237734403834]
  Generative Adversarial Networks (GANs) produce high-quality images but are challenging to train. We make significant headway on these issues by projecting generated and real samples into a fixed, pretrained feature space. Our Projected GAN improves image quality, sample efficiency, and convergence speed.
  arXiv  Detail & Related papers  (2021-11-01T15:11:01Z)
• Attaining Real-Time Super-Resolution for Microscopic Images Using GAN [0.06345523830122167]
  This paper focuses on improving an existing deep-learning based method to perform Super-Resolution Microscopy in real-time using a standard GPU. We suggest simple changes to the architecture of the generator and the discriminator of SRGAN. We compare the quality and the running time for the outputs produced by our model, opening its applications in different areas like low-end benchtop and even mobile microscopy.
  arXiv  Detail & Related papers  (2020-10-09T15:26:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.