SETGAN: Scale and Energy Trade-off GANs for Image Applications on Mobile Platforms

• URL: http://arxiv.org/abs/2103.12896v1
• Date: Tue, 23 Mar 2021 23:51:22 GMT
• Title: SETGAN: Scale and Energy Trade-off GANs for Image Applications on Mobile Platforms
• Authors: Nitthilan Kannappan Jayakodi, Janardhan Rao Doppa, Partha Pratim Pande
• Abstract summary: We propose a novel approach to trade-off image generation accuracy of a GAN for the energy consumed at run-time called Scale-Energy Tradeoff GAN (SETGAN) We use SinGAN, a single image unconditional generative model, that contains a pyramid of fully convolutional GANs. With SETGAN's unique client-server-based architecture, we were able to achieve a 56% gain in energy for a loss of 3% to 12% SSIM accuracy.
• Score: 15.992829133103921
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the task of photo-realistic unconditional image generation (generate high quality, diverse samples that carry the same visual content as the image) on mobile platforms using Generative Adversarial Networks (GANs). In this paper, we propose a novel approach to trade-off image generation accuracy of a GAN for the energy consumed (compute) at run-time called Scale-Energy Tradeoff GAN (SETGAN). GANs usually take a long time to train and consume a huge memory hence making it difficult to run on edge devices. The key idea behind SETGAN for an image generation task is for a given input image, we train a GAN on a remote server and use the trained model on edge devices. We use SinGAN, a single image unconditional generative model, that contains a pyramid of fully convolutional GANs, each responsible for learning the patch distribution at a different scale of the image. During the training process, we determine the optimal number of scales for a given input image and the energy constraint from the target edge device. Results show that with SETGAN's unique client-server-based architecture, we were able to achieve a 56% gain in energy for a loss of 3% to 12% SSIM accuracy. Also, with the parallel multi-scale training, we obtain around 4x gain in training time on the server.

Related papers

• Faster Projected GAN: Towards Faster Few-Shot Image Generation [10.068622488926172]
  This paper proposes an improved GAN network model, which is named Faster Projected GAN, based on Projected GAN. By introducing depth separable convolution (DSC), the number of parameters of the Projected GAN is reduced, the training speed is accelerated, and memory is saved.
  arXiv  Detail & Related papers  (2024-01-23T07:55:27Z)
• E$^{2}$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation [69.72194342962615]
  We introduce and address a novel research direction: can the process of distilling GANs from diffusion models be made significantly more efficient? First, we construct a base GAN model with generalized features, adaptable to different concepts through fine-tuning, eliminating the need for training from scratch. Second, we identify crucial layers within the base GAN model and employ Low-Rank Adaptation (LoRA) with a simple yet effective rank search process, rather than fine-tuning the entire base model. Third, we investigate the minimal amount of data necessary for fine-tuning, further reducing the overall training time.
  arXiv  Detail & Related papers  (2024-01-11T18:59:14Z)
• Using a Conditional Generative Adversarial Network to Control the Statistical Characteristics of Generated Images for IACT Data Analysis [55.41644538483948]
  We divide images into several classes according to the value of some property of the image, and then specify the required class when generating new images. In the case of images from Imaging Atmospheric Cherenkov Telescopes (IACTs), an important property is the total brightness of all image pixels (image size) We used a cGAN technique to generate images similar to whose obtained in the TAIGA-IACT experiment.
  arXiv  Detail & Related papers  (2022-11-28T22:30:33Z)
• BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations [89.42397034542189]
  We synthesize a large labeled dataset via a generative adversarial network (GAN) We take image samples from the class-conditional generative model BigGAN trained on ImageNet, and manually annotate 5 images per class, for all 1k classes. We create a new ImageNet benchmark by labeling an additional set of 8k real images and evaluate segmentation performance in a variety of settings.
  arXiv  Detail & Related papers  (2022-01-12T20:28:34Z)
• A Shared Representation for Photorealistic Driving Simulators [83.5985178314263]
  We propose to improve the quality of generated images by rethinking the discriminator architecture. The focus is on the class of problems where images are generated given semantic inputs, such as scene segmentation maps or human body poses. We aim to learn a shared latent representation that encodes enough information to jointly do semantic segmentation, content reconstruction, along with a coarse-to-fine grained adversarial reasoning.
  arXiv  Detail & Related papers  (2021-12-09T18:59:21Z)
• InvGAN: Invertible GANs [88.58338626299837]
  InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model. This allows us to perform image inpainting, merging, and online data augmentation.
  arXiv  Detail & Related papers  (2021-12-08T21:39:00Z)
• Projected GANs Converge Faster [50.23237734403834]
  Generative Adversarial Networks (GANs) produce high-quality images but are challenging to train. We make significant headway on these issues by projecting generated and real samples into a fixed, pretrained feature space. Our Projected GAN improves image quality, sample efficiency, and convergence speed.
  arXiv  Detail & Related papers  (2021-11-01T15:11:01Z)
• Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis [21.40315235087551]
  We propose a light-weight GAN structure that gains superior quality on 1024*1024 resolution. We show our model's superior performance compared to the state-of-the-art StyleGAN2, when data and computing budget are limited.
  arXiv  Detail & Related papers  (2021-01-12T22:02:54Z)
• TinyGAN: Distilling BigGAN for Conditional Image Generation [2.8072597424460466]
  BigGAN has significantly improved the quality of image generation on ImageNet, but it requires a huge model, making it hard to deploy on resource-constrained devices. We propose a black-box knowledge distillation framework for compressing GANs, which highlights a stable and efficient training process. Given BigGAN as the teacher network, we manage to train a much smaller student network to mimic its functionality, achieving competitive performance on Inception and FID scores with the generator having $16times$ fewer parameters.
  arXiv  Detail & Related papers  (2020-09-29T07:33:49Z)
• Training End-to-end Single Image Generators without GANs [27.393821783237186]
  AugurOne is a novel approach for training single image generative models. Our approach trains an upscaling neural network using non-affine augmentations of the (single) input image. A compact latent space is jointly learned allowing for controlled image synthesis.
  arXiv  Detail & Related papers  (2020-04-07T17:58:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.