Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis

• URL: http://arxiv.org/abs/2101.04775v1
• Date: Tue, 12 Jan 2021 22:02:54 GMT
• Title: Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis
• Authors: Bingchen Liu, Yizhe Zhu, Kunpeng Song, Ahmed Elgammal
• Abstract summary: We propose a light-weight GAN structure that gains superior quality on 1024*1024 resolution. We show our model's superior performance compared to the state-of-the-art StyleGAN2, when data and computing budget are limited.
• Score: 21.40315235087551
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training Generative Adversarial Networks (GAN) on high-fidelity images usually requires large-scale GPU-clusters and a vast number of training images. In this paper, we study the few-shot image synthesis task for GAN with minimum computing cost. We propose a light-weight GAN structure that gains superior quality on 1024*1024 resolution. Notably, the model converges from scratch with just a few hours of training on a single RTX-2080 GPU, and has a consistent performance, even with less than 100 training samples. Two technique designs constitute our work, a skip-layer channel-wise excitation module and a self-supervised discriminator trained as a feature-encoder. With thirteen datasets covering a wide variety of image domains (The datasets and code are available at: https://github.com/odegeasslbc/FastGAN-pytorch), we show our model's superior performance compared to the state-of-the-art StyleGAN2, when data and computing budget are limited.

Related papers

• CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation [69.43106794519193]
  We propose the CtrLoRA framework, which trains a Base ControlNet to learn the common knowledge of image-to-image generation from multiple base conditions. Our framework reduces the learnable parameters by 90% compared to ControlNet, significantly lowering the threshold to distribute and deploy the model weights.
  arXiv  Detail & Related papers  (2024-10-12T07:04:32Z)
• E$^{2}$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation [69.72194342962615]
  We introduce and address a novel research direction: can the process of distilling GANs from diffusion models be made significantly more efficient? First, we construct a base GAN model with generalized features, adaptable to different concepts through fine-tuning, eliminating the need for training from scratch. Second, we identify crucial layers within the base GAN model and employ Low-Rank Adaptation (LoRA) with a simple yet effective rank search process, rather than fine-tuning the entire base model. Third, we investigate the minimal amount of data necessary for fine-tuning, further reducing the overall training time.
  arXiv  Detail & Related papers  (2024-01-11T18:59:14Z)
• Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models [6.821399706256863]
  W"urstchen is a novel architecture for text-to-image synthesis that combines competitive performance with unprecedented cost-effectiveness. A key contribution of our work is to develop a latent diffusion technique in which we learn a detailed but extremely compact semantic image representation.
  arXiv  Detail & Related papers  (2023-06-01T13:00:53Z)
• InvGAN: Invertible GANs [88.58338626299837]
  InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model. This allows us to perform image inpainting, merging, and online data augmentation.
  arXiv  Detail & Related papers  (2021-12-08T21:39:00Z)
• Ultra-Data-Efficient GAN Training: Drawing A Lottery Ticket First, Then Training It Toughly [114.81028176850404]
  Training generative adversarial networks (GANs) with limited data generally results in deteriorated performance and collapsed models. We decompose the data-hungry GAN training into two sequential sub-problems. Such a coordinated framework enables us to focus on lower-complexity and more data-efficient sub-problems.
  arXiv  Detail & Related papers  (2021-02-28T05:20:29Z)
• Improving the Speed and Quality of GAN by Adversarial Training [87.70013107142142]
  We develop FastGAN to improve the speed and quality of GAN training based on the adversarial training technique. Our training algorithm brings ImageNet training to the broader public by requiring 2-4 GPUs.
  arXiv  Detail & Related papers  (2020-08-07T20:21:31Z)
• Training End-to-end Single Image Generators without GANs [27.393821783237186]
  AugurOne is a novel approach for training single image generative models. Our approach trains an upscaling neural network using non-affine augmentations of the (single) input image. A compact latent space is jointly learned allowing for controlled image synthesis.
  arXiv  Detail & Related papers  (2020-04-07T17:58:03Z)
• GAN Compression: Efficient Architectures for Interactive Conditional GANs [45.012173624111185]
  Recent Conditional Generative Adversarial Networks (cGANs) are 1-2 orders of magnitude more compute-intensive than modern recognition CNNs. We propose a general-purpose compression framework for reducing the inference time and model size of the generator in cGANs.
  arXiv  Detail & Related papers  (2020-03-19T17:59:05Z)
• Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs [104.85633684716296]
  We show that simple fine-tuning of GANs with frozen lower layers of the discriminator performs surprisingly well. This simple baseline, FreezeD, significantly outperforms previous techniques used in both unconditional and conditional GANs.
  arXiv  Detail & Related papers  (2020-02-25T15:30:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.