FusedProp: Towards Efficient Training of Generative Adversarial Networks

• URL: http://arxiv.org/abs/2004.03335v1
• Date: Mon, 30 Mar 2020 06:46:29 GMT
• Title: FusedProp: Towards Efficient Training of Generative Adversarial Networks
• Authors: Zachary Polizzi, Chuan-Yung Tsai
• Abstract summary: We propose the fused propagation algorithm which can be used to efficiently train the discriminator and the generator of common GANs simultaneously. We show that FusedProp achieves 1.49 times the training speed compared to the conventional training of GANs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative adversarial networks (GANs) are capable of generating strikingly realistic samples but state-of-the-art GANs can be extremely computationally expensive to train. In this paper, we propose the fused propagation (FusedProp) algorithm which can be used to efficiently train the discriminator and the generator of common GANs simultaneously using only one forward and one backward propagation. We show that FusedProp achieves 1.49 times the training speed compared to the conventional training of GANs, although further studies are required to improve its stability. By reporting our preliminary results and open-sourcing our implementation, we hope to accelerate future research on the training of GANs.

Related papers

• Pre-Training and Fine-Tuning Generative Flow Networks [61.90529626590415]
  We introduce a novel approach for reward-free pre-training of GFlowNets. By framing the training as a self-supervised problem, we propose an outcome-conditioned GFlowNet that learns to explore the candidate space. We show that the pre-trained OC-GFN model can allow for a direct extraction of a policy capable of sampling from any new reward functions in downstream tasks.
  arXiv  Detail & Related papers  (2023-10-05T09:53:22Z)
• Balanced Training for Sparse GANs [16.045866864231417]
  We propose a novel metric called the balance ratio (BR) to study the balance between the sparse generator and discriminator. We also introduce a new method called balanced dynamic sparse training (ADAPT), which seeks to control the BR during GAN training to achieve a good trade-off between performance and computational cost.
  arXiv  Detail & Related papers  (2023-02-28T15:34:01Z)
• SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks [20.18957052535565]
  We provide a new efficient version of the backpropagation algorithm, specialized to the case where the weights of the neural network being trained are sparse. Our algorithm is general, as it applies to arbitrary (unstructured) sparsity and common layer types. We show that it can yield speedups in end-to-end runtime experiments, both in transfer learning using already-sparsified networks, and in training sparse networks from scratch.
  arXiv  Detail & Related papers  (2023-02-09T18:54:05Z)
• Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance [47.94567935516651]
  Generative adversarial networks (GANs) have received an upsurging interest since being proposed due to the high quality of the generated data. For inference, the existing model compression techniques can reduce the model complexity with comparable performance. In this paper, we explore the possibility of directly training sparse GAN from scratch without involving any dense or pre-training steps.
  arXiv  Detail & Related papers  (2022-03-05T15:18:03Z)
• Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data [125.7135706352493]
  Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images. Recent studies have shown that training GANs with limited data remains formidable due to discriminator overfitting. This paper introduces a novel strategy called Adaptive Pseudo Augmentation (APA) to encourage healthy competition between the generator and the discriminator.
  arXiv  Detail & Related papers  (2021-11-12T18:13:45Z)
• Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions [66.05472746340142]
  Generative networks (GAN) are among the most successful for learning high-complexity, real-world distributions. In this paper we show how GANs can efficiently learn to the distribution of real-life images.
  arXiv  Detail & Related papers  (2021-06-04T17:33:29Z)
• Training Generative Adversarial Networks in One Stage [58.983325666852856]
  We introduce a general training scheme that enables training GANs efficiently in only one stage. We show that the proposed method is readily applicable to other adversarial-training scenarios, such as data-free knowledge distillation.
  arXiv  Detail & Related papers  (2021-02-28T09:03:39Z)
• Improving the Speed and Quality of GAN by Adversarial Training [87.70013107142142]
  We develop FastGAN to improve the speed and quality of GAN training based on the adversarial training technique. Our training algorithm brings ImageNet training to the broader public by requiring 2-4 GPUs.
  arXiv  Detail & Related papers  (2020-08-07T20:21:31Z)
• Parallel/distributed implementation of cellular training for generative adversarial neural networks [7.504722086511921]
  Generative adversarial networks (GANs) are widely used to learn generative models. This article presents a parallel/distributed implementation of a cellular competitive coevolutionary method to train two populations of GANs.
  arXiv  Detail & Related papers  (2020-04-07T16:01:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.