Training Generative Adversarial Networks in One Stage

• URL: http://arxiv.org/abs/2103.00430v2
• Date: Thu, 4 Mar 2021 02:56:12 GMT
• Title: Training Generative Adversarial Networks in One Stage
• Authors: Chengchao Shen, Youtan Yin, Xinchao Wang, Xubin LI, Jie Song, Mingli Song
• Abstract summary: We introduce a general training scheme that enables training GANs efficiently in only one stage. We show that the proposed method is readily applicable to other adversarial-training scenarios, such as data-free knowledge distillation.
• Score: 58.983325666852856
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative Adversarial Networks (GANs) have demonstrated unprecedented success in various image generation tasks. The encouraging results, however, come at the price of a cumbersome training process, during which the generator and discriminator are alternately updated in two stages. In this paper, we investigate a general training scheme that enables training GANs efficiently in only one stage. Based on the adversarial losses of the generator and discriminator, we categorize GANs into two classes, Symmetric GANs and Asymmetric GANs, and introduce a novel gradient decomposition method to unify the two, allowing us to train both classes in one stage and hence alleviate the training effort. Computational analysis and experimental results on several datasets and various network architectures demonstrate that, the proposed one-stage training scheme yields a solid 1.5$\times$ acceleration over conventional training schemes, regardless of the network architectures of the generator and discriminator. Furthermore, we show that the proposed method is readily applicable to other adversarial-training scenarios, such as data-free knowledge distillation. Our source code will be published soon.

Related papers

• Structural Credit Assignment with Coordinated Exploration [0.0]
  Methods aimed at improving structural credit assignment can generally be classified into two categories. We propose the use of Boltzmann machines or a recurrent network for coordinated exploration. Experimental results demonstrate that coordinated exploration significantly exceeds independent exploration in training speed.
  arXiv  Detail & Related papers  (2023-07-25T04:55:45Z)
• CoopInit: Initializing Generative Adversarial Networks via Cooperative Learning [50.90384817689249]
  CoopInit is a cooperative learning-based strategy that can quickly learn a good starting point for GANs. We demonstrate the effectiveness of the proposed approach on image generation and one-sided unpaired image-to-image translation tasks.
  arXiv  Detail & Related papers  (2023-03-21T07:49:32Z)
• Learning from Data with Noisy Labels Using Temporal Self-Ensemble [11.245833546360386]
  Deep neural networks (DNNs) have an enormous capacity to memorize noisy labels. Current state-of-the-art methods present a co-training scheme that trains dual networks using samples associated with small losses. We propose a simple yet effective robust training scheme that operates by training only a single network.
  arXiv  Detail & Related papers  (2022-07-21T08:16:31Z)
• Self-Ensembling GAN for Cross-Domain Semantic Segmentation [107.27377745720243]
  This paper proposes a self-ensembling generative adversarial network (SE-GAN) exploiting cross-domain data for semantic segmentation. In SE-GAN, a teacher network and a student network constitute a self-ensembling model for generating semantic segmentation maps, which together with a discriminator, forms a GAN. Despite its simplicity, we find SE-GAN can significantly boost the performance of adversarial training and enhance the stability of the model.
  arXiv  Detail & Related papers  (2021-12-15T09:50:25Z)
• Self-Supervised Learning for Binary Networks by Joint Classifier Training [11.612308609123566]
  We propose a self-supervised learning method for binary networks. For better training of the binary network, we propose a feature similarity loss, a dynamic balancing scheme of loss terms, and modified multi-stage training. Our empirical validations show that BSSL outperforms self-supervised learning baselines for binary networks in various downstream tasks and outperforms supervised pretraining in certain tasks.
  arXiv  Detail & Related papers  (2021-10-17T15:38:39Z)
• Training ELECTRA Augmented with Multi-word Selection [53.77046731238381]
  We present a new text encoder pre-training method that improves ELECTRA based on multi-task learning. Specifically, we train the discriminator to simultaneously detect replaced tokens and select original tokens from candidate sets.
  arXiv  Detail & Related papers  (2021-05-31T23:19:00Z)
• Regularized Generative Adversarial Network [0.0]
  We propose a framework for generating samples from a probability distribution that differs from the probability distribution of the training set. We refer to this new model as regularized generative adversarial network (RegGAN)
  arXiv  Detail & Related papers  (2021-02-09T01:13:36Z)
• Improving GAN Training with Probability Ratio Clipping and Sample Reweighting [145.5106274085799]
  generative adversarial networks (GANs) often suffer from inferior performance due to unstable training. We propose a new variational GAN training framework which enjoys superior training stability. By plugging the training approach in diverse state-of-the-art GAN architectures, we obtain significantly improved performance over a range of tasks.
  arXiv  Detail & Related papers  (2020-06-12T01:39:48Z)
• Parallel/distributed implementation of cellular training for generative adversarial neural networks [7.504722086511921]
  Generative adversarial networks (GANs) are widely used to learn generative models. This article presents a parallel/distributed implementation of a cellular competitive coevolutionary method to train two populations of GANs.
  arXiv  Detail & Related papers  (2020-04-07T16:01:58Z)
• Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN [80.17705319689139]
  We propose a data-free knowledge amalgamate strategy to craft a well-behaved multi-task student network from multiple single/multi-task teachers. The proposed method without any training data achieves the surprisingly competitive results, even compared with some full-supervised methods.
  arXiv  Detail & Related papers  (2020-03-20T03:20:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.