EAGAN: Efficient Two-stage Evolutionary Architecture Search for GANs

• URL: http://arxiv.org/abs/2111.15097v1
• Date: Tue, 30 Nov 2021 03:28:09 GMT
• Title: EAGAN: Efficient Two-stage Evolutionary Architecture Search for GANs
• Authors: Guohao Ying, Xin He, Bin Gao, Bo Han, Xiaowen Chu
• Abstract summary: Generative Adversarial Networks (GANs) have been proven hugely successful in image generation tasks, but GAN training has the problem of instability. We propose an efficient two-stage evolutionary algorithm (EA) based NAS framework to discover GANs, dubbed textbfEAGAN.
• Score: 25.791031022393643
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative Adversarial Networks (GANs) have been proven hugely successful in image generation tasks, but GAN training has the problem of instability. Many works have improved the stability of GAN training by manually modifying the GAN architecture, which requires human expertise and extensive trial-and-error. Thus, neural architecture search (NAS), which aims to automate the model design, has been applied to search GANs on the task of unconditional image generation. The early NAS-GAN works only search generators for reducing the difficulty. Some recent works have attempted to search both generator (G) and discriminator (D) to improve GAN performance, but they still suffer from the instability of GAN training during the search. To alleviate the instability issue, we propose an efficient two-stage evolutionary algorithm (EA) based NAS framework to discover GANs, dubbed \textbf{EAGAN}. Specifically, we decouple the search of G and D into two stages and propose the weight-resetting strategy to improve the stability of GAN training. Besides, we perform evolution operations to produce the Pareto-front architectures based on multiple objectives, resulting in a superior combination of G and D. By leveraging the weight-sharing strategy and low-fidelity evaluation, EAGAN can significantly shorten the search time. EAGAN achieves highly competitive results on the CIFAR-10 (IS=8.81$\pm$0.10, FID=9.91) and surpasses previous NAS-searched GANs on the STL-10 dataset (IS=10.44$\pm$0.087, FID=22.18).

Related papers

• Improving Routability Prediction via NAS Using a Smooth One-shot Augmented Predictor [4.381471268889061]
  Routability optimization in modern EDA tools has benefited greatly from using machine learning (ML) models. NAS techniques struggle to perform well on routability prediction as a result of two primary factors. We craft a novel NAS technique, coined SOAP-NAS, to address these challenges through novel data augmentation techniques and a novel combination of one-shot and predictor-based NAS.
  arXiv  Detail & Related papers  (2024-11-21T16:42:41Z)
• ParaGAN: A Scalable Distributed Training Framework for Generative Adversarial Networks [8.736828068085263]
  ParaGAN is a scalable distributed GAN training framework that leverages asynchronous training and asymmetric optimization policy to accelerate GAN training. With ParaGAN, we reduce the training time of BigGAN from 15 days to 14 hours while achieving 91% scaling efficiency. ParaGAN enables unprecedented high-resolution image generation using BigGAN.
  arXiv  Detail & Related papers  (2024-11-06T15:40:46Z)
• On the Convergence of (Stochastic) Gradient Descent for Kolmogorov--Arnold Networks [56.78271181959529]
  Kolmogorov--Arnold Networks (KANs) have gained significant attention in the deep learning community. Empirical investigations demonstrate that KANs optimized via gradient descent (SGD) are capable of achieving near-zero training loss.
  arXiv  Detail & Related papers  (2024-10-10T15:34:10Z)
• A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanism [58.855741970337675]
  Neural architecture search (NAS) enables re-searchers to automatically explore vast search spaces and find efficient neural networks. NAS suffers from a key bottleneck, i.e., numerous architectures need to be evaluated during the search process. We propose the SMEM-NAS, a pairwise com-parison relation-assisted multi-objective evolutionary algorithm based on a multi-population mechanism.
  arXiv  Detail & Related papers  (2024-07-22T12:46:22Z)
• DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit CNNs [53.82853297675979]
  1-bit convolutional neural networks (CNNs) with binary weights and activations show their potential for resource-limited embedded devices. One natural approach is to use 1-bit CNNs to reduce the computation and memory cost of NAS. We introduce Discrepant Child-Parent Neural Architecture Search (DCP-NAS) to efficiently search 1-bit CNNs.
  arXiv  Detail & Related papers  (2023-06-27T11:28:29Z)
• DiffusionNAG: Predictor-guided Neural Architecture Generation with Diffusion Models [56.584561770857306]
  We propose a novel conditional Neural Architecture Generation (NAG) framework based on diffusion models, dubbed DiffusionNAG. Specifically, we consider the neural architectures as directed graphs and propose a graph diffusion model for generating them. We validate the effectiveness of DiffusionNAG through extensive experiments in two predictor-based NAS scenarios: Transferable NAS and Bayesian Optimization (BO)-based NAS. When integrated into a BO-based algorithm, DiffusionNAG outperforms existing BO-based NAS approaches, particularly in the large MobileNetV3 search space on the ImageNet 1K dataset.
  arXiv  Detail & Related papers  (2023-05-26T13:58:18Z)
• Guided Evolution for Neural Architecture Search [1.0499611180329804]
  We propose a novel approach for guided evolutionary Neural Architecture Search (NAS) The rationale behind G-EA, is to explore the search space by generating and evaluating several architectures in each generation. G-EA forces exploitation of the most performant networks by descendant generation while at the same time forcing exploration by parent mutation.
  arXiv  Detail & Related papers  (2021-10-28T15:43:20Z)
• Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics [117.4281417428145]
  This work targets designing a principled and unified training-free framework for Neural Architecture Search (NAS) NAS has been explosively studied to automate the discovery of top-performer neural networks, but suffers from heavy resource consumption and often incurs search bias due to truncated training or approximations. We present a unified framework to understand and accelerate NAS, by disentangling "TEG" characteristics of searched networks.
  arXiv  Detail & Related papers  (2021-08-26T17:52:07Z)
• DO-GAN: A Double Oracle Framework for Generative Adversarial Networks [28.904057977044374]
  We propose a new approach to train Generative Adversarial Networks (GANs) We deploy a double-oracle framework using the generator and discriminator oracles. We apply our framework to established GAN architectures such as vanilla GAN, Deep Convolutional GAN, Spectral Normalization GAN and Stacked GAN.
  arXiv  Detail & Related papers  (2021-02-17T05:11:18Z)
• Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search [50.40004966087121]
  We introduce a new reinforcement learning based neural architecture search (NAS) methodology for generative adversarial network (GAN) architecture search. The key idea is to formulate the GAN architecture search problem as a Markov decision process (MDP) for smoother architecture sampling. We exploit an off-policy GAN architecture search algorithm that makes efficient use of the samples generated by previous policies.
  arXiv  Detail & Related papers  (2020-07-17T18:29:17Z)
• AlphaGAN: Fully Differentiable Architecture Search for Generative Adversarial Networks [15.740179244963116]
  Generative Adversarial Networks (GANs) are formulated as minimax game problems, whereby generators attempt to approach real data distributions by virtue of adversarial learning against discriminators. In this work, we aim to boost model learning from the perspective of network architectures, by incorporating recent progress on automated architecture search into GANs. We propose a fully differentiable search framework for generative adversarial networks, dubbed alphaGAN.
  arXiv  Detail & Related papers  (2020-06-16T13:27:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.