Generative Adversarial Neural Architecture Search
- URL: http://arxiv.org/abs/2105.09356v1
- Date: Wed, 19 May 2021 18:54:44 GMT
- Title: Generative Adversarial Neural Architecture Search
- Authors: Seyed Saeed Changiz Rezaei, Fred X. Han, Di Niu, Mohammad Salameh,
Keith Mills, Shuo Lian, Wei Lu, and Shangling Jui
- Abstract summary: We propose Generative Adversarial NAS (GA-NAS) with theoretically provable convergence guarantees.
We show that GA-NAS can be used to improve already optimized baselines found by other NAS methods.
- Score: 21.05611902967155
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Despite the empirical success of neural architecture search (NAS) in deep
learning applications, the optimality, reproducibility and cost of NAS schemes
remain hard to assess. In this paper, we propose Generative Adversarial NAS
(GA-NAS) with theoretically provable convergence guarantees, promoting
stability and reproducibility in neural architecture search. Inspired by
importance sampling, GA-NAS iteratively fits a generator to previously
discovered top architectures, thus increasingly focusing on important parts of
a large search space. Furthermore, we propose an efficient adversarial learning
approach, where the generator is trained by reinforcement learning based on
rewards provided by a discriminator, thus being able to explore the search
space without evaluating a large number of architectures. Extensive experiments
show that GA-NAS beats the best published results under several cases on three
public NAS benchmarks. In the meantime, GA-NAS can handle ad-hoc search
constraints and search spaces. We show that GA-NAS can be used to improve
already optimized baselines found by other NAS methods, including EfficientNet
and ProxylessNAS, in terms of ImageNet accuracy or the number of parameters, in
their original search space.
Related papers
Err
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.