Neural Architecture Search based on Cartesian Genetic Programming Coding Method

• URL: http://arxiv.org/abs/2103.07173v5
• Date: Tue, 28 Sep 2021 23:42:53 GMT
• Title: Neural Architecture Search based on Cartesian Genetic Programming Coding Method
• Authors: Xuan Wu, Linhan Jia, Xiuyi Zhang, Liang Chen, Yanchun Liang, You Zhou and Chunguo Wu
• Abstract summary: We propose an evolutionary approach of NAS based on CGP, called CGPNAS, to solve sentence classification task. The experimental results show that the searched architectures are comparable with the performance of human-designed architectures.
• Score: 6.519170476143571
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural architecture search (NAS) is a hot topic in the field of automated machine learning and outperforms humans in designing neural architectures on quite a few machine learning tasks. Motivated by the natural representation form of neural networks by the Cartesian genetic programming (CGP), we propose an evolutionary approach of NAS based on CGP, called CGPNAS, to solve sentence classification task. To evolve the architectures under the framework of CGP, the operations such as convolution are identified as the types of function nodes of CGP, and the evolutionary operations are designed based on Evolutionary Strategy. The experimental results show that the searched architectures are comparable with the performance of human-designed architectures. We verify the ability of domain transfer of our evolved architectures. The transfer experimental results show that the accuracy deterioration is lower than 2-5%. Finally, the ablation study identifies the Attention function as the single key function node and the linear transformations along could keep the accuracy similar with the full evolved architectures, which is worthy of investigation in the future.

Related papers

• Cartesian Genetic Programming Approach for Designing Convolutional Neural Networks [0.0]
  In designing artificial neural networks, one crucial aspect of the innovative approach is suggesting a novel neural architecture. In this work, we use pure Genetic Programming Approach to design CNNs, which employs only one genetic operation. In the course of preliminary experiments, our methodology yields promising results.
  arXiv  Detail & Related papers  (2024-09-30T18:10:06Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Neural Architecture Search for Speech Emotion Recognition [72.1966266171951]
  We propose to apply neural architecture search (NAS) techniques to automatically configure the SER models. We show that NAS can improve SER performance (54.89% to 56.28%) while maintaining model parameter sizes.
  arXiv  Detail & Related papers  (2022-03-31T10:16:10Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• On the Exploitation of Neuroevolutionary Information: Analyzing the Past for a More Efficient Future [60.99717891994599]
  We propose an approach that extracts information from neuroevolutionary runs, and use it to build a metamodel. We inspect the best structures found during neuroevolutionary searches of generative adversarial networks with varying characteristics.
  arXiv  Detail & Related papers  (2021-05-26T20:55:29Z)
• Evolutionary Architecture Search for Graph Neural Networks [23.691915813153496]
  We propose a novel AutoML framework through the evolution of individual models in a large Graph Neural Networks (GNN) architecture space. To the best of our knowledge, this is the first work to introduce and evaluate evolutionary architecture search for GNN models.
  arXiv  Detail & Related papers  (2020-09-21T22:11:53Z)
• Evolutionary NAS with Gene Expression Programming of Cellular Encoding [0.0]
  We present a new generative encoding scheme which embeds local graph transformations in chromosomes of linear fixed-length string. In experiments, the effectiveness of SLGE is shown in discovering architectures that improve the performance of CNN architectures.
  arXiv  Detail & Related papers  (2020-05-27T01:19:32Z)
• A Semi-Supervised Assessor of Neural Architectures [157.76189339451565]
  We employ an auto-encoder to discover meaningful representations of neural architectures. A graph convolutional neural network is introduced to predict the performance of architectures.
  arXiv  Detail & Related papers  (2020-05-14T09:02:33Z)
• Binarizing MobileNet via Evolution-based Searching [66.94247681870125]
  We propose a use of evolutionary search to facilitate the construction and training scheme when binarizing MobileNet. Inspired by one-shot architecture search frameworks, we manipulate the idea of group convolution to design efficient 1-Bit Convolutional Neural Networks (CNNs) Our objective is to come up with a tiny yet efficient binary neural architecture by exploring the best candidates of the group convolution.
  arXiv  Detail & Related papers  (2020-05-13T13:25:51Z)
• A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAS [18.409809742204896]
  This work proposes a novel Graph-based neural ArchiTecture Scheme, a.k.a. a GATES, to improve the predictor-based neural architecture search. Gates models the operations as the transformation of the propagating information, which mimics the actual data processing of neural architecture.
  arXiv  Detail & Related papers  (2020-04-04T09:54:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.