Evolutionary NAS with Gene Expression Programming of Cellular Encoding
- URL: http://arxiv.org/abs/2005.13110v2
- Date: Thu, 3 Dec 2020 15:41:20 GMT
- Title: Evolutionary NAS with Gene Expression Programming of Cellular Encoding
- Authors: Clifford Broni-Bediako, Yuki Murata, Luiz Henrique Mormille and
Masayasu Atsumi
- Abstract summary: We present a new generative encoding scheme which embeds local graph transformations in chromosomes of linear fixed-length string.
In experiments, the effectiveness of SLGE is shown in discovering architectures that improve the performance of CNN architectures.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The renaissance of neural architecture search (NAS) has seen classical
methods such as genetic algorithms (GA) and genetic programming (GP) being
exploited for convolutional neural network (CNN) architectures. While recent
work have achieved promising performance on visual perception tasks, the direct
encoding scheme of both GA and GP has functional complexity deficiency and does
not scale well on large architectures like CNN. To address this, we present a
new generative encoding scheme -- $symbolic\ linear\ generative\ encoding$
(SLGE) -- simple, yet powerful scheme which embeds local graph transformations
in chromosomes of linear fixed-length string to develop CNN architectures of
variant shapes and sizes via evolutionary process of gene expression
programming. In experiments, the effectiveness of SLGE is shown in discovering
architectures that improve the performance of the state-of-the-art handcrafted
CNN architectures on CIFAR-10 and CIFAR-100 image classification tasks; and
achieves a competitive classification error rate with the existing NAS methods
using less GPU resources.
Related papers
- Efficient Heterogeneous Graph Learning via Random Projection [65.65132884606072]
Heterogeneous Graph Neural Networks (HGNNs) are powerful tools for deep learning on heterogeneous graphs.
Recent pre-computation-based HGNNs use one-time message passing to transform a heterogeneous graph into regular-shaped tensors.
We propose a hybrid pre-computation-based HGNN, named Random Projection Heterogeneous Graph Neural Network (RpHGNN)
arXiv Detail & Related papers (2023-10-23T01:25:44Z) - Continuous Cartesian Genetic Programming based representation for
Multi-Objective Neural Architecture Search [12.545742558041583]
We propose a novel approach for designing less complex yet highly effective convolutional neural networks (CNNs)
Our approach combines real-based and block-chained CNNs representations based on cartesian genetic programming (CGP) for neural architecture search (NAS)
Two variants are introduced that differ in the granularity of the search space they consider.
arXiv Detail & Related papers (2023-06-05T07:32:47Z) - Set-based Neural Network Encoding [57.15855198512551]
We propose an approach to neural network weight encoding for generalization performance prediction.
Our approach is capable of encoding neural networks in a modelzoo of mixed architecture.
We introduce two new tasks for neural network generalization performance prediction: cross-dataset and cross-architecture.
arXiv Detail & Related papers (2023-05-26T04:34:28Z) - Neural Architecture Search Using Genetic Algorithm for Facial Expression
Recognition [2.7504274245107303]
We propose a genetic algorithm that uses an ingenious encoding-decoding mechanism that allows to automatically evolve CNNs on FER tasks.
The proposed algorithm achieves the best-known results on the CK+ and FERG datasets as well as competitive results on the JAFFE dataset.
arXiv Detail & Related papers (2023-04-12T16:36:07Z) - Simple and Efficient Heterogeneous Graph Neural Network [55.56564522532328]
Heterogeneous graph neural networks (HGNNs) have powerful capability to embed rich structural and semantic information of a heterogeneous graph into node representations.
Existing HGNNs inherit many mechanisms from graph neural networks (GNNs) over homogeneous graphs, especially the attention mechanism and the multi-layer structure.
This paper conducts an in-depth and detailed study of these mechanisms and proposes Simple and Efficient Heterogeneous Graph Neural Network (SeHGNN)
arXiv Detail & Related papers (2022-07-06T10:01:46Z) - ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network [72.16255675586089]
We propose an Adaptive Curvature Exploration Hyperbolic Graph NeuralNetwork named ACE-HGNN to adaptively learn the optimal curvature according to the input graph and downstream tasks.
Experiments on multiple real-world graph datasets demonstrate a significant and consistent performance improvement in model quality with competitive performance and good generalization ability.
arXiv Detail & Related papers (2021-10-15T07:18:57Z) - Neural Architecture Search based on Cartesian Genetic Programming Coding
Method [6.519170476143571]
We propose an evolutionary approach of NAS based on CGP, called CGPNAS, to solve sentence classification task.
The experimental results show that the searched architectures are comparable with the performance of human-designed architectures.
arXiv Detail & Related papers (2021-03-12T09:51:03Z) - Genetic U-Net: Automatically Designed Deep Networks for Retinal Vessel
Segmentation Using a Genetic Algorithm [2.6629444004809826]
Genetic U-Net is proposed to generate a U-shaped convolutional neural network (CNN) that can achieve better retinal vessel segmentation but with fewer architecture-based parameters.
The experimental results show that the architecture obtained using the proposed method offered a superior performance with less than 1% of the number of the original U-Net parameters in particular.
arXiv Detail & Related papers (2020-10-29T13:31:36Z) - A Semi-Supervised Assessor of Neural Architectures [157.76189339451565]
We employ an auto-encoder to discover meaningful representations of neural architectures.
A graph convolutional neural network is introduced to predict the performance of architectures.
arXiv Detail & Related papers (2020-05-14T09:02:33Z) - Binarized Graph Neural Network [65.20589262811677]
We develop a binarized graph neural network to learn the binary representations of the nodes with binary network parameters.
Our proposed method can be seamlessly integrated into the existing GNN-based embedding approaches.
Experiments indicate that the proposed binarized graph neural network, namely BGN, is orders of magnitude more efficient in terms of both time and space.
arXiv Detail & Related papers (2020-04-19T09:43:14Z) - A Generic Graph-based Neural Architecture Encoding Scheme for
Predictor-based NAS [18.409809742204896]
This work proposes a novel Graph-based neural ArchiTecture Scheme, a.k.a. a GATES, to improve the predictor-based neural architecture search.
Gates models the operations as the transformation of the propagating information, which mimics the actual data processing of neural architecture.
arXiv Detail & Related papers (2020-04-04T09:54:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.