Knowledge-aware Evolutionary Graph Neural Architecture Search

• URL: http://arxiv.org/abs/2411.17339v1
• Date: Tue, 26 Nov 2024 11:32:45 GMT
• Title: Knowledge-aware Evolutionary Graph Neural Architecture Search
• Authors: Chao Wang, Jiaxuan Zhao, Lingling Li, Licheng Jiao, Fang Liu, Xu Liu, Shuyuan Yang,
• Abstract summary: Graph neural architecture search (GNAS) can customize high-performance graph neural network architectures for specific graph tasks or datasets. Existing GNAS methods begin searching for architectures from a zero-knowledge state, ignoring the prior knowledge that may improve the search efficiency. This study proposes exploiting such prior knowledge to accelerate the multi-objective evolutionary search on a new graph dataset.
• Score: 49.13787973318586
• License:
• Abstract: Graph neural architecture search (GNAS) can customize high-performance graph neural network architectures for specific graph tasks or datasets. However, existing GNAS methods begin searching for architectures from a zero-knowledge state, ignoring the prior knowledge that may improve the search efficiency. The available knowledge base (e.g. NAS-Bench-Graph) contains many rich architectures and their multiple performance metrics, such as the accuracy (#Acc) and number of parameters (#Params). This study proposes exploiting such prior knowledge to accelerate the multi-objective evolutionary search on a new graph dataset, named knowledge-aware evolutionary GNAS (KEGNAS). KEGNAS employs the knowledge base to train a knowledge model and a deep multi-output Gaussian process (DMOGP) in one go, which generates and evaluates transfer architectures in only a few GPU seconds. The knowledge model first establishes a dataset-to-architecture mapping, which can quickly generate candidate transfer architectures for a new dataset. Subsequently, the DMOGP with architecture and dataset encodings is designed to predict multiple performance metrics for candidate transfer architectures on the new dataset. According to the predicted metrics, non-dominated candidate transfer architectures are selected to warm-start the multi-objective evolutionary algorithm for optimizing the #Acc and #Params on a new dataset. Empirical studies on NAS-Bench-Graph and five real-world datasets show that KEGNAS swiftly generates top-performance architectures, achieving 4.27% higher accuracy than advanced evolutionary baselines and 11.54% higher accuracy than advanced differentiable baselines. In addition, ablation studies demonstrate that the use of prior knowledge significantly improves the search performance.

Related papers

• Towards Lightweight Graph Neural Network Search with Curriculum Graph Sparsification [48.334100429553644]
  This paper proposes to design a joint graph data and architecture mechanism, which identifies important sub-architectures via the valuable graph data. To search for optimal lightweight Graph Neural Networks (GNNs), we propose a Lightweight Graph Neural Architecture Search with Graph SparsIfication and Network Pruning (GASSIP) method. Our method achieves on-par or even higher node classification performance with half or fewer model parameters of searched GNNs and a sparser graph.
  arXiv  Detail & Related papers  (2024-06-24T06:53:37Z)
• GeNAS: Neural Architecture Search with Better Generalization [14.92869716323226]
  Recent neural architecture search (NAS) approaches rely on validation loss or accuracy to find the superior network for the target data. In this paper, we investigate a new neural architecture search measure for excavating architectures with better generalization.
  arXiv  Detail & Related papers  (2023-05-15T12:44:54Z)
• Network Graph Based Neural Architecture Search [57.78724765340237]
  We search neural network by rewiring the corresponding graph and predict the architecture performance by graph properties. Because we do not perform machine learning over the entire graph space, the searching process is remarkably efficient.
  arXiv  Detail & Related papers  (2021-12-15T00:12:03Z)
• Edge-featured Graph Neural Architecture Search [131.4361207769865]
  We propose Edge-featured Graph Neural Architecture Search to find the optimal GNN architecture. Specifically, we design rich entity and edge updating operations to learn high-order representations. We show EGNAS can search better GNNs with higher performance than current state-of-the-art human-designed and searched-based GNNs.
  arXiv  Detail & Related papers  (2021-09-03T07:53:18Z)
• Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets [42.993720854755736]
  We propose an efficient Neural Search (NAS) framework that is trained once on a database consisting of datasets and pretrained networks. We show that our model meta-learned on subsets of ImageNet-1K and architectures from NAS-Bench 201 search space successfully generalizes to multiple unseen datasets.
  arXiv  Detail & Related papers  (2021-07-02T06:33:59Z)
• Weak NAS Predictors Are All You Need [91.11570424233709]
  Recent predictor-based NAS approaches attempt to solve the problem with two key steps: sampling some architecture-performance pairs and fitting a proxy accuracy predictor. We shift the paradigm from finding a complicated predictor that covers the whole architecture space to a set of weaker predictors that progressively move towards the high-performance sub-space. Our method costs fewer samples to find the top-performance architectures on NAS-Bench-101 and NAS-Bench-201, and it achieves the state-of-the-art ImageNet performance on the NASNet search space.
  arXiv  Detail & Related papers  (2021-02-21T01:58:43Z)
• Evolving Neural Architecture Using One Shot Model [5.188825486231326]
  We propose a novel way of applying a simple genetic algorithm to the NAS problem called EvNAS (Evolving Neural Architecture using One Shot Model) EvNAS searches for the architecture on the proxy dataset i.e. CIFAR-10 for 4.4 GPU day on a single GPU and achieves top-1 test error of 2.47%. Results show the potential of evolutionary methods in solving the architecture search problem.
  arXiv  Detail & Related papers  (2020-12-23T08:40:53Z)
• A Semi-Supervised Assessor of Neural Architectures [157.76189339451565]
  We employ an auto-encoder to discover meaningful representations of neural architectures. A graph convolutional neural network is introduced to predict the performance of architectures.
  arXiv  Detail & Related papers  (2020-05-14T09:02:33Z)
• A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAS [18.409809742204896]
  This work proposes a novel Graph-based neural ArchiTecture Scheme, a.k.a. a GATES, to improve the predictor-based neural architecture search. Gates models the operations as the transformation of the propagating information, which mimics the actual data processing of neural architecture.
  arXiv  Detail & Related papers  (2020-04-04T09:54:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.