Search to aggregate neighborhood for graph neural network

• URL: http://arxiv.org/abs/2104.06608v1
• Date: Wed, 14 Apr 2021 03:15:19 GMT
• Title: Search to aggregate neighborhood for graph neural network
• Authors: Huan Zhao, Quanming Yao, Weiwei Tu
• Abstract summary: We propose a framework, which tries to Search to Aggregate NEighborhood (SANE) to automatically design data-specific GNN architectures. By designing a novel and expressive search space, we propose a differentiable search algorithm, which is more efficient than previous reinforcement learning based methods.
• Score: 47.47628113034479
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent years have witnessed the popularity and success of graph neural networks (GNN) in various scenarios. To obtain data-specific GNN architectures, researchers turn to neural architecture search (NAS), which has made impressive success in discovering effective architectures in convolutional neural networks. However, it is non-trivial to apply NAS approaches to GNN due to challenges in search space design and the expensive searching cost of existing NAS methods. In this work, to obtain the data-specific GNN architectures and address the computational challenges facing by NAS approaches, we propose a framework, which tries to Search to Aggregate NEighborhood (SANE), to automatically design data-specific GNN architectures. By designing a novel and expressive search space, we propose a differentiable search algorithm, which is more efficient than previous reinforcement learning based methods. Experimental results on four tasks and seven real-world datasets demonstrate the superiority of SANE compared to existing GNN models and NAS approaches in terms of effectiveness and efficiency. (Code is available at: https://github.com/AutoML-4Paradigm/SANE).

Related papers

• DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions [121.05720140641189]
  We develop a family of models with the distilling neural architecture (DNA) techniques. Our proposed DNA models can rate all architecture candidates, as opposed to previous works that can only access a sub- search space using algorithms. Our models achieve state-of-the-art top-1 accuracy of 78.9% and 83.6% on ImageNet for a mobile convolutional network and a small vision transformer, respectively.
  arXiv  Detail & Related papers  (2024-03-02T22:16:47Z)
• Efficient and Explainable Graph Neural Architecture Search via Monte-Carlo Tree Search [5.076419064097733]
  Graph neural networks (GNNs) are powerful tools for performing data science tasks in various domains. To save human efforts and computational costs, graph neural architecture search (Graph NAS) has been used to search for a sub-optimal GNN architecture. We propose ExGNAS, which consists of (i) a simple search space that can adapt to various graphs and (ii) a search algorithm that makes the decision process explainable.
  arXiv  Detail & Related papers  (2023-08-30T03:21:45Z)
• Do Not Train It: A Linear Neural Architecture Search of Graph Neural Networks [15.823247346294089]
  We develop a novel NAS-GNNs method, namely neural architecture coding (NAC) Our approach leads to state-of-the-art performance, which is up to $200times$ faster and $18.8%$ more accurate than the strong baselines.
  arXiv  Detail & Related papers  (2023-05-23T13:44:04Z)
• GeNAS: Neural Architecture Search with Better Generalization [14.92869716323226]
  Recent neural architecture search (NAS) approaches rely on validation loss or accuracy to find the superior network for the target data. In this paper, we investigate a new neural architecture search measure for excavating architectures with better generalization.
  arXiv  Detail & Related papers  (2023-05-15T12:44:54Z)
• GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Models [48.57083463364353]
  We study neural architecture search (NAS) through the lens of learning random graph models. We propose GraphPNAS a deep graph generative model that learns a distribution of well-performing architectures. We show that our proposed graph generator consistently outperforms RNN-based one and achieves better or comparable performances than state-of-the-art NAS methods.
  arXiv  Detail & Related papers  (2022-11-28T09:09:06Z)
• UnrealNAS: Can We Search Neural Architectures with Unreal Data? [84.78460976605425]
  Neural architecture search (NAS) has shown great success in the automatic design of deep neural networks (DNNs) Previous work has analyzed the necessity of having ground-truth labels in NAS and inspired broad interest. We take a further step to question whether real data is necessary for NAS to be effective.
  arXiv  Detail & Related papers  (2022-05-04T16:30:26Z)
• NAS-FCOS: Efficient Search for Object Detection Architectures [113.47766862146389]
  We propose an efficient method to obtain better object detectors by searching for the feature pyramid network (FPN) and the prediction head of a simple anchor-free object detector. With carefully designed search space, search algorithms, and strategies for evaluating network quality, we are able to find top-performing detection architectures within 4 days using 8 V100 GPUs.
  arXiv  Detail & Related papers  (2021-10-24T12:20:04Z)
• Going Beyond Neural Architecture Search with Sampling-based Neural Ensemble Search [31.059040393415003]
  We present two novel sampling algorithms under our Neural Ensemble Search via Sampling (NESS) framework. Our NESS algorithms are shown to be able to achieve improved performance in both classification and adversarial defense tasks.
  arXiv  Detail & Related papers  (2021-09-06T15:18:37Z)
• Neural Architecture Search of SPD Manifold Networks [79.45110063435617]
  We propose a new neural architecture search (NAS) problem of Symmetric Positive Definite (SPD) manifold networks. We first introduce a geometrically rich and diverse SPD neural architecture search space for an efficient SPD cell design. We exploit a differentiable NAS algorithm on our relaxed continuous search space for SPD neural architecture search.
  arXiv  Detail & Related papers  (2020-10-27T18:08:57Z)
• Simplifying Architecture Search for Graph Neural Network [38.45540097927176]
  We propose SNAG framework, consisting of a novel search space and a reinforcement learning based search algorithm. Experiments on real-world datasets demonstrate the effectiveness of SNAG framework compared to human-designed GNNs and NAS methods.
  arXiv  Detail & Related papers  (2020-08-26T16:24:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.