GRASP-GCN: Graph-Shape Prioritization for Neural Architecture Search under Distribution Shifts

• URL: http://arxiv.org/abs/2405.06994v1
• Date: Sat, 11 May 2024 12:02:24 GMT
• Title: GRASP-GCN: Graph-Shape Prioritization for Neural Architecture Search under Distribution Shifts
• Authors: Sofia Casarin, Oswald Lanz, Sergio Escalera,
• Abstract summary: We propose a simple and efficient way of improving prediction performance when dealing with data distribution shifts. We exploit the Kronecker-product on the randomly wired search-space and create a small NAS benchmark composed of networks trained over four different datasets. To improve the generalization abilities, we propose GRASP-GCN, a ranking Graph Convolutional Network that takes as additional input the shape of the layers of the neural networks.
• Score: 39.19675815138566
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Neural Architecture Search (NAS) methods have shown to output networks that largely outperform human-designed networks. However, conventional NAS methods have mostly tackled the single dataset scenario, incuring in a large computational cost as the procedure has to be run from scratch for every new dataset. In this work, we focus on predictor-based algorithms and propose a simple and efficient way of improving their prediction performance when dealing with data distribution shifts. We exploit the Kronecker-product on the randomly wired search-space and create a small NAS benchmark composed of networks trained over four different datasets. To improve the generalization abilities, we propose GRASP-GCN, a ranking Graph Convolutional Network that takes as additional input the shape of the layers of the neural networks. GRASP-GCN is trained with the not-at-convergence accuracies, and improves the state-of-the-art of 3.3 % for Cifar-10 and increasing moreover the generalization abilities under data distribution shift.

Related papers

• FR-NAS: Forward-and-Reverse Graph Predictor for Efficient Neural Architecture Search [10.699485270006601]
  We introduce a novel Graph Neural Networks (GNN) predictor for Neural Architecture Search (NAS) This predictor renders neural architectures into vector representations by combining both the conventional and inverse graph views. The experimental results showcase a significant improvement in prediction accuracy, with a 3%--16% increase in Kendall-tau correlation.
  arXiv  Detail & Related papers  (2024-04-24T03:22:49Z)
• Efficient Heterogeneous Graph Learning via Random Projection [58.4138636866903]
  Heterogeneous Graph Neural Networks (HGNNs) are powerful tools for deep learning on heterogeneous graphs. Recent pre-computation-based HGNNs use one-time message passing to transform a heterogeneous graph into regular-shaped tensors. We propose a hybrid pre-computation-based HGNN, named Random Projection Heterogeneous Graph Neural Network (RpHGNN)
  arXiv  Detail & Related papers  (2023-10-23T01:25:44Z)
• Scalable Neural Network Training over Distributed Graphs [45.151244961817454]
  Realworld graph data must often be stored across many machines just because capacity constraints. Network communication is costly and becomes main bottleneck to train GNNs. First framework that can be used to train GNNs at all network decentralization levels.
  arXiv  Detail & Related papers  (2023-02-25T10:42:34Z)
• EGRC-Net: Embedding-induced Graph Refinement Clustering Network [66.44293190793294]
  We propose a novel graph clustering network called Embedding-Induced Graph Refinement Clustering Network (EGRC-Net) EGRC-Net effectively utilizes the learned embedding to adaptively refine the initial graph and enhance the clustering performance. Our proposed methods consistently outperform several state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-19T09:08:43Z)
• CCasGNN: Collaborative Cascade Prediction Based on Graph Neural Networks [0.49269463638915806]
  Cascade prediction aims at modeling information diffusion in the network. Recent efforts devoted to combining network structure and sequence features by graph neural networks and recurrent neural networks. We propose a novel method CCasGNN considering the individual profile, structural features, and sequence information.
  arXiv  Detail & Related papers  (2021-12-07T11:37:36Z)
• Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognition [126.51241919472356]
  We design a simple and highly modularized graph convolutional network architecture for skeleton-based action recognition. Our network is constructed by repeating a building block that aggregates multi-granularity information from both the spatial and temporal paths.
  arXiv  Detail & Related papers  (2020-11-26T14:43:04Z)
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training [62.73470368851127]
  Graph representation learning has emerged as a powerful technique for addressing real-world problems. We design Graph Contrastive Coding -- a self-supervised graph neural network pre-training framework. We conduct experiments on three graph learning tasks and ten graph datasets.
  arXiv  Detail & Related papers  (2020-06-17T16:18:35Z)
• Self-supervised Training of Graph Convolutional Networks [39.80867112204255]
  Graph Convolutional Networks (GCNs) have been successfully applied to analyze non-grid data. In this paper, we propose two types of self-supervised learning strategies to exploit available information from the input graph structure data itself.
  arXiv  Detail & Related papers  (2020-06-03T16:53:37Z)
• Large-Scale Gradient-Free Deep Learning with Recursive Local Representation Alignment [84.57874289554839]
  Training deep neural networks on large-scale datasets requires significant hardware resources. Backpropagation, the workhorse for training these networks, is an inherently sequential process that is difficult to parallelize. We propose a neuro-biologically-plausible alternative to backprop that can be used to train deep networks.
  arXiv  Detail & Related papers  (2020-02-10T16:20:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.