Graph is all you need? Lightweight data-agnostic neural architecture search without training

• URL: http://arxiv.org/abs/2405.01306v1
• Date: Thu, 2 May 2024 14:12:58 GMT
• Title: Graph is all you need? Lightweight data-agnostic neural architecture search without training
• Authors: Zhenhan Huang, Tejaswini Pedapati, Pin-Yu Chen, Chunhen Jiang, Jianxi Gao,
• Abstract summary: Neural architecture search (NAS) enables the automatic design of neural network models. Our method, dubbed nasgraph, remarkably reduces the computational costs by converting neural architectures to graphs. It can find the best architecture among 200 randomly sampled architectures from NAS-Bench201 in 217 CPU seconds.
• Score: 45.79667238486864
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural architecture search (NAS) enables the automatic design of neural network models. However, training the candidates generated by the search algorithm for performance evaluation incurs considerable computational overhead. Our method, dubbed nasgraph, remarkably reduces the computational costs by converting neural architectures to graphs and using the average degree, a graph measure, as the proxy in lieu of the evaluation metric. Our training-free NAS method is data-agnostic and light-weight. It can find the best architecture among 200 randomly sampled architectures from NAS-Bench201 in 217 CPU seconds. Besides, our method is able to achieve competitive performance on various datasets including NASBench-101, NASBench-201, and NDS search spaces. We also demonstrate that nasgraph generalizes to more challenging tasks on Micro TransNAS-Bench-101.

Related papers

• DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions [121.05720140641189]
  We develop a family of models with the distilling neural architecture (DNA) techniques. Our proposed DNA models can rate all architecture candidates, as opposed to previous works that can only access a sub- search space using algorithms. Our models achieve state-of-the-art top-1 accuracy of 78.9% and 83.6% on ImageNet for a mobile convolutional network and a small vision transformer, respectively.
  arXiv  Detail & Related papers  (2024-03-02T22:16:47Z)
• GeNAS: Neural Architecture Search with Better Generalization [14.92869716323226]
  Recent neural architecture search (NAS) approaches rely on validation loss or accuracy to find the superior network for the target data. In this paper, we investigate a new neural architecture search measure for excavating architectures with better generalization.
  arXiv  Detail & Related papers  (2023-05-15T12:44:54Z)
• Neural Architecture Search via Two Constant Shared Weights Initialisations [0.0]
  We present a zero-cost metric that highly correlated with the train set accuracy across the NAS-Bench-101, NAS-Bench-201 and NAS-Bench-NLP benchmark datasets. Our method is easy to integrate within existing NAS algorithms and takes a fraction of a second to evaluate a single network.
  arXiv  Detail & Related papers  (2023-02-09T02:25:38Z)
• GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Models [48.57083463364353]
  We study neural architecture search (NAS) through the lens of learning random graph models. We propose GraphPNAS a deep graph generative model that learns a distribution of well-performing architectures. We show that our proposed graph generator consistently outperforms RNN-based one and achieves better or comparable performances than state-of-the-art NAS methods.
  arXiv  Detail & Related papers  (2022-11-28T09:09:06Z)
• NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search [55.75621026447599]
  We propose NAS-Bench-Graph, a tailored benchmark that supports unified, reproducible, and efficient evaluations for GraphNAS. Specifically, we construct a unified, expressive yet compact search space, covering 26,206 unique graph neural network (GNN) architectures. Based on our proposed benchmark, the performance of GNN architectures can be directly obtained by a look-up table without any further computation.
  arXiv  Detail & Related papers  (2022-06-18T10:17:15Z)
• UnrealNAS: Can We Search Neural Architectures with Unreal Data? [84.78460976605425]
  Neural architecture search (NAS) has shown great success in the automatic design of deep neural networks (DNNs) Previous work has analyzed the necessity of having ground-truth labels in NAS and inspired broad interest. We take a further step to question whether real data is necessary for NAS to be effective.
  arXiv  Detail & Related papers  (2022-05-04T16:30:26Z)
• Generic Neural Architecture Search via Regression [27.78105839644199]
  We propose a novel and generic neural architecture search (NAS) framework, termed Generic NAS (GenNAS) GenNAS does not use task-specific labels but instead adopts textitregression on a set of manually designed synthetic signal bases for architecture evaluation. We then propose an automatic task search to optimize the combination of synthetic signals using limited downstream-task-specific labels.
  arXiv  Detail & Related papers  (2021-08-04T08:21:12Z)
• Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective [88.39981851247727]
  We propose a novel framework called training-free neural architecture search (TE-NAS) TE-NAS ranks architectures by analyzing the spectrum of the neural tangent kernel (NTK) and the number of linear regions in the input space. We show that: (1) these two measurements imply the trainability and expressivity of a neural network; (2) they strongly correlate with the network's test accuracy.
  arXiv  Detail & Related papers  (2021-02-23T07:50:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.