S3NAS: Fast NPU-aware Neural Architecture Search Methodology

• URL: http://arxiv.org/abs/2009.02009v1
• Date: Fri, 4 Sep 2020 04:45:50 GMT
• Title: S3NAS: Fast NPU-aware Neural Architecture Search Methodology
• Authors: Jaeseong Lee, Duseok Kang and Soonhoi Ha
• Abstract summary: We present a fast NPU-aware NAS methodology, called S3NAS, to find a CNN architecture with higher accuracy than the existing ones. We are able to find a network in 3 hours using TPUv3, which shows 82.72% top-1 accuracy on ImageNet with 11.66 ms latency.
• Score: 2.607400740040335
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As the application area of convolutional neural networks (CNN) is growing in embedded devices, it becomes popular to use a hardware CNN accelerator, called neural processing unit (NPU), to achieve higher performance per watt than CPUs or GPUs. Recently, automated neural architecture search (NAS) emerges as the default technique to find a state-of-the-art CNN architecture with higher accuracy than manually-designed architectures for image classification. In this paper, we present a fast NPU-aware NAS methodology, called S3NAS, to find a CNN architecture with higher accuracy than the existing ones under a given latency constraint. It consists of three steps: supernet design, Single-Path NAS for fast architecture exploration, and scaling. To widen the search space of the supernet structure that consists of stages, we allow stages to have a different number of blocks and blocks to have parallel layers of different kernel sizes. For a fast neural architecture search, we apply a modified Single-Path NAS technique to the proposed supernet structure. In this step, we assume a shorter latency constraint than the required to reduce the search space and the search time. The last step is to scale up the network maximally within the latency constraint. For accurate latency estimation, an analytical latency estimator is devised, based on a cycle-level NPU simulator that runs an entire CNN considering the memory access overhead accurately. With the proposed methodology, we are able to find a network in 3 hours using TPUv3, which shows 82.72% top-1 accuracy on ImageNet with 11.66 ms latency. Code are released at https://github.com/cap-lab/S3NAS

Related papers

• RATs-NAS: Redirection of Adjacent Trails on GCN for Neural Architecture Search [6.117917355232904]
  We propose the Redirected Adjacent Trails NAS (RATs-NAS) to quickly search for the desired neural network architecture. RATs-NAS consists of two components: the Redirected Adjacent Trails GCN (RATs-GCN) and the Predictor-based Search Space Sampling (P3S) module.
  arXiv  Detail & Related papers  (2023-05-07T07:13:33Z)
• When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search [117.89827740405694]
  Key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures. TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37% in four GPU hours in NAS-Bench-201.
  arXiv  Detail & Related papers  (2022-04-11T07:34:21Z)
• HyperSegNAS: Bridging One-Shot Neural Architecture Search with 3D Medical Image Segmentation using HyperNet [51.60655410423093]
  We introduce HyperSegNAS to enable one-shot Neural Architecture Search (NAS) for medical image segmentation. We show that HyperSegNAS yields better performing and more intuitive architectures compared to the previous state-of-the-art (SOTA) segmentation networks. Our method is evaluated on public datasets from the Medical Decathlon (MSD) challenge, and achieves SOTA performances.
  arXiv  Detail & Related papers  (2021-12-20T16:21:09Z)
• FLASH: Fast Neural Architecture Search with Hardware Optimization [7.263481020106725]
  Neural architecture search (NAS) is a promising technique to design efficient and high-performance deep neural networks (DNNs) This paper proposes FLASH, a very fast NAS methodology that co-optimizes the DNN accuracy and performance on a real hardware platform.
  arXiv  Detail & Related papers  (2021-08-01T23:46:48Z)
• Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective [88.39981851247727]
  We propose a novel framework called training-free neural architecture search (TE-NAS) TE-NAS ranks architectures by analyzing the spectrum of the neural tangent kernel (NTK) and the number of linear regions in the input space. We show that: (1) these two measurements imply the trainability and expressivity of a neural network; (2) they strongly correlate with the network's test accuracy.
  arXiv  Detail & Related papers  (2021-02-23T07:50:44Z)
• LC-NAS: Latency Constrained Neural Architecture Search for Point Cloud Networks [73.78551758828294]
  LC-NAS is able to find state-of-the-art architectures for point cloud classification with minimal computational cost. We show how our searched architectures achieve any desired latency with a reasonably low drop in accuracy.
  arXiv  Detail & Related papers  (2020-08-24T10:30:21Z)
• TF-NAS: Rethinking Three Search Freedoms of Latency-Constrained Differentiable Neural Architecture Search [85.96350089047398]
  We propose Three-Freedom NAS (TF-NAS) to achieve both good classification accuracy and precise latency constraint. Experiments on ImageNet demonstrate the effectiveness of TF-NAS. Particularly, our searched TF-NAS-A obtains 76.9% top-1 accuracy, achieving state-of-the-art results with less latency.
  arXiv  Detail & Related papers  (2020-08-12T13:44:20Z)
• HourNAS: Extremely Fast Neural Architecture Search Through an Hourglass Lens [125.39301622207674]
  Neural Architecture Search (NAS) refers to automatically design the architecture. We propose an hourglass-inspired approach (HourNAS) for this problem that is motivated by the fact that the effects of the architecture often proceed from the vital few blocks. Experimental results on the ImageNet show that only using 3 hours (0.1 days) with one GPU, our HourNAS can search an architecture that achieves a 77.0% Top-1 accuracy.
  arXiv  Detail & Related papers  (2020-05-29T08:35:32Z)
• Fast Neural Network Adaptation via Parameter Remapping and Architecture Search [35.61441231491448]
  Deep neural networks achieve remarkable performance in many computer vision tasks. Most state-of-the-art (SOTA) semantic segmentation and object detection approaches reuse neural network architectures designed for image classification as the backbone. One major challenge though, is that ImageNet pre-training of the search space representation incurs huge computational cost. In this paper, we propose a Fast Neural Network Adaptation (FNA) method, which can adapt both the architecture and parameters of a seed network.
  arXiv  Detail & Related papers  (2020-01-08T13:45:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.