NAAS: Neural Accelerator Architecture Search

• URL: http://arxiv.org/abs/2105.13258v1
• Date: Thu, 27 May 2021 15:56:41 GMT
• Title: NAAS: Neural Accelerator Architecture Search
• Authors: Yujun Lin, Mengtian Yang and Song Han
• Abstract summary: We propose Neural Accelerator Architecture Search (NAAS) to holistically search the neural network architecture, accelerator architecture, and compiler mappings. As a data-driven approach, NAAS rivals the human design Eyeriss by 4.4x EDP reduction with 2.7% accuracy improvement on ImageNet.
• Score: 16.934625310654553
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data-driven, automatic design space exploration of neural accelerator architecture is desirable for specialization and productivity. Previous frameworks focus on sizing the numerical architectural hyper-parameters while neglect searching the PE connectivities and compiler mappings. To tackle this challenge, we propose Neural Accelerator Architecture Search (NAAS) which holistically searches the neural network architecture, accelerator architecture, and compiler mapping in one optimization loop. NAAS composes highly matched architectures together with efficient mapping. As a data-driven approach, NAAS rivals the human design Eyeriss by 4.4x EDP reduction with 2.7% accuracy improvement on ImageNet under the same computation resource, and offers 1.4x to 3.5x EDP reduction than only sizing the architectural hyper-parameters.

Related papers

• EM-DARTS: Hierarchical Differentiable Architecture Search for Eye Movement Recognition [54.99121380536659]
  Eye movement biometrics have received increasing attention thanks to its high secure identification. Deep learning (DL) models have been recently successfully applied for eye movement recognition. DL architecture still is determined by human prior knowledge. We propose EM-DARTS, a hierarchical differentiable architecture search algorithm to automatically design the DL architecture for eye movement recognition.
  arXiv  Detail & Related papers  (2024-09-22T13:11:08Z)
• Neural Architecture Search for Speech Emotion Recognition [72.1966266171951]
  We propose to apply neural architecture search (NAS) techniques to automatically configure the SER models. We show that NAS can improve SER performance (54.89% to 56.28%) while maintaining model parameter sizes.
  arXiv  Detail & Related papers  (2022-03-31T10:16:10Z)
• iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients [75.41173109807735]
  Differentiable ARchiTecture Search (DARTS) has recently become the mainstream of neural architecture search (NAS) We tackle the hypergradient computation in DARTS based on the implicit function theorem. We show that the architecture optimisation with the proposed method, named iDARTS, is expected to converge to a stationary point.
  arXiv  Detail & Related papers  (2021-06-21T00:44:11Z)
• Evolving Neural Architecture Using One Shot Model [5.188825486231326]
  We propose a novel way of applying a simple genetic algorithm to the NAS problem called EvNAS (Evolving Neural Architecture using One Shot Model) EvNAS searches for the architecture on the proxy dataset i.e. CIFAR-10 for 4.4 GPU day on a single GPU and achieves top-1 test error of 2.47%. Results show the potential of evolutionary methods in solving the architecture search problem.
  arXiv  Detail & Related papers  (2020-12-23T08:40:53Z)
• Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search [50.40004966087121]
  We introduce a new reinforcement learning based neural architecture search (NAS) methodology for generative adversarial network (GAN) architecture search. The key idea is to formulate the GAN architecture search problem as a Markov decision process (MDP) for smoother architecture sampling. We exploit an off-policy GAN architecture search algorithm that makes efficient use of the samples generated by previous policies.
  arXiv  Detail & Related papers  (2020-07-17T18:29:17Z)
• Multi-Objective Neural Architecture Search Based on Diverse Structures and Adaptive Recommendation [4.595675084986132]
  The search space of neural architecture search (NAS) for convolutional neural network (CNN) is huge. We propose MoARR algorithm, which utilizes the existing research results and historical information to quickly find architectures that are both lightweight and accurate. Experimental results show that our MoARR can achieve a powerful and lightweight model (with 1.9% error rate and 2.3M parameters) on CIFAR-10 in 6 GPU hours.
  arXiv  Detail & Related papers  (2020-07-06T13:42:33Z)
• Neural Architecture Optimization with Graph VAE [21.126140965779534]
  We propose an efficient NAS approach to optimize network architectures in a continuous space. The framework jointly learns four components: the encoder, the performance predictor, the complexity predictor and the decoder.
  arXiv  Detail & Related papers  (2020-06-18T07:05:48Z)
• FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining [65.39532971991778]
  We present an accuracy predictor that scores architecture and training recipes jointly, guiding both sample selection and ranking. We run fast evolutionary searches in just CPU minutes to generate architecture-recipe pairs for a variety of resource constraints. FBNetV3 makes up a family of state-of-the-art compact neural networks that outperform both automatically and manually-designed competitors.
  arXiv  Detail & Related papers  (2020-06-03T05:20:21Z)
• A Semi-Supervised Assessor of Neural Architectures [157.76189339451565]
  We employ an auto-encoder to discover meaningful representations of neural architectures. A graph convolutional neural network is introduced to predict the performance of architectures.
  arXiv  Detail & Related papers  (2020-05-14T09:02:33Z)
• ADWPNAS: Architecture-Driven Weight Prediction for Neural Architecture Search [6.458169480971417]
  We propose an Architecture-Driven Weight Prediction (ADWP) approach for neural architecture search (NAS) In our approach, we first design an architecture-intensive search space and then train a HyperNetwork by inputting encoding architecture parameters. Results show that one search procedure can be completed in 4.0 GPU hours on CIFAR-10.
  arXiv  Detail & Related papers  (2020-03-03T05:06:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.