NASCaps: A Framework for Neural Architecture Search to Optimize the
Accuracy and Hardware Efficiency of Convolutional Capsule Networks
- URL: http://arxiv.org/abs/2008.08476v1
- Date: Wed, 19 Aug 2020 14:29:36 GMT
- Title: NASCaps: A Framework for Neural Architecture Search to Optimize the
Accuracy and Hardware Efficiency of Convolutional Capsule Networks
- Authors: Alberto Marchisio, Andrea Massa, Vojtech Mrazek, Beatrice Bussolino,
Maurizio Martina, Muhammad Shafique
- Abstract summary: We propose NASCaps, an automated framework for the hardware-aware NAS of different types of Deep Neural Networks (DNNs)
We study the efficacy of deploying a multi-objective Genetic Algorithm (e.g., based on the NSGA-II algorithm)
Our framework is the first to model and supports the specialized capsule layers and dynamic routing in the NAS-flow.
- Score: 10.946374356026679
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep Neural Networks (DNNs) have made significant improvements to reach the
desired accuracy to be employed in a wide variety of Machine Learning (ML)
applications. Recently the Google Brain's team demonstrated the ability of
Capsule Networks (CapsNets) to encode and learn spatial correlations between
different input features, thereby obtaining superior learning capabilities
compared to traditional (i.e., non-capsule based) DNNs. However, designing
CapsNets using conventional methods is a tedious job and incurs significant
training effort. Recent studies have shown that powerful methods to
automatically select the best/optimal DNN model configuration for a given set
of applications and a training dataset are based on the Neural Architecture
Search (NAS) algorithms. Moreover, due to their extreme computational and
memory requirements, DNNs are employed using the specialized hardware
accelerators in IoT-Edge/CPS devices. In this paper, we propose NASCaps, an
automated framework for the hardware-aware NAS of different types of DNNs,
covering both traditional convolutional DNNs and CapsNets. We study the
efficacy of deploying a multi-objective Genetic Algorithm (e.g., based on the
NSGA-II algorithm). The proposed framework can jointly optimize the network
accuracy and the corresponding hardware efficiency, expressed in terms of
energy, memory, and latency of a given hardware accelerator executing the DNN
inference. Besides supporting the traditional DNN layers, our framework is the
first to model and supports the specialized capsule layers and dynamic routing
in the NAS-flow. We evaluate our framework on different datasets, generating
different network configurations, and demonstrate the tradeoffs between the
different output metrics. We will open-source the complete framework and
configurations of the Pareto-optimal architectures at
https://github.com/ehw-fit/nascaps.
Related papers
- RNC: Efficient RRAM-aware NAS and Compilation for DNNs on Resource-Constrained Edge Devices [0.30458577208819987]
We aim to develop edge-friendly deep neural networks (DNNs) for accelerators based on resistive random-access memory (RRAM)
We propose an edge compilation and resource-constrained RRAM-aware neural architecture search (NAS) framework to search for optimized neural networks meeting specific hardware constraints.
The resulting model from NAS optimized for speed achieved 5x-30x speedup.
arXiv Detail & Related papers (2024-09-27T15:35:36Z) - Optimizing Convolutional Neural Network Architecture [0.0]
Convolutional Neural Networks (CNN) are widely used to face challenging tasks like speech recognition, natural language processing or computer vision.
We propose Optimizing Convolutional Neural Network Architecture (OCNNA), a novel CNN optimization and construction method based on pruning and knowledge distillation.
Our method has been compared with more than 20 convolutional neural network simplification algorithms obtaining outstanding results.
arXiv Detail & Related papers (2023-12-17T12:23:11Z) - DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit
CNNs [53.82853297675979]
1-bit convolutional neural networks (CNNs) with binary weights and activations show their potential for resource-limited embedded devices.
One natural approach is to use 1-bit CNNs to reduce the computation and memory cost of NAS.
We introduce Discrepant Child-Parent Neural Architecture Search (DCP-NAS) to efficiently search 1-bit CNNs.
arXiv Detail & Related papers (2023-06-27T11:28:29Z) - HQNAS: Auto CNN deployment framework for joint quantization and
architecture search [30.45926484863791]
We propose a novel neural network design framework called Hardware-aware Quantized Neural Architecture Search(HQNAS)
It takes only 4 GPU hours to discover an outstanding NN policy on CIFAR10.
It also takes only %10 GPU time to generate a comparable model on Imagenet.
arXiv Detail & Related papers (2022-10-16T08:32:18Z) - Neural Attentive Circuits [93.95502541529115]
We introduce a general purpose, yet modular neural architecture called Neural Attentive Circuits (NACs)
NACs learn the parameterization and a sparse connectivity of neural modules without using domain knowledge.
NACs achieve an 8x speedup at inference time while losing less than 3% performance.
arXiv Detail & Related papers (2022-10-14T18:00:07Z) - RoHNAS: A Neural Architecture Search Framework with Conjoint
Optimization for Adversarial Robustness and Hardware Efficiency of
Convolutional and Capsule Networks [10.946374356026679]
RoHNAS is a novel framework that jointly optimize for adversarial-robustness and hardware-efficiency of Deep Neural Network (DNN)
For reducing the exploration time, RoHNAS analyzes and selects appropriate values of adversarial perturbation for each dataset to employ in the NAS flow.
arXiv Detail & Related papers (2022-10-11T09:14:56Z) - Comparison Analysis of Traditional Machine Learning and Deep Learning
Techniques for Data and Image Classification [62.997667081978825]
The purpose of the study is to analyse and compare the most common machine learning and deep learning techniques used for computer vision 2D object classification tasks.
Firstly, we will present the theoretical background of the Bag of Visual words model and Deep Convolutional Neural Networks (DCNN)
Secondly, we will implement a Bag of Visual Words model, the VGG16 CNN Architecture.
arXiv Detail & Related papers (2022-04-11T11:34:43Z) - U-Boost NAS: Utilization-Boosted Differentiable Neural Architecture
Search [50.33956216274694]
optimizing resource utilization in target platforms is key to achieving high performance during DNN inference.
We propose a novel hardware-aware NAS framework that does not only optimize for task accuracy and inference latency, but also for resource utilization.
We achieve 2.8 - 4x speedup for DNN inference compared to prior hardware-aware NAS methods.
arXiv Detail & Related papers (2022-03-23T13:44:15Z) - Evolutionary Neural Cascade Search across Supernetworks [68.8204255655161]
We introduce ENCAS - Evolutionary Neural Cascade Search.
ENCAS can be used to search over multiple pretrained supernetworks.
We test ENCAS on common computer vision benchmarks.
arXiv Detail & Related papers (2022-03-08T11:06:01Z) - FLASH: Fast Neural Architecture Search with Hardware Optimization [7.263481020106725]
Neural architecture search (NAS) is a promising technique to design efficient and high-performance deep neural networks (DNNs)
This paper proposes FLASH, a very fast NAS methodology that co-optimizes the DNN accuracy and performance on a real hardware platform.
arXiv Detail & Related papers (2021-08-01T23:46:48Z) - MS-RANAS: Multi-Scale Resource-Aware Neural Architecture Search [94.80212602202518]
We propose Multi-Scale Resource-Aware Neural Architecture Search (MS-RANAS)
We employ a one-shot architecture search approach in order to obtain a reduced search cost.
We achieve state-of-the-art results in terms of accuracy-speed trade-off.
arXiv Detail & Related papers (2020-09-29T11:56:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.