SuperNet in Neural Architecture Search: A Taxonomic Survey
- URL: http://arxiv.org/abs/2204.03916v1
- Date: Fri, 8 Apr 2022 08:29:52 GMT
- Title: SuperNet in Neural Architecture Search: A Taxonomic Survey
- Authors: Stephen Cha, Taehyeon Kim, Hayeon Lee, Se-Young Yun
- Abstract summary: This survey focuses on the supernet optimization that builds a neural network that assembles all the architectures as its sub models by using weight sharing.
We aim to accomplish that by proposing them as solutions to the common challenges found in the literature: data-side optimization, poor rank correlation alleviation, and transferable NAS for a number of deployment scenarios.
- Score: 14.037182039950505
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep Neural Networks (DNN) have made significant progress in a wide range of
visual recognition tasks such as image classification, object detection, and
semantic segmentation. The evolution of convolutional architectures has led to
better performance by incurring expensive computational costs. In addition,
network design has become a difficult task, which is labor-intensive and
requires a high level of domain knowledge. To mitigate such issues, there have
been studies for a variety of neural architecture search methods that
automatically search for optimal architectures, achieving models with
impressive performance that outperform human-designed counterparts. This survey
aims to provide an overview of existing works in this field of research and
specifically focus on the supernet optimization that builds a neural network
that assembles all the architectures as its sub models by using weight sharing.
We aim to accomplish that by categorizing supernet optimization by proposing
them as solutions to the common challenges found in the literature: data-side
optimization, poor rank correlation alleviation, and transferable NAS for a
number of deployment scenarios.
Related papers
- EM-DARTS: Hierarchical Differentiable Architecture Search for Eye Movement Recognition [54.99121380536659]
Eye movement biometrics have received increasing attention thanks to its high secure identification.
Deep learning (DL) models have been recently successfully applied for eye movement recognition.
DL architecture still is determined by human prior knowledge.
We propose EM-DARTS, a hierarchical differentiable architecture search algorithm to automatically design the DL architecture for eye movement recognition.
arXiv Detail & Related papers (2024-09-22T13:11:08Z) - GeNAS: Neural Architecture Search with Better Generalization [14.92869716323226]
Recent neural architecture search (NAS) approaches rely on validation loss or accuracy to find the superior network for the target data.
In this paper, we investigate a new neural architecture search measure for excavating architectures with better generalization.
arXiv Detail & Related papers (2023-05-15T12:44:54Z) - DQNAS: Neural Architecture Search using Reinforcement Learning [6.33280703577189]
Convolutional Neural Networks have been used in a variety of image related applications.
In this paper, we propose an automated Neural Architecture Search framework, guided by the principles of Reinforcement Learning.
arXiv Detail & Related papers (2023-01-17T04:01:47Z) - Search Space Adaptation for Differentiable Neural Architecture Search in
Image Classification [15.641353388251465]
Differentiable neural architecture search (NAS) has a great impact by reducing the search cost to the level of training a single network.
In this paper, we propose an adaptation scheme of the search space by introducing a search scope.
The effectiveness of proposed method is demonstrated with ProxylessNAS for the image classification task.
arXiv Detail & Related papers (2022-06-05T05:27:12Z) - Neural Architecture Search for Dense Prediction Tasks in Computer Vision [74.9839082859151]
Deep learning has led to a rising demand for neural network architecture engineering.
neural architecture search (NAS) aims at automatically designing neural network architectures in a data-driven manner rather than manually.
NAS has become applicable to a much wider range of problems in computer vision.
arXiv Detail & Related papers (2022-02-15T08:06:50Z) - Learning Interpretable Models Through Multi-Objective Neural
Architecture Search [0.9990687944474739]
We propose a framework to optimize for both task performance and "introspectability," a surrogate metric for aspects of interpretability.
We demonstrate that jointly optimizing for task error and introspectability leads to more disentangled and debuggable architectures that perform within error.
arXiv Detail & Related papers (2021-12-16T05:50:55Z) - Redefining Neural Architecture Search of Heterogeneous Multi-Network
Models by Characterizing Variation Operators and Model Components [71.03032589756434]
We investigate the effect of different variation operators in a complex domain, that of multi-network heterogeneous neural models.
We characterize both the variation operators, according to their effect on the complexity and performance of the model; and the models, relying on diverse metrics which estimate the quality of the different parts composing it.
arXiv Detail & Related papers (2021-06-16T17:12:26Z) - NAS-Navigator: Visual Steering for Explainable One-Shot Deep Neural
Network Synthesis [53.106414896248246]
We present a framework that allows analysts to effectively build the solution sub-graph space and guide the network search by injecting their domain knowledge.
Applying this technique in an iterative manner allows analysts to converge to the best performing neural network architecture for a given application.
arXiv Detail & Related papers (2020-09-28T01:48:45Z) - A Semi-Supervised Assessor of Neural Architectures [157.76189339451565]
We employ an auto-encoder to discover meaningful representations of neural architectures.
A graph convolutional neural network is introduced to predict the performance of architectures.
arXiv Detail & Related papers (2020-05-14T09:02:33Z) - Stage-Wise Neural Architecture Search [65.03109178056937]
Modern convolutional networks such as ResNet and NASNet have achieved state-of-the-art results in many computer vision applications.
These networks consist of stages, which are sets of layers that operate on representations in the same resolution.
It has been demonstrated that increasing the number of layers in each stage improves the prediction ability of the network.
However, the resulting architecture becomes computationally expensive in terms of floating point operations, memory requirements and inference time.
arXiv Detail & Related papers (2020-04-23T14:16:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.