VEGA: Towards an End-to-End Configurable AutoML Pipeline

• URL: http://arxiv.org/abs/2011.01507v4
• Date: Thu, 26 Nov 2020 05:28:50 GMT
• Title: VEGA: Towards an End-to-End Configurable AutoML Pipeline
• Authors: Bochao Wang, Hang Xu, Jiajin Zhang, Chen Chen, Xiaozhi Fang, Yixing Xu, Ning Kang, Lanqing Hong, Chenhan Jiang, Xinyue Cai, Jiawei Li, Fengwei Zhou, Yong Li, Zhicheng Liu, Xinghao Chen, Kai Han, Han Shu, Dehua Song, Yunhe Wang, Wei Zhang, Chunjing Xu, Zhenguo Li, Wenzhi Liu, Tong Zhang
• Abstract summary: VEGA is an efficient and comprehensive AutoML framework that is compatible and optimized for multiple hardware platforms. VEGA can improve the existing AutoML algorithms and discover new high-performance models against SOTA methods.
• Score: 101.07003005736719
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated Machine Learning (AutoML) is an important industrial solution for automatic discovery and deployment of the machine learning models. However, designing an integrated AutoML system faces four great challenges of configurability, scalability, integrability, and platform diversity. In this work, we present VEGA, an efficient and comprehensive AutoML framework that is compatible and optimized for multiple hardware platforms. a) The VEGA pipeline integrates various modules of AutoML, including Neural Architecture Search (NAS), Hyperparameter Optimization (HPO), Auto Data Augmentation, Model Compression, and Fully Train. b) To support a variety of search algorithms and tasks, we design a novel fine-grained search space and its description language to enable easy adaptation to different search algorithms and tasks. c) We abstract the common components of deep learning frameworks into a unified interface. VEGA can be executed with multiple back-ends and hardwares. Extensive benchmark experiments on multiple tasks demonstrate that VEGA can improve the existing AutoML algorithms and discover new high-performance models against SOTA methods, e.g. the searched DNet model zoo for Ascend 10x faster than EfficientNet-B5 and 9.2x faster than RegNetX-32GF on ImageNet. VEGA is open-sourced at https://github.com/huawei-noah/vega.

Related papers

• Multi-objective Differentiable Neural Architecture Search [58.67218773054753]
  We propose a novel NAS algorithm that encodes user preferences for the trade-off between performance and hardware metrics. Our method outperforms existing MOO NAS methods across a broad range of qualitatively different search spaces and datasets.
  arXiv  Detail & Related papers  (2024-02-28T10:09:04Z)
• AutoTransfer: AutoML with Knowledge Transfer -- An Application to Graph Neural Networks [75.11008617118908]
  AutoML techniques consider each task independently from scratch, leading to high computational cost. Here we propose AutoTransfer, an AutoML solution that improves search efficiency by transferring the prior architectural design knowledge to the novel task of interest.
  arXiv  Detail & Related papers  (2023-03-14T07:23:16Z)
• GANDALF: Gated Adaptive Network for Deep Automated Learning of Features [0.0]
  Gated Adaptive Network for Deep Automated Learning of Features (GANDALF) GANDALF relies on a new tabular processing unit with a gating mechanism and in-built feature selection called Gated Feature Learning Unit (GFLU) We demonstrate that GANDALF outperforms or stays at-par with SOTA approaches like XGBoost, SAINT, FT-Transformers, etc.
  arXiv  Detail & Related papers  (2022-07-18T12:12:24Z)
• AutoCoMet: Smart Neural Architecture Search via Co-Regulated Shaping Reinforcement [5.026843258629663]
  AutoCoMet learns the most suitable deep model architecture optimized for varied types of device hardware and task contexts, 3x faster. Our novel co-regulated shaping reinforcement controller together with the high fidelity hardware meta-behavior predictor produces a smart, fast NAS framework.
  arXiv  Detail & Related papers  (2022-03-29T10:11:22Z)
• Pruning Self-attentions into Convolutional Layers in Single Path [89.55361659622305]
  Vision Transformers (ViTs) have achieved impressive performance over various computer vision tasks. We propose Single-Path Vision Transformer pruning (SPViT) to efficiently and automatically compress the pre-trained ViTs. Our SPViT can trim 52.0% FLOPs for DeiT-B and get an impressive 0.6% top-1 accuracy gain simultaneously.
  arXiv  Detail & Related papers  (2021-11-23T11:35:54Z)
• AutoGL: A Library for Automated Graph Learning [67.63587865669372]
  We present Automated Graph Learning (AutoGL), the first dedicated library for automated machine learning on graphs. AutoGL is open-source, easy to use, and flexible to be extended. We also present AutoGL-light, a lightweight version of AutoGL to facilitate customizing pipelines and enriching applications.
  arXiv  Detail & Related papers  (2021-04-11T10:49:23Z)
• Resource-Aware Pareto-Optimal Automated Machine Learning Platform [1.6746303554275583]
  novel platform Resource-Aware AutoML (RA-AutoML) RA-AutoML enables flexible and generalized algorithms to build machine learning models subjected to multiple objectives.
  arXiv  Detail & Related papers  (2020-10-30T19:37:48Z)
• Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL [53.40030379661183]
  Auto-PyTorch is a framework to enable fully automated deep learning (AutoDL) It combines multi-fidelity optimization with portfolio construction for warmstarting and ensembling of deep neural networks (DNNs) We show that Auto-PyTorch performs better than several state-of-the-art competitors on average.
  arXiv  Detail & Related papers  (2020-06-24T15:15:17Z)
• PHOTONAI -- A Python API for Rapid Machine Learning Model Development [2.414341608751139]
  PHOTONAI is a high-level Python API designed to simplify and accelerate machine learning model development. It functions as a unifying framework allowing the user to easily access and combine algorithms from different toolboxes into custom algorithm sequences.
  arXiv  Detail & Related papers  (2020-02-13T10:33:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.