Mining Robust Default Configurations for Resource-constrained AutoML

• URL: http://arxiv.org/abs/2202.09927v1
• Date: Sun, 20 Feb 2022 23:08:04 GMT
• Title: Mining Robust Default Configurations for Resource-constrained AutoML
• Authors: Moe Kayali and Chi Wang
• Abstract summary: We present a novel method of selecting performant configurations for a given task by performing offline autoML and mining over a diverse set of tasks. We show that our approach is effective for warm-starting existing autoML platforms.
• Score: 18.326426020906215
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic machine learning (AutoML) is a key enabler of the mass deployment of the next generation of machine learning systems. A key desideratum for future ML systems is the automatic selection of models and hyperparameters. We present a novel method of selecting performant configurations for a given task by performing offline autoML and mining over a diverse set of tasks. By mining the training tasks, we can select a compact portfolio of configurations that perform well over a wide variety of tasks, as well as learn a strategy to select portfolio configurations for yet-unseen tasks. The algorithm runs in a zero-shot manner, that is without training any models online except the chosen one. In a compute- or time-constrained setting, this virtually instant selection is highly performant. Further, we show that our approach is effective for warm-starting existing autoML platforms. In both settings, we demonstrate an improvement on the state-of-the-art by testing over 62 classification and regression datasets. We also demonstrate the utility of recommending data-dependent default configurations that outperform widely used hand-crafted defaults.

Related papers

• AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML [56.565200973244146]
  Automated machine learning (AutoML) accelerates AI development by automating tasks in the development pipeline. Recent works have started exploiting large language models (LLM) to lessen such burden. This paper proposes AutoML-Agent, a novel multi-agent framework tailored for full-pipeline AutoML.
  arXiv  Detail & Related papers  (2024-10-03T20:01:09Z)
• Towards Modular LLMs by Building and Reusing a Library of LoRAs [64.43376695346538]
  We study how to best build a library of adapters given multi-task data. We introduce model-based clustering, MBC, a method that groups tasks based on the similarity of their adapter parameters. To re-use the library, we present a novel zero-shot routing mechanism, Arrow, which enables dynamic selection of the most relevant adapters.
  arXiv  Detail & Related papers  (2024-05-18T03:02:23Z)
• AutoXPCR: Automated Multi-Objective Model Selection for Time Series Forecasting [1.0515439489916734]
  We propose AutoXPCR - a novel method for automated and explainable multi-objective model selection. Our approach leverages meta-learning to estimate any model's performance along PCR criteria, which encompass (P)redictive error, (C)omplexity, and (R)esource demand. Our method clearly outperforms other model selection approaches - on average, it only requires 20% of computation costs for recommending models with 90% of the best-possible quality.
  arXiv  Detail & Related papers  (2023-12-20T14:04:57Z)
• AutoMix: Automatically Mixing Language Models [62.51238143437967]
  Large language models (LLMs) are now available from cloud API providers in various sizes and configurations. We present Automix, an approach that strategically routes queries to larger LMs, based on the approximate correctness of outputs from a smaller LM.
  arXiv  Detail & Related papers  (2023-10-19T17:57:39Z)
• AutoML-GPT: Large Language Model for AutoML [5.9145212342776805]
  We have established a framework called AutoML-GPT that integrates a comprehensive set of tools and libraries. Through a conversational interface, users can specify their requirements, constraints, and evaluation metrics. We have demonstrated that AutoML-GPT significantly reduces the time and effort required for machine learning tasks.
  arXiv  Detail & Related papers  (2023-09-03T09:39:49Z)
• MILO: Model-Agnostic Subset Selection Framework for Efficient Model Training and Tuning [68.12870241637636]
  We propose MILO, a model-agnostic subset selection framework that decouples the subset selection from model training. Our empirical results indicate that MILO can train models $3times - 10 times$ faster and tune hyperparameters $20times - 75 times$ faster than full-dataset training or tuning without performance.
  arXiv  Detail & Related papers  (2023-01-30T20:59:30Z)
• Hyperparameter optimization in deep multi-target prediction [16.778802088570412]
  We offer a single AutoML framework for most problem settings that fall under the umbrella of multi-target prediction. Our work can be seen as the first attempt at offering a single AutoML framework for most problem settings that fall under the umbrella of multi-target prediction.
  arXiv  Detail & Related papers  (2022-11-08T16:33:36Z)
• AutoDES: AutoML Pipeline Generation of Classification with Dynamic Ensemble Strategy Selection [0.0]
  We present a novel framework for automated machine learning that incorporates advances in dynamic ensemble selection. Our approach is the first in the field of AutoML to search and optimize ensemble strategies. In comparison experiments, our method outperforms the state-of-the-art automated machine learning frameworks with the same CPU time.
  arXiv  Detail & Related papers  (2022-01-01T15:17:07Z)
• Learning Multi-Objective Curricula for Deep Reinforcement Learning [55.27879754113767]
  Various automatic curriculum learning (ACL) methods have been proposed to improve the sample efficiency and final performance of deep reinforcement learning (DRL) In this paper, we propose a unified automatic curriculum learning framework to create multi-objective but coherent curricula. In addition to existing hand-designed curricula paradigms, we further design a flexible memory mechanism to learn an abstract curriculum.
  arXiv  Detail & Related papers  (2021-10-06T19:30:25Z)
• Amazon SageMaker Automatic Model Tuning: Scalable Black-box Optimization [23.52446054521187]
  Amazon SageMaker Automatic Model Tuning (AMT) is a fully managed system for black-box optimization at scale. AMT finds the best version of a machine learning model by repeatedly training it with different hyperparameter configurations. It can be used with built-in algorithms, custom algorithms, and Amazon SageMaker pre-built containers for machine learning frameworks.
  arXiv  Detail & Related papers  (2020-12-15T18:34:34Z)
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction [75.16836697734995]
  We propose a two-stage algorithm called Automatic Feature Interaction Selection (AutoFIS) AutoFIS can automatically identify important feature interactions for factorization models with computational cost just equivalent to training the target model to convergence. AutoFIS has been deployed onto the training platform of Huawei App Store recommendation service.
  arXiv  Detail & Related papers  (2020-03-25T06:53:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.