Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning

• URL: http://arxiv.org/abs/2007.04074v3
• Date: Tue, 4 Oct 2022 12:18:34 GMT
• Title: Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning
• Authors: Matthias Feurer, Katharina Eggensperger, Stefan Falkner, Marius Lindauer and Frank Hutter
• Abstract summary: We introduce new AutoML approaches motivated by our winning submission to the second ChaLearn AutoML challenge. We develop PoSH Auto-sklearn, which enables AutoML systems to work well on large datasets under rigid time limits. We also propose a solution towards truly hands-free AutoML.
• Score: 45.643809726832764
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated Machine Learning (AutoML) supports practitioners and researchers with the tedious task of designing machine learning pipelines and has recently achieved substantial success. In this paper, we introduce new AutoML approaches motivated by our winning submission to the second ChaLearn AutoML challenge. We develop PoSH Auto-sklearn, which enables AutoML systems to work well on large datasets under rigid time limits by using a new, simple and meta-feature-free meta-learning technique and by employing a successful bandit strategy for budget allocation. However, PoSH Auto-sklearn introduces even more ways of running AutoML and might make it harder for users to set it up correctly. Therefore, we also go one step further and study the design space of AutoML itself, proposing a solution towards truly hands-free AutoML. Together, these changes give rise to the next generation of our AutoML system, Auto-sklearn 2.0. We verify the improvements by these additions in an extensive experimental study on 39 AutoML benchmark datasets. We conclude the paper by comparing to other popular AutoML frameworks and Auto-sklearn 1.0, reducing the relative error by up to a factor of 4.5, and yielding a performance in 10 minutes that is substantially better than what Auto-sklearn 1.0 achieves within an hour.

Related papers

• AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML [56.565200973244146]
  Automated machine learning (AutoML) accelerates AI development by automating tasks in the development pipeline. Recent works have started exploiting large language models (LLM) to lessen such burden. This paper proposes AutoML-Agent, a novel multi-agent framework tailored for full-pipeline AutoML.
  arXiv  Detail & Related papers  (2024-10-03T20:01:09Z)
• Position: A Call to Action for a Human-Centered AutoML Paradigm [83.78883610871867]
  Automated machine learning (AutoML) was formed around the fundamental objectives of automatically and efficiently configuring machine learning (ML) We argue that a key to unlocking AutoML's full potential lies in addressing the currently underexplored aspect of user interaction with AutoML systems.
  arXiv  Detail & Related papers  (2024-06-05T15:05:24Z)
• The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation [93.01964988474755]
  AutoMQM is a prompting technique which asks large language models to identify and categorize errors in translations. We study the impact of labeled data through in-context learning and finetuning. We then evaluate AutoMQM with PaLM-2 models, and we find that it improves performance compared to just prompting for scores.
  arXiv  Detail & Related papers  (2023-08-14T17:17:21Z)
• Towards Green Automated Machine Learning: Status Quo and Future Directions [71.86820260846369]
  AutoML is being criticised for its high resource consumption. This paper proposes Green AutoML, a paradigm to make the whole AutoML process more environmentally friendly.
  arXiv  Detail & Related papers  (2021-11-10T18:57:27Z)
• Naive Automated Machine Learning -- A Late Baseline for AutoML [0.0]
  Automated Machine Learning (AutoML) is the problem of automatically finding the pipeline with the best generalization performance on some given dataset. We present Naive AutoML, a very simple solution to AutoML that exploits important meta-knowledge about machine learning problems.
  arXiv  Detail & Related papers  (2021-03-18T19:52:12Z)
• A Neophyte With AutoML: Evaluating the Promises of Automatic Machine Learning Tools [1.713291434132985]
  This paper discusses modern Auto Machine Learning (AutoML) tools from the perspective of a person with little prior experience in Machine Learning (ML) There are many AutoML tools both ready-to-use and under development, which are created to simplify and democratize usage of ML technologies in everyday life.
  arXiv  Detail & Related papers  (2021-01-14T19:28:57Z)
• Leveraging Automated Machine Learning for Text Classification: Evaluation of AutoML Tools and Comparison with Human Performance [0.07734726150561087]
  This work compares four AutoML tools on 13 different popular datasets. Results show that the AutoML tools perform better than the machine learning community in 4 out of 13 tasks.
  arXiv  Detail & Related papers  (2020-12-07T10:31:13Z)
• Is deep learning necessary for simple classification tasks? [3.3793659640122717]
  Automated machine learning (AutoML) and deep learning (DL) are two cutting-edge paradigms used to solve inductive learning tasks. We compare AutoML and DL in the context of binary classification on 6 well-characterized public datasets. We also evaluate a new tool for genetic programming-based AutoML that incorporates deep estimators.
  arXiv  Detail & Related papers  (2020-06-11T18:41:47Z)
• Adaptation Strategies for Automated Machine Learning on Evolving Data [7.843067454030999]
  This study is to understand the effect of data stream challenges such as concept drift on the performance of AutoML methods. We propose 6 concept drift adaptation strategies and evaluate their effectiveness on different AutoML approaches.
  arXiv  Detail & Related papers  (2020-06-09T14:29:16Z)
• AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data [120.2298620652828]
  We introduce AutoGluon-Tabular, an open-source AutoML framework that requires only a single line of Python to train highly accurate machine learning models. Tests on a suite of 50 classification and regression tasks from Kaggle and the OpenML AutoML Benchmark reveal that AutoGluon is faster, more robust, and much more accurate.
  arXiv  Detail & Related papers  (2020-03-13T23:10:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.