A Multivocal Literature Review on the Benefits and Limitations of Automated Machine Learning Tools

• URL: http://arxiv.org/abs/2401.11366v1
• Date: Sun, 21 Jan 2024 01:39:39 GMT
• Title: A Multivocal Literature Review on the Benefits and Limitations of Automated Machine Learning Tools
• Authors: Kelly Azevedo, Luigi Quaranta, Fabio Calefato, Marcos Kalinowski
• Abstract summary: We conducted a multivocal literature review, which allowed us to identify 54 sources from the academic literature and 108 sources from the grey literature reporting on AutoML benefits and limitations. Concerning the benefits, we highlight that AutoML tools can help streamline the core steps of ML. We highlight several limitations that may represent obstacles to the widespread adoption of AutoML.
• Score: 9.69672653683112
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Context. Advancements in Machine Learning (ML) are revolutionizing every application domain, driving unprecedented transformations and fostering innovation. However, despite these advances, several organizations are experiencing friction in the adoption of ML-based technologies, mainly due to the shortage of ML professionals. In this context, Automated Machine Learning (AutoML) techniques have been presented as a promising solution to democratize ML adoption. Objective. We aim to provide an overview of the evidence on the benefits and limitations of using AutoML tools. Method. We conducted a multivocal literature review, which allowed us to identify 54 sources from the academic literature and 108 sources from the grey literature reporting on AutoML benefits and limitations. We extracted reported benefits and limitations from the papers and applied thematic analysis. Results. We identified 18 benefits and 25 limitations. Concerning the benefits, we highlight that AutoML tools can help streamline the core steps of ML workflows, namely data preparation, feature engineering, model construction, and hyperparameter tuning, with concrete benefits on model performance, efficiency, and scalability. In addition, AutoML empowers both novice and experienced data scientists, promoting ML accessibility. On the other hand, we highlight several limitations that may represent obstacles to the widespread adoption of AutoML. For instance, AutoML tools may introduce barriers to transparency and interoperability, exhibit limited flexibility for complex scenarios, and offer inconsistent coverage of the ML workflow. Conclusions. The effectiveness of AutoML in facilitating the adoption of machine learning by users may vary depending on the tool and the context in which it is used. As of today, AutoML tools are used to increase human expertise rather than replace it, and, as such, they require skilled users.

Related papers

• UniAutoML: A Human-Centered Framework for Unified Discriminative and Generative AutoML with Large Language Models [5.725785427377439]
  We introduce UniAutoML, a human-centered AutoML framework that unifies AutoML for both discriminative and generative tasks. The human-centered design of UniAutoML innovatively features a conversational user interface (CUI) that facilitates natural language interactions. This design enhances transparency and user control throughout the AutoML training process, allowing users to seamlessly break down or modify the model being trained.
  arXiv  Detail & Related papers  (2024-10-09T17:33:15Z)
• AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML [56.565200973244146]
  Automated machine learning (AutoML) accelerates AI development by automating tasks in the development pipeline. Recent works have started exploiting large language models (LLM) to lessen such burden. This paper proposes AutoML-Agent, a novel multi-agent framework tailored for full-pipeline AutoML.
  arXiv  Detail & Related papers  (2024-10-03T20:01:09Z)
• Position: A Call to Action for a Human-Centered AutoML Paradigm [83.78883610871867]
  Automated machine learning (AutoML) was formed around the fundamental objectives of automatically and efficiently configuring machine learning (ML) We argue that a key to unlocking AutoML's full potential lies in addressing the currently underexplored aspect of user interaction with AutoML systems.
  arXiv  Detail & Related papers  (2024-06-05T15:05:24Z)
• Assessing the Use of AutoML for Data-Driven Software Engineering [10.40771687966477]
  AutoML promises to automate the building of end-to-end AI/ML pipelines. Despite the growing interest and high expectations, there is a dearth of information about the extent to which AutoML is currently adopted.
  arXiv  Detail & Related papers  (2023-07-20T11:14:24Z)
• Benchmarking Automated Machine Learning Methods for Price Forecasting Applications [58.720142291102135]
  We show the possibility of substituting manually created ML pipelines with automated machine learning (AutoML) solutions. Based on the CRISP-DM process, we split the manual ML pipeline into a machine learning and non-machine learning part. We show in a case study for the industrial use case of price forecasting, that domain knowledge combined with AutoML can weaken the dependence on ML experts.
  arXiv  Detail & Related papers  (2023-04-28T10:27:38Z)
• Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML [52.86328317233883]
  We present a comprehensive overview of different ways in which fairness-related harm can arise. We highlight several open technical challenges for future work in this direction.
  arXiv  Detail & Related papers  (2023-03-15T09:40:08Z)
• OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System [85.8338446357469]
  We introduce OmniForce, a human-centered AutoML system that yields both human-assisted ML and ML-assisted human techniques. We show how OmniForce can put an AutoML system into practice and build adaptive AI in open-environment scenarios.
  arXiv  Detail & Related papers  (2023-03-01T13:35:22Z)
• AutoML in The Wild: Obstacles, Workarounds, and Expectations [37.813441975457735]
  This study focuses on understanding the limitations of AutoML encountered by users in their real-world practices. Our findings reveal that users actively exercise user agency to overcome three major challenges arising from customizability, transparency, and privacy.
  arXiv  Detail & Related papers  (2023-02-21T17:06:46Z)
• An Empirical Study on the Usage of Automated Machine Learning Tools [10.901346577426542]
  The popularity of automated machine learning (AutoML) tools has increased over the past few years. Recent work performed qualitative studies on practitioners' experiences of using AutoML tools. We conducted an empirical study to understand how ML practitioners use AutoML tools in their projects.
  arXiv  Detail & Related papers  (2022-08-28T02:01:58Z)
• Practical Machine Learning Safety: A Survey and Primer [81.73857913779534]
  Open-world deployment of Machine Learning algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities. New models and training techniques to reduce generalization error, achieve domain adaptation, and detect outlier examples and adversarial attacks. Our organization maps state-of-the-art ML techniques to safety strategies in order to enhance the dependability of the ML algorithm from different aspects.
  arXiv  Detail & Related papers  (2021-06-09T05:56:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.