The Technological Emergence of AutoML: A Survey of Performant Software and Applications in the Context of Industry

• URL: http://arxiv.org/abs/2211.04148v1
• Date: Tue, 8 Nov 2022 10:42:08 GMT
• Title: The Technological Emergence of AutoML: A Survey of Performant Software and Applications in the Context of Industry
• Authors: Alexander Scriven, David Jacob Kedziora, Katarzyna Musial, Bogdan Gabrys
• Abstract summary: Automated/Autonomous Machine Learning (AutoML/AutonoML) is a relatively young field. This review makes two primary contributions to knowledge around this topic. It provides the most up-to-date and comprehensive survey of existing AutoML tools, both open-source and commercial.
• Score: 72.10607978091492
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With most technical fields, there exists a delay between fundamental academic research and practical industrial uptake. Whilst some sciences have robust and well-established processes for commercialisation, such as the pharmaceutical practice of regimented drug trials, other fields face transitory periods in which fundamental academic advancements diffuse gradually into the space of commerce and industry. For the still relatively young field of Automated/Autonomous Machine Learning (AutoML/AutonoML), that transitory period is under way, spurred on by a burgeoning interest from broader society. Yet, to date, little research has been undertaken to assess the current state of this dissemination and its uptake. Thus, this review makes two primary contributions to knowledge around this topic. Firstly, it provides the most up-to-date and comprehensive survey of existing AutoML tools, both open-source and commercial. Secondly, it motivates and outlines a framework for assessing whether an AutoML solution designed for real-world application is 'performant'; this framework extends beyond the limitations of typical academic criteria, considering a variety of stakeholder needs and the human-computer interactions required to service them. Thus, additionally supported by an extensive assessment and comparison of academic and commercial case-studies, this review evaluates mainstream engagement with AutoML in the early 2020s, identifying obstacles and opportunities for accelerating future uptake.

Related papers

• Aligning Academia with Industry: An Empirical Study of Industrial Needs and Academic Capabilities in AI-Driven Software Engineering [45.09204791294318]
  The rapid advancement of large language models (LLMs) is fundamentally reshaping software engineering (SE)<n>While top-tier SE venues continue to show sustained or emerging focus on areas like automated testing and program repair, the alignment of these academic advances with real industrial needs remains unclear.<n>This study aims to refocus academic attention on these important yet under-explored problems and to guide future SE research toward greater industrial impact.
  arXiv  Detail & Related papers  (2025-12-17T07:29:18Z)
• Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents [63.03252293761656]
  This paper systematically reviews the technologies, applications, and evaluation methods of industry agents based on large language models (LLMs)<n>We examine the three key technological pillars that support the advancement of agent capabilities: Memory, Planning, and Tool Use.<n>We provide an overview of the application of industry agents in real-world domains such as digital engineering, scientific discovery, embodied intelligence, collaborative business execution, and complex system simulation.
  arXiv  Detail & Related papers  (2025-10-20T12:46:55Z)
• FML-bench: A Benchmark for Automatic ML Research Agents Highlighting the Importance of Exploration Breadth [43.606494515048524]
  Large language models (LLMs) have sparked growing interest in automatic machine learning research agents.<n>Existing benchmarks tend to overemphasize engineering aspects while neglecting academic rigor.<n>We introduce FML-bench, a benchmark designed to evaluate automatic machine learning research agents on 8 diverse and fundamental machine learning research problems.
  arXiv  Detail & Related papers  (2025-10-12T06:41:05Z)
• Retrieval-Augmented Generation in Industry: An Interview Study on Use Cases, Requirements, Challenges, and Evaluation [0.0]
  Retrieval-Augmented Generation (RAG) is a rapidly evolving field within AI.<n>There is a significant lack of research on its practical application in industrial contexts.<n>Our study investigates how companies apply RAG in practice.
  arXiv  Detail & Related papers  (2025-08-11T09:40:54Z)
• Surveying the MLLM Landscape: A Meta-Review of Current Surveys [17.372501468675303]
  Multimodal Large Language Models (MLLMs) have become a transformative force in the field of artificial intelligence. This survey aims to provide a systematic review of benchmark tests and evaluation methods for MLLMs.
  arXiv  Detail & Related papers  (2024-09-17T14:35:38Z)
• RelevAI-Reviewer: A Benchmark on AI Reviewers for Survey Paper Relevance [0.8089605035945486]
  We propose RelevAI-Reviewer, an automatic system that conceptualizes the task of survey paper review as a classification problem. We introduce a novel dataset comprised of 25,164 instances. Each instance contains one prompt and four candidate papers, each varying in relevance to the prompt. We develop a machine learning (ML) model capable of determining the relevance of each paper and identifying the most pertinent one.
  arXiv  Detail & Related papers  (2024-06-13T06:42:32Z)
• Position: A Call to Action for a Human-Centered AutoML Paradigm [83.78883610871867]
  Automated machine learning (AutoML) was formed around the fundamental objectives of automatically and efficiently configuring machine learning (ML) We argue that a key to unlocking AutoML's full potential lies in addressing the currently underexplored aspect of user interaction with AutoML systems.
  arXiv  Detail & Related papers  (2024-06-05T15:05:24Z)
• Why Autonomous Vehicles Are Not Ready Yet: A Multi-Disciplinary Review of Problems, Attempted Solutions, and Future Directions [43.389650174195914]
  The present review adopts an integrative and multidisciplinary approach to investigate the major challenges faced by the automative sector. The review examines the limitations and risks associated with current technologies and the most promising solutions devised by the researchers.
  arXiv  Detail & Related papers  (2023-11-15T16:41:18Z)
• A Survey on Large Language Model based Autonomous Agents [105.2509166861984]
  Large language models (LLMs) have demonstrated remarkable potential in achieving human-level intelligence. This paper delivers a systematic review of the field of LLM-based autonomous agents from a holistic perspective. We present a comprehensive overview of the diverse applications of LLM-based autonomous agents in the fields of social science, natural science, and engineering.
  arXiv  Detail & Related papers  (2023-08-22T13:30:37Z)
• Benchmarking Automated Machine Learning Methods for Price Forecasting Applications [58.720142291102135]
  We show the possibility of substituting manually created ML pipelines with automated machine learning (AutoML) solutions. Based on the CRISP-DM process, we split the manual ML pipeline into a machine learning and non-machine learning part. We show in a case study for the industrial use case of price forecasting, that domain knowledge combined with AutoML can weaken the dependence on ML experts.
  arXiv  Detail & Related papers  (2023-04-28T10:27:38Z)
• Interactive Machine Learning: A State of the Art Review [0.0]
  We provide a comprehensive analysis of the state-of-the-art of interactive machine learning (iML) Research works on adversarial black-box attacks and corresponding iML based defense system, exploratory machine learning, resource constrained learning, and iML performance evaluation are analyzed.
  arXiv  Detail & Related papers  (2022-07-13T13:43:16Z)
• The Roles and Modes of Human Interactions with Automated Machine Learning Systems [7.670270099306412]
  Automated machine learning (AutoML) systems continue to progress in both sophistication and performance. It becomes important to understand the how' and why' of human-computer interaction (HCI) within these frameworks. This review serves to identify key research directions aimed at better facilitating the roles and modes of human interactions with both current and future AutoML systems.
  arXiv  Detail & Related papers  (2022-05-09T09:28:43Z)
• Automated Machine Learning, Bounded Rationality, and Rational Metareasoning [62.997667081978825]
  We will look at automated machine learning (AutoML) and related problems from the perspective of bounded rationality. Taking actions under bounded resources requires an agent to reflect on how to use these resources in an optimal way.
  arXiv  Detail & Related papers  (2021-09-10T09:10:20Z)
• Artificial Intelligence for IT Operations (AIOPS) Workshop White Paper [50.25428141435537]
  Artificial Intelligence for IT Operations (AIOps) is an emerging interdisciplinary field arising in the intersection between machine learning, big data, streaming analytics, and the management of IT operations. Main aim of the AIOPS workshop is to bring together researchers from both academia and industry to present their experiences, results, and work in progress in this field.
  arXiv  Detail & Related papers  (2021-01-15T10:43:10Z)
• AutoML to Date and Beyond: Challenges and Opportunities [30.60364966752454]
  AutoML tools aim to make machine learning accessible for non-machine learning experts. We introduce a new classification system for AutoML systems. We lay out a roadmap for the future, pinpointing the research required to further automate the end-to-end machine learning pipeline.
  arXiv  Detail & Related papers  (2020-10-21T06:08:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.