AutoM3L: An Automated Multimodal Machine Learning Framework with Large Language Models

• URL: http://arxiv.org/abs/2408.00665v1
• Date: Thu, 1 Aug 2024 16:01:51 GMT
• Title: AutoM3L: An Automated Multimodal Machine Learning Framework with Large Language Models
• Authors: Daqin Luo, Chengjian Feng, Yuxuan Nong, Yiqing Shen,
• Abstract summary: We introduce AutoM3L, an innovative Automated Multimodal Machine Learning framework. AutoM3L comprehends data modalities and selects appropriate models based on user requirements. We evaluate the performance of AutoM3L on six diverse multimodal datasets.
• Score: 6.496539724366041
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated Machine Learning (AutoML) offers a promising approach to streamline the training of machine learning models. However, existing AutoML frameworks are often limited to unimodal scenarios and require extensive manual configuration. Recent advancements in Large Language Models (LLMs) have showcased their exceptional abilities in reasoning, interaction, and code generation, presenting an opportunity to develop a more automated and user-friendly framework. To this end, we introduce AutoM3L, an innovative Automated Multimodal Machine Learning framework that leverages LLMs as controllers to automatically construct multimodal training pipelines. AutoM3L comprehends data modalities and selects appropriate models based on user requirements, providing automation and interactivity. By eliminating the need for manual feature engineering and hyperparameter optimization, our framework simplifies user engagement and enables customization through directives, addressing the limitations of previous rule-based AutoML approaches. We evaluate the performance of AutoM3L on six diverse multimodal datasets spanning classification, regression, and retrieval tasks, as well as a comprehensive set of unimodal datasets. The results demonstrate that AutoM3L achieves competitive or superior performance compared to traditional rule-based AutoML methods. Furthermore, a user study highlights the user-friendliness and usability of our framework, compared to the rule-based AutoML methods.

Related papers

• UniAutoML: A Human-Centered Framework for Unified Discriminative and Generative AutoML with Large Language Models [5.725785427377439]
  We introduce UniAutoML, a human-centered AutoML framework that unifies AutoML for both discriminative and generative tasks. The human-centered design of UniAutoML innovatively features a conversational user interface (CUI) that facilitates natural language interactions. This design enhances transparency and user control throughout the AutoML training process, allowing users to seamlessly break down or modify the model being trained.
  arXiv  Detail & Related papers  (2024-10-09T17:33:15Z)
• AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML [56.565200973244146]
  Automated machine learning (AutoML) accelerates AI development by automating tasks in the development pipeline. Recent works have started exploiting large language models (LLM) to lessen such burden. This paper proposes AutoML-Agent, a novel multi-agent framework tailored for full-pipeline AutoML.
  arXiv  Detail & Related papers  (2024-10-03T20:01:09Z)
• Position: A Call to Action for a Human-Centered AutoML Paradigm [83.78883610871867]
  Automated machine learning (AutoML) was formed around the fundamental objectives of automatically and efficiently configuring machine learning (ML) We argue that a key to unlocking AutoML's full potential lies in addressing the currently underexplored aspect of user interaction with AutoML systems.
  arXiv  Detail & Related papers  (2024-06-05T15:05:24Z)
• AutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Models [31.816755598468077]
  AutoMM enables fine-tuning of foundation models with just three lines of code. AutoMM offers a comprehensive suite of functionalities spanning classification, regression, object detection, semantic matching, and image segmentation.
  arXiv  Detail & Related papers  (2024-04-24T22:28:12Z)
• AutoMMLab: Automatically Generating Deployable Models from Language Instructions for Computer Vision Tasks [39.71649832548044]
  AutoMMLab is a general-purpose LLM-empowered AutoML system that follows user's language instructions. The proposed AutoMMLab system effectively employs LLMs as the bridge to connect AutoML and OpenMMLab community. Experiments show that our AutoMMLab system is versatile and covers a wide range of mainstream tasks.
  arXiv  Detail & Related papers  (2024-02-23T14:38:19Z)
• TaskBench: Benchmarking Large Language Models for Task Automation [82.2932794189585]
  We introduce TaskBench, a framework to evaluate the capability of large language models (LLMs) in task automation. Specifically, task decomposition, tool selection, and parameter prediction are assessed. Our approach combines automated construction with rigorous human verification, ensuring high consistency with human evaluation.
  arXiv  Detail & Related papers  (2023-11-30T18:02:44Z)
• AutoML-GPT: Large Language Model for AutoML [5.9145212342776805]
  We have established a framework called AutoML-GPT that integrates a comprehensive set of tools and libraries. Through a conversational interface, users can specify their requirements, constraints, and evaluation metrics. We have demonstrated that AutoML-GPT significantly reduces the time and effort required for machine learning tasks.
  arXiv  Detail & Related papers  (2023-09-03T09:39:49Z)
• AutoML-GPT: Automatic Machine Learning with GPT [74.30699827690596]
  We propose developing task-oriented prompts and automatically utilizing large language models (LLMs) to automate the training pipeline. We present the AutoML-GPT, which employs GPT as the bridge to diverse AI models and dynamically trains models with optimized hyper parameters. This approach achieves remarkable results in computer vision, natural language processing, and other challenging areas.
  arXiv  Detail & Related papers  (2023-05-04T02:09:43Z)
• Automatic Componentwise Boosting: An Interpretable AutoML System [1.1709030738577393]
  We propose an AutoML system that constructs an interpretable additive model that can be fitted using a highly scalable componentwise boosting algorithm. Our system provides tools for easy model interpretation such as visualizing partial effects and pairwise interactions. Despite its restriction to an interpretable model space, our system is competitive in terms of predictive performance on most data sets.
  arXiv  Detail & Related papers  (2021-09-12T18:34:33Z)
• Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL [53.40030379661183]
  Auto-PyTorch is a framework to enable fully automated deep learning (AutoDL) It combines multi-fidelity optimization with portfolio construction for warmstarting and ensembling of deep neural networks (DNNs) We show that Auto-PyTorch performs better than several state-of-the-art competitors on average.
  arXiv  Detail & Related papers  (2020-06-24T15:15:17Z)
• AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data [120.2298620652828]
  We introduce AutoGluon-Tabular, an open-source AutoML framework that requires only a single line of Python to train highly accurate machine learning models. Tests on a suite of 50 classification and regression tasks from Kaggle and the OpenML AutoML Benchmark reveal that AutoGluon is faster, more robust, and much more accurate.
  arXiv  Detail & Related papers  (2020-03-13T23:10:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.