The Potential of AutoML for Recommender Systems

• URL: http://arxiv.org/abs/2402.04453v1
• Date: Tue, 6 Feb 2024 22:42:28 GMT
• Title: The Potential of AutoML for Recommender Systems
• Authors: Tobias Vente, Joeran Beel
• Abstract summary: We compare the predictive performance of 60 AutoML, AutoRecSys, ML, and RecSys algorithms from 15 libraries. We found that AutoML and AutoRecSys libraries performed best for six of the 14 datasets. The single-best library was the AutoRecSys library Auto-Surprise, which performed best on five datasets.
• Score: 0.135975510645475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated Machine Learning (AutoML) has greatly advanced applications of Machine Learning (ML) including model compression, machine translation, and computer vision. Recommender Systems (RecSys) can be seen as an application of ML. Yet, AutoML has found little attention in the RecSys community; nor has RecSys found notable attention in the AutoML community. Only few and relatively simple Automated Recommender Systems (AutoRecSys) libraries exist that adopt AutoML techniques. However, these libraries are based on student projects and do not offer the features and thorough development of AutoML libraries. We set out to determine how AutoML libraries perform in the scenario of an inexperienced user who wants to implement a recommender system. We compared the predictive performance of 60 AutoML, AutoRecSys, ML, and RecSys algorithms from 15 libraries, including a mean predictor baseline, on 14 explicit feedback RecSys datasets. To simulate the perspective of an inexperienced user, the algorithms were evaluated with default hyperparameters. We found that AutoML and AutoRecSys libraries performed best. AutoML libraries performed best for six of the 14 datasets (43%), but it was not always the same AutoML library performing best. The single-best library was the AutoRecSys library Auto-Surprise, which performed best on five datasets (36%). On three datasets (21%), AutoML libraries performed poorly, and RecSys libraries with default parameters performed best. Although, while obtaining 50% of all placements in the top five per dataset, RecSys algorithms fall behind AutoML on average. ML algorithms generally performed the worst.

Related papers

• AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML [56.565200973244146]
  Automated machine learning (AutoML) accelerates AI development by automating tasks in the development pipeline. Recent works have started exploiting large language models (LLM) to lessen such burden. This paper proposes AutoML-Agent, a novel multi-agent framework tailored for full-pipeline AutoML.
  arXiv  Detail & Related papers  (2024-10-03T20:01:09Z)
• MLLM-DataEngine: An Iterative Refinement Approach for MLLM [62.30753425449056]
  We propose a novel closed-loop system that bridges data generation, model training, and evaluation. Within each loop, the MLLM-DataEngine first analyze the weakness of the model based on the evaluation results. For targeting, we propose an Adaptive Bad-case Sampling module, which adjusts the ratio of different types of data. For quality, we resort to GPT-4 to generate high-quality data with each given data type.
  arXiv  Detail & Related papers  (2023-08-25T01:41:04Z)
• Towards Green Automated Machine Learning: Status Quo and Future Directions [71.86820260846369]
  AutoML is being criticised for its high resource consumption. This paper proposes Green AutoML, a paradigm to make the whole AutoML process more environmentally friendly.
  arXiv  Detail & Related papers  (2021-11-10T18:57:27Z)
• LightAutoML: AutoML Solution for a Large Financial Services Ecosystem [108.09104876115428]
  We present an AutoML system called LightAutoML developed for a large European financial services company. Our framework was piloted and deployed in numerous applications and performed at the level of the experienced data scientists.
  arXiv  Detail & Related papers  (2021-09-03T13:52:32Z)
• Comparison of Automated Machine Learning Tools for SMS Spam Message Filtering [0.0]
  Short Message Service (SMS) is a popular service used for communication by mobile users. In this work, a classification performance comparison was conducted between three automatic machine learning (AutoML) tools for SMS spam message filtering. Experimental results showed that ensemble models achieved the best classification performance.
  arXiv  Detail & Related papers  (2021-06-16T10:16:07Z)
• A Neophyte With AutoML: Evaluating the Promises of Automatic Machine Learning Tools [1.713291434132985]
  This paper discusses modern Auto Machine Learning (AutoML) tools from the perspective of a person with little prior experience in Machine Learning (ML) There are many AutoML tools both ready-to-use and under development, which are created to simplify and democratize usage of ML technologies in everyday life.
  arXiv  Detail & Related papers  (2021-01-14T19:28:57Z)
• Can AutoML outperform humans? An evaluation on popular OpenML datasets using AutoML Benchmark [0.05156484100374058]
  This paper compares four AutoML frameworks on 12 different popular datasets from OpenML. Results show that the automated frameworks perform better or equal than the machine learning community in 7 out of 12 OpenML tasks.
  arXiv  Detail & Related papers  (2020-09-03T10:25:34Z)
• Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning [45.643809726832764]
  We introduce new AutoML approaches motivated by our winning submission to the second ChaLearn AutoML challenge. We develop PoSH Auto-sklearn, which enables AutoML systems to work well on large datasets under rigid time limits. We also propose a solution towards truly hands-free AutoML.
  arXiv  Detail & Related papers  (2020-07-08T12:41:03Z)
• AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data [120.2298620652828]
  We introduce AutoGluon-Tabular, an open-source AutoML framework that requires only a single line of Python to train highly accurate machine learning models. Tests on a suite of 50 classification and regression tasks from Kaggle and the OpenML AutoML Benchmark reveal that AutoGluon is faster, more robust, and much more accurate.
  arXiv  Detail & Related papers  (2020-03-13T23:10:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.