A Robust Experimental Evaluation of Automated Multi-Label Classification Methods

• URL: http://arxiv.org/abs/2005.08083v2
• Date: Fri, 31 Jul 2020 16:48:27 GMT
• Title: A Robust Experimental Evaluation of Automated Multi-Label Classification Methods
• Authors: Alex G. C. de S\'a, Cristiano G. Pimenta, Gisele L. Pappa and Alex A. Freitas
• Abstract summary: This paper approaches AutoML for multi-label classification (MLC) problems. In MLC, each example can be simultaneously associated to several class labels. Overall, we observe that the most prominent method is the one based on a canonical grammar-based genetic programming (GGP) search method.
• Score: 0.735996217853436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated Machine Learning (AutoML) has emerged to deal with the selection and configuration of algorithms for a given learning task. With the progression of AutoML, several effective methods were introduced, especially for traditional classification and regression problems. Apart from the AutoML success, several issues remain open. One issue, in particular, is the lack of ability of AutoML methods to deal with different types of data. Based on this scenario, this paper approaches AutoML for multi-label classification (MLC) problems. In MLC, each example can be simultaneously associated to several class labels, unlike the standard classification task, where an example is associated to just one class label. In this work, we provide a general comparison of five automated multi-label classification methods -- two evolutionary methods, one Bayesian optimization method, one random search and one greedy search -- on 14 datasets and three designed search spaces. Overall, we observe that the most prominent method is the one based on a canonical grammar-based genetic programming (GGP) search method, namely Auto-MEKA$_{GGP}$. Auto-MEKA$_{GGP}$ presented the best average results in our comparison and was statistically better than all the other methods in different search spaces and evaluated measures, except when compared to the greedy search method.

Related papers

• POGEMA: A Benchmark Platform for Cooperative Multi-Agent Navigation [76.67608003501479]
  We introduce and specify an evaluation protocol defining a range of domain-related metrics computed on the basics of the primary evaluation indicators. The results of such a comparison, which involves a variety of state-of-the-art MARL, search-based, and hybrid methods, are presented.
  arXiv  Detail & Related papers  (2024-07-20T16:37:21Z)
• Automated Machine Learning for Multi-Label Classification [3.2634122554914002]
  We devise a novel AutoML approach for single-label classification tasks, consisting of two algorithms at most. We investigate how well AutoML approaches that form the state of the art for single-label classification tasks scale with the increased problem complexity of AutoML for multi-label classification.
  arXiv  Detail & Related papers  (2024-02-28T09:40:36Z)
• Boosting Prompt-Based Self-Training With Mapping-Free Automatic Verbalizer for Multi-Class Classification [3.647905567437244]
  We introduce a novel, efficient verbalizer structure, named Mapping-free Automatic Verbal Modelingizer (MAV) MAV serves as a trainable verbalizer that automatically extracts the requisite word features for classification by capitalizing on all available information from predictions. Experimental results on five multi-class classification datasets indicate MAV's superior self-training efficacy.
  arXiv  Detail & Related papers  (2023-12-08T11:43:00Z)
• Unified Functional Hashing in Automatic Machine Learning [58.77232199682271]
  We show that large efficiency gains can be obtained by employing a fast unified functional hash. Our hash is "functional" in that it identifies equivalent candidates even if they were represented or coded differently. We show dramatic improvements on multiple AutoML domains, including neural architecture search and algorithm discovery.
  arXiv  Detail & Related papers  (2023-02-10T18:50:37Z)
• Multi-Instance Partial-Label Learning: Towards Exploiting Dual Inexact Supervision [53.530957567507365]
  In some real-world tasks, each training sample is associated with a candidate label set that contains one ground-truth label and some false positive labels. In this paper, we formalize such problems as multi-instance partial-label learning (MIPL) Existing multi-instance learning algorithms and partial-label learning algorithms are suboptimal for solving MIPL problems.
  arXiv  Detail & Related papers  (2022-12-18T03:28:51Z)
• Benchmarking AutoML algorithms on a collection of binary problems [3.3793659640122717]
  This paper compares the performance of four different AutoML algorithms: Tree-based Pipeline Optimization Tool (TPOT), Auto-Sklearn, Auto-Sklearn 2, and H2O AutoML. We confirm that AutoML can identify pipelines that perform well on all included datasets.
  arXiv  Detail & Related papers  (2022-12-06T01:53:50Z)
• How to Train Your MAML to Excel in Few-Shot Classification [26.51244463209443]
  We show how to train MAML to excel in few-shot classification. Our approach, which we name UNICORN-MAML, performs on a par with or even outperforms state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2021-06-30T17:56:15Z)
• An Extensive Experimental Evaluation of Automated Machine Learning Methods for Recommending Classification Algorithms (Extended Version) [4.400989370979334]
  Three of these methods are based on Evolutionary Algorithms (EAs), and the other is Auto-WEKA, a well-known AutoML method. We performed controlled experiments where these four AutoML methods were given the same runtime limit for different values of this limit. In general, the difference in predictive accuracy of the three best AutoML methods was not statistically significant.
  arXiv  Detail & Related papers  (2020-09-16T02:36:43Z)
• SPL-MLL: Selecting Predictable Landmarks for Multi-Label Learning [87.27700889147144]
  We propose to select a small subset of labels as landmarks which are easy to predict according to input (predictable) and can well recover the other possible labels (representative) We employ the Alternating Direction Method (ADM) to solve our problem. Empirical studies on real-world datasets show that our method achieves superior classification performance over other state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-16T11:07:44Z)
• Interaction Matching for Long-Tail Multi-Label Classification [57.262792333593644]
  We present an elegant and effective approach for addressing limitations in existing multi-label classification models. By performing soft n-gram interaction matching, we match labels with natural language descriptions.
  arXiv  Detail & Related papers  (2020-05-18T15:27:55Z)
• AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data [120.2298620652828]
  We introduce AutoGluon-Tabular, an open-source AutoML framework that requires only a single line of Python to train highly accurate machine learning models. Tests on a suite of 50 classification and regression tasks from Kaggle and the OpenML AutoML Benchmark reveal that AutoGluon is faster, more robust, and much more accurate.
  arXiv  Detail & Related papers  (2020-03-13T23:10:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.