Tree-Based Dynamic Classifier Chains

• URL: http://arxiv.org/abs/2112.06672v1
• Date: Mon, 13 Dec 2021 13:49:49 GMT
• Title: Tree-Based Dynamic Classifier Chains
• Authors: Eneldo Loza Menc\'ia, Moritz Kulessa, Simon Bohlender, Johannes F\"urnkranz
• Abstract summary: Dynamic classification chains denote the idea that for each instance to classify, the order in which the labels are predicted is dynamically chosen. We show that a dynamic selection of the next label improves over the use of a static ordering under an otherwise unchanged random decision tree model. Our results show that this variant outperforms random decision trees and other tree-based multi-label classification methods.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Classifier chains are an effective technique for modeling label dependencies in multi-label classification. However, the method requires a fixed, static order of the labels. While in theory, any order is sufficient, in practice, this order has a substantial impact on the quality of the final prediction. Dynamic classifier chains denote the idea that for each instance to classify, the order in which the labels are predicted is dynamically chosen. The complexity of a naive implementation of such an approach is prohibitive, because it would require to train a sequence of classifiers for every possible permutation of the labels. To tackle this problem efficiently, we propose a new approach based on random decision trees which can dynamically select the label ordering for each prediction. We show empirically that a dynamic selection of the next label improves over the use of a static ordering under an otherwise unchanged random decision tree model. % and experimental environment. In addition, we also demonstrate an alternative approach based on extreme gradient boosted trees, which allows for a more target-oriented training of dynamic classifier chains. Our results show that this variant outperforms random decision trees and other tree-based multi-label classification methods. More importantly, the dynamic selection strategy allows to considerably speed up training and prediction.

Related papers

• Exploring Beyond Logits: Hierarchical Dynamic Labeling Based on Embeddings for Semi-Supervised Classification [49.09505771145326]
  We propose a Hierarchical Dynamic Labeling (HDL) algorithm that does not depend on model predictions and utilizes image embeddings to generate sample labels. Our approach has the potential to change the paradigm of pseudo-label generation in semi-supervised learning.
  arXiv  Detail & Related papers  (2024-04-26T06:00:27Z)
• Combining Self-labeling with Selective Sampling [2.0305676256390934]
  This work combines self-labeling techniques with active learning in a selective sampling scenario. We show that naive application of self-labeling can harm performance by introducing bias towards selected classes. The proposed method matches current selective sampling methods or achieves better results.
  arXiv  Detail & Related papers  (2023-01-11T11:58:45Z)
• Set-valued prediction in hierarchical classification with constrained representation complexity [4.258263831866309]
  We focus on hierarchical multi-class classification problems, where valid sets correspond to internal nodes of the hierarchy. We propose three methods and evaluate them on benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-13T15:13:19Z)
• Optimal randomized classification trees [0.0]
  Classification and Regression Trees (CARTs) are off-the-shelf techniques in modern Statistics and Machine Learning. CARTs are built by means of a greedy procedure, sequentially deciding the splitting predictor variable(s) and the associated threshold. This greedy approach trains trees very fast, but, by its nature, their classification accuracy may not be competitive against other state-of-the-art procedures.
  arXiv  Detail & Related papers  (2021-10-19T11:41:12Z)
• Minimax Active Learning [61.729667575374606]
  Active learning aims to develop label-efficient algorithms by querying the most representative samples to be labeled by a human annotator. Current active learning techniques either rely on model uncertainty to select the most uncertain samples or use clustering or reconstruction to choose the most diverse set of unlabeled examples. We develop a semi-supervised minimax entropy-based active learning algorithm that leverages both uncertainty and diversity in an adversarial manner.
  arXiv  Detail & Related papers  (2020-12-18T19:03:40Z)
• Online Active Model Selection for Pre-trained Classifiers [72.84853880948894]
  We design an online selective sampling approach that actively selects informative examples to label and outputs the best model with high probability at any round. Our algorithm can be used for online prediction tasks for both adversarial and streams.
  arXiv  Detail & Related papers  (2020-10-19T19:53:15Z)
• SPL-MLL: Selecting Predictable Landmarks for Multi-Label Learning [87.27700889147144]
  We propose to select a small subset of labels as landmarks which are easy to predict according to input (predictable) and can well recover the other possible labels (representative) We employ the Alternating Direction Method (ADM) to solve our problem. Empirical studies on real-world datasets show that our method achieves superior classification performance over other state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-16T11:07:44Z)
• Probabilistic Decoupling of Labels in Classification [4.865747672937677]
  We develop a principled, probabilistic, unified approach to non-standard classification tasks. We train a classifier on the given labels to predict the label-distribution. We then infer the underlying class-distributions by variationally optimizing a model of label-class transitions.
  arXiv  Detail & Related papers  (2020-06-16T10:07:50Z)
• Extreme Gradient Boosted Multi-label Trees for Dynamic Classifier Chains [1.357087732949916]
  Dynamic classifier chains (DCC) the label ordering is chosen for each prediction dynamically depending on the respective instance at hand. We combine this concept with the boosting of extreme gradient boosted trees (XGBoost), an effective and scalable state-of-the-art technique. As only positive labels have to be predicted and these are usually only few, the training costs can be substantially reduced.
  arXiv  Detail & Related papers  (2020-06-15T02:35:32Z)
• Interaction Matching for Long-Tail Multi-Label Classification [57.262792333593644]
  We present an elegant and effective approach for addressing limitations in existing multi-label classification models. By performing soft n-gram interaction matching, we match labels with natural language descriptions.
  arXiv  Detail & Related papers  (2020-05-18T15:27:55Z)
• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing [105.91827623768724]
  Machine learning algorithms are susceptible to data poisoning attacks. We present a unifying view of randomized smoothing over arbitrary functions. We propose a new strategy for building classifiers that are pointwise-certifiably robust to general data poisoning attacks.
  arXiv  Detail & Related papers  (2020-02-07T21:28:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.