A Novel Memetic Strategy for Optimized Learning of Classification Trees

• URL: http://arxiv.org/abs/2305.07959v1
• Date: Sat, 13 May 2023 16:29:10 GMT
• Title: A Novel Memetic Strategy for Optimized Learning of Classification Trees
• Authors: Tommaso Aldinucci
• Abstract summary: We propose a novel evolutionary algorithm for the induction of classification trees that exploits a memetic approach that is able to handle datasets with thousands of points. Our procedure combines the exploration of the feasible space of solutions with local searches to obtain structures with generalization capabilities that are competitive with the state-of-the-art methods.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given the increasing interest in interpretable machine learning, classification trees have again attracted the attention of the scientific community because of their glass-box structure. These models are usually built using greedy procedures, solving subproblems to find cuts in the feature space that minimize some impurity measures. In contrast to this standard greedy approach and to the recent advances in the definition of the learning problem through MILP-based exact formulations, in this paper we propose a novel evolutionary algorithm for the induction of classification trees that exploits a memetic approach that is able to handle datasets with thousands of points. Our procedure combines the exploration of the feasible space of solutions with local searches to obtain structures with generalization capabilities that are competitive with the state-of-the-art methods.

Related papers

• Anti-Collapse Loss for Deep Metric Learning Based on Coding Rate Metric [99.19559537966538]
  DML aims to learn a discriminative high-dimensional embedding space for downstream tasks like classification, clustering, and retrieval. To maintain the structure of embedding space and avoid feature collapse, we propose a novel loss function called Anti-Collapse Loss. Comprehensive experiments on benchmark datasets demonstrate that our proposed method outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-03T13:44:20Z)
• Hierarchical clustering with dot products recovers hidden tree structure [53.68551192799585]
  In this paper we offer a new perspective on the well established agglomerative clustering algorithm, focusing on recovery of hierarchical structure. We recommend a simple variant of the standard algorithm, in which clusters are merged by maximum average dot product and not, for example, by minimum distance or within-cluster variance. We demonstrate that the tree output by this algorithm provides a bona fide estimate of generative hierarchical structure in data, under a generic probabilistic graphical model.
  arXiv  Detail & Related papers  (2023-05-24T11:05:12Z)
• A Mathematical Programming Approach to Optimal Classification Forests [1.0499611180329806]
  This paper introduces Weighted Optimal Classification Forests (WOCFs) WOCFs take advantage of an optimal ensemble of decision trees to derive accurate and interpretable classifiers. Overall, WOCFs complement existing methods such as CART, Optimal Classification Trees, Random Forests and XGBoost.
  arXiv  Detail & Related papers  (2022-11-18T20:33:08Z)
• Supervised Dimensionality Reduction and Classification with Convolutional Autoencoders [1.1164202369517053]
  A Convolutional Autoencoder is combined to simultaneously produce supervised dimensionality reduction and predictions. The resulting Latent Space can be utilized to improve traditional, interpretable classification algorithms. The proposed methodology introduces advanced explainability regarding, not only the data structure through the produced latent space, but also about the classification behaviour.
  arXiv  Detail & Related papers  (2022-08-25T15:18:33Z)
• United We Learn Better: Harvesting Learning Improvements From Class Hierarchies Across Tasks [9.687531080021813]
  We present a theoretical framework based on probability and set theory for extracting parent predictions and a hierarchical loss. Results show results across classification and detection benchmarks and opening up the possibility of hierarchical learning for sigmoid-based detection architectures.
  arXiv  Detail & Related papers  (2021-07-28T20:25:37Z)
• Probabilistic DAG Search [29.47649645431227]
  We develop a probabilistic framework to exploit a search space's latent structure and share information across the search tree. We empirically find our algorithm to compare favorably to existing non-probabilistic alternatives in Tic-Tac-Toe and a feature selection application.
  arXiv  Detail & Related papers  (2021-06-16T11:35:19Z)
• A Survey on Deep Semi-supervised Learning [51.26862262550445]
  We first present a taxonomy for deep semi-supervised learning that categorizes existing methods. We then offer a detailed comparison of these methods in terms of the type of losses, contributions, and architecture differences.
  arXiv  Detail & Related papers  (2021-02-28T16:22:58Z)
• Unsupervised Embedding of Hierarchical Structure in Euclidean Space [30.507049058838025]
  We consider learning a non-linear embedding of data into Euclidean space as a way to improve the hierarchical clustering produced by agglomerative algorithms. We show that rescaling the latent space embedding leads to improved results for both dendrogram purity and the Moseley-Wang cost function.
  arXiv  Detail & Related papers  (2020-10-30T03:57:09Z)
• Reinforcement Learning as Iterative and Amortised Inference [62.997667081978825]
  We use the control as inference framework to outline a novel classification scheme based on amortised and iterative inference. We show that taking this perspective allows us to identify parts of the algorithmic design space which have been relatively unexplored.
  arXiv  Detail & Related papers  (2020-06-13T16:10:03Z)
• Parameterizing Branch-and-Bound Search Trees to Learn Branching Policies [76.83991682238666]
  Branch and Bound (B&B) is the exact tree search method typically used to solve Mixed-Integer Linear Programming problems (MILPs) We propose a novel imitation learning framework, and introduce new input features and architectures to represent branching.
  arXiv  Detail & Related papers  (2020-02-12T17:43:23Z)
• Deep Metric Structured Learning For Facial Expression Recognition [58.7528672474537]
  We propose a deep metric learning model to create embedded sub-spaces with a well defined structure. A new loss function that imposes Gaussian structures on the output space is introduced to create these sub-spaces. We experimentally demonstrate that the learned embedding can be successfully used for various applications including expression retrieval and emotion recognition.
  arXiv  Detail & Related papers  (2020-01-18T06:23:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.