Multi forests: Variable importance for multi-class outcomes

• URL: http://arxiv.org/abs/2409.08925v1
• Date: Fri, 13 Sep 2024 15:40:29 GMT
• Title: Multi forests: Variable importance for multi-class outcomes
• Abstract summary: We introduce a new type of variable importance measure (VIM) based on a novel RF variant called multi forests (MuFs) We show that MuFs have slightly lower predictive performance compared to conventional RFs. This is, however, not a limiting factor given the algorithm's primary purpose of calculating the multi-class VIM.
• Score: 49.1574468325115
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In prediction tasks with multi-class outcomes, identifying covariates specifically associated with one or more outcome classes can be important. Conventional variable importance measures (VIMs) from random forests (RFs), like permutation and Gini importance, focus on overall predictive performance or node purity, without differentiating between the classes. Therefore, they can be expected to fail to distinguish class-associated covariates from covariates that only distinguish between groups of classes. We introduce a VIM called multi-class VIM, tailored for identifying exclusively class-associated covariates, via a novel RF variant called multi forests (MuFs). The trees in MuFs use both multi-way and binary splitting. The multi-way splits generate child nodes for each class, using a split criterion that evaluates how well these nodes represent their respective classes. This setup forms the basis of the multi-class VIM, which measures the discriminatory ability of the splits performed in the respective covariates with regard to this split criterion. Alongside the multi-class VIM, we introduce a second VIM, the discriminatory VIM. This measure, based on the binary splits, assesses the strength of the general influence of the covariates, irrespective of their class-associatedness. Simulation studies demonstrate that the multi-class VIM specifically ranks class-associated covariates highly, unlike conventional VIMs which also rank other types of covariates highly. Analyses of 121 datasets reveal that MuFs often have slightly lower predictive performance compared to conventional RFs. This is, however, not a limiting factor given the algorithm's primary purpose of calculating the multi-class VIM.

Related papers

• Unity Forests: Improving Interaction Modelling and Interpretability in Random Forests [45.88028371034407]
  Random forests (RFs) are widely used for prediction and variable importance analysis.<n>In UFOs, the first few splits of each tree are optimized jointly.<n>UFOs achieved higher discrimination and predictive accuracy than standard RFs.
  arXiv  Detail & Related papers  (2026-01-11T17:36:58Z)
• Hierarchical Identity Learning for Unsupervised Visible-Infrared Person Re-Identification [81.3063589622217]
  Unsupervised visible-infrared person re-identification (USVI-ReID) aims to learn modality-invariant image features from unlabeled cross-modal person datasets.
  arXiv  Detail & Related papers  (2025-09-15T05:10:43Z)
• Class Similarity-Based Multimodal Classification under Heterogeneous Category Sets [22.03742325512164]
  We propose the practical setting termed Multi-Modal Heterogeneous Category-set Learning (MMHCL)<n>Our method significantly outperforms existing state-of-the-art approaches on multiple benchmark datasets.
  arXiv  Detail & Related papers  (2025-06-11T13:49:22Z)
• Multiclass threshold-based classification [2.66269503676104]
  We introduce a threshold-based framework for multiclass classification that generalizes the standard argmax rule.<n>We show that the multidimensional threshold-based setting yields consistent performance improvements across various networks and datasets.
  arXiv  Detail & Related papers  (2025-05-16T14:11:26Z)
• Multi-Prototype Embedding Refinement for Semi-Supervised Medical Image Segmentation [1.6683976936678229]
  We propose a Multi-Prototype-based Embedding Refinement method for semi-supervised medical image segmentation. Specifically, we design a multi-prototype-based classification strategy, rethinking the segmentation from the perspective of structural relationships between voxel embeddings. In the thorough evaluation on two popular benchmarks, our method achieves superior performance compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-18T15:23:52Z)
• Class-Independent Increment: An Efficient Approach for Multi-label Class-Incremental Learning [49.65841002338575]
  This paper focuses on the challenging yet practical multi-label class-incremental learning (MLCIL) problem. We propose a novel class-independent incremental network (CINet) to extract multiple class-level embeddings for multi-label samples. It learns and preserves the knowledge of different classes by constructing class-specific tokens.
  arXiv  Detail & Related papers  (2025-03-01T14:40:52Z)
• Balanced Classification: A Unified Framework for Long-Tailed Object Detection [74.94216414011326]
  Conventional detectors suffer from performance degradation when dealing with long-tailed data due to a classification bias towards the majority head categories. We introduce a unified framework called BAlanced CLassification (BACL), which enables adaptive rectification of inequalities caused by disparities in category distribution. BACL consistently achieves performance improvements across various datasets with different backbones and architectures.
  arXiv  Detail & Related papers  (2023-08-04T09:11:07Z)
• MHCCL: Masked Hierarchical Cluster-Wise Contrastive Learning for Multivariate Time Series [20.008535430484475]
  Masked Hierarchical Cluster-wise Contrastive Learning model is presented. It exploits semantic information obtained from the hierarchical structure consisting of multiple latent partitions for time series. It is shown to be superior to state-of-the-art approaches for unsupervised time series representation learning.
  arXiv  Detail & Related papers  (2022-12-02T12:42:53Z)
• Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning [79.83792914684985]
  We prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem.
  arXiv  Detail & Related papers  (2022-11-26T21:02:09Z)
• GFlowNets and variational inference [64.22223306224903]
  This paper builds bridges between two families of probabilistic algorithms:hierarchical variational inference (VI) and generative flow networks (GFlowNets) We demonstrate that, in certain cases, VI algorithms are equivalent to special cases of GFlowNets in the sense of equality of expected gradients of their learning objectives.
  arXiv  Detail & Related papers  (2022-10-02T17:41:01Z)
• Class-Incremental Lifelong Learning in Multi-Label Classification [3.711485819097916]
  This paper studies Lifelong Multi-Label (LML) classification, which builds an online class-incremental classifier in a sequential multi-label classification data stream. To solve the problem, the study proposes an Augmented Graph Convolutional Network (AGCN) with a built Augmented Correlation Matrix (ACM) across sequential partial-label tasks.
  arXiv  Detail & Related papers  (2022-07-16T05:14:07Z)
• Maximum Batch Frobenius Norm for Multi-Domain Text Classification [19.393393465837377]
  We propose a maximum batch Frobenius norm (MBF) method to boost the feature discriminability for multi-domain text classification. Experiments on two MDTC benchmarks show that our MBF approach can effectively advance the performance of the state-of-the-art.
  arXiv  Detail & Related papers  (2022-01-29T14:37:56Z)
• Gated recurrent units and temporal convolutional network for multilabel classification [122.84638446560663]
  This work proposes a new ensemble method for managing multilabel classification. The core of the proposed approach combines a set of gated recurrent units and temporal convolutional neural networks trained with variants of the Adam gradients optimization approach.
  arXiv  Detail & Related papers  (2021-10-09T00:00:16Z)
• Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation [75.93960390191262]
  We exploit prior knowledge of the relations among object categories to cluster fine-grained classes into coarser parent classes. We propose a simple yet effective resampling method, NMS Resampling, to re-balance the data distribution. Our method, termed as Forest R-CNN, can serve as a plug-and-play module being applied to most object recognition models.
  arXiv  Detail & Related papers  (2020-08-13T03:52:37Z)
• Nonparallel Hyperplane Classifiers for Multi-category Classification [0.3867363075280544]
  Nonparallel hyperplanes classification algorithms (NHCAs) have been proposed, which are comparable in terms of classification accuracy when compared with SVM. In this paper, we present a comparative study of four NHCAs i.e. Twin SVM (TWSVM), Generalized eigenvalue proximal SVM (GEPSVM), Regularized GEPSVM (RegGEPSVM) and Improved GEPSVM (IGEPSVM) for multi-category classification. The experimental results show that TDS-TWSVM outperforms other methods in terms of classification accuracy and BT-RegGEPSVM takes
  arXiv  Detail & Related papers  (2020-04-16T08:03:40Z)
• Adversarial Multi-Binary Neural Network for Multi-class Classification [19.298875915675502]
  We use a multi-task framework to address multi-class classification. We employ adversarial training to distinguish the class-specific features and the class-agnostic features.
  arXiv  Detail & Related papers  (2020-03-25T02:19:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.