On Aggregation in Ensembles of Multilabel Classifiers

• URL: http://arxiv.org/abs/2006.11916v1
• Date: Sun, 21 Jun 2020 21:43:24 GMT
• Title: On Aggregation in Ensembles of Multilabel Classifiers
• Authors: Vu-Linh Nguyen and Eyke H\"ullermeier and Michael Rapp and Eneldo Loza Menc\'ia and Johannes F\"urnkranz
• Abstract summary: "predict then combine" (PTC) and "combine then predict" (CTP) are two principal approaches to ensemble multilabel classification. We show that PTC is the better choice for non-decomposable losses.
• Score: 4.842945656927122
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While a variety of ensemble methods for multilabel classification have been proposed in the literature, the question of how to aggregate the predictions of the individual members of the ensemble has received little attention so far. In this paper, we introduce a formal framework of ensemble multilabel classification, in which we distinguish two principal approaches: "predict then combine" (PTC), where the ensemble members first make loss minimizing predictions which are subsequently combined, and "combine then predict" (CTP), which first aggregates information such as marginal label probabilities from the individual ensemble members, and then derives a prediction from this aggregation. While both approaches generalize voting techniques commonly used for multilabel ensembles, they allow to explicitly take the target performance measure into account. Therefore, concrete instantiations of CTP and PTC can be tailored to concrete loss functions. Experimentally, we show that standard voting techniques are indeed outperformed by suitable instantiations of CTP and PTC, and provide some evidence that CTP performs well for decomposable loss functions, whereas PTC is the better choice for non-decomposable losses.

Related papers

• Reliable Hierarchical Operating System Fingerprinting via Conformal Prediction [62.40452053128524]
  Conformal Prediction (CP) could be wrapped around existing methods to obtain prediction sets with guaranteed coverage.<n>This work addresses these limitations by introducing and evaluating two distinct structured CP strategies.<n>While both methods satisfy validity guarantees, they expose a fundamental trade-off between level-wise efficiency and structural consistency.
  arXiv  Detail & Related papers  (2026-02-13T11:20:48Z)
• Calibratable Disambiguation Loss for Multi-Instance Partial-Label Learning [53.9713678229744]
  Multi-instance partial-label learning (MIPL) is a weakly supervised framework that addresses the challenges of inexact supervision in both instance and label spaces.<n>Existing MIPL approaches often suffer from poor calibration, undermining reliability.<n>We propose a plug-and-play calibratable disambiguation loss (CDL) that simultaneously improves classification accuracy and calibration performance.
  arXiv  Detail & Related papers  (2025-12-19T16:58:31Z)
• Symmetric Aggregation of Conformity Scores for Efficient Uncertainty Sets [6.673032375204486]
  We propose SACP (Symmetric Aggregated Conformal Prediction), a novel method that aggregates nonconformity scores from multiple predictors.<n>SACP transforms these scores into e-values and combines them using any symmetric aggregation function.<n>We show that SACP consistently improves efficiency and often outperforms state-of-the-art model aggregation baselines.
  arXiv  Detail & Related papers  (2025-12-07T17:54:07Z)
• Enhancing Conformal Prediction via Class Similarity [13.38174941551702]
  Conformal Prediction (CP) has emerged as a powerful statistical framework for high-stakes classification applications.<n>We propose augmenting the CP score function with a term that penalizes predictions with out-of-group errors.<n>We show mathematically that, for common class partitions, it can also reduce the average set size of any CP score function.
  arXiv  Detail & Related papers  (2025-11-24T17:56:42Z)
• Rethinking Consistent Multi-Label Classification under Inexact Supervision [60.79309683889278]
  In partial multi-label learning, each instance is annotated with a candidate label set, among which only some labels are relevant.<n>In complementary multi-label learning, each instance is annotated with complementary labels indicating the classes to which the instance does not belong.
  arXiv  Detail & Related papers  (2025-10-05T08:30:32Z)
• Hierarchical Conformal Classification [5.964388602612373]
  Conformal prediction (CP) is a powerful framework for quantifying uncertainty in machine learning models.<n>Standard CP treats classes as flat and unstructured, ignoring relationships such as semantic or hierarchical structure among class labels.<n>This paper presents hierarchical HCCal classification (HCC), an extension of CP that incorporates class hierarchies into both the structure and semantics of prediction sets.
  arXiv  Detail & Related papers  (2025-08-18T18:05:55Z)
• Class conditional conformal prediction for multiple inputs by p-value aggregation [11.198836025239963]
  We introduce an innovative refinement to conformal prediction methods for classification tasks.<n>Our approach is motivated by applications in citizen science, where multiple images of the same plant or animal are captured by individuals.<n>We evaluate our method on simulated and real data, with a particular focus on Pl@ntNet, a prominent citizen science platform.
  arXiv  Detail & Related papers  (2025-07-09T09:17:17Z)
• Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness [53.96714099151378]
  We propose a three-step approach for parameter-efficient fine-tuning of image-text foundation models. Our method improves its two key components: minority samples identification and the robust training algorithm. Our theoretical analysis shows that our PPA enhances minority group identification and is Bayes optimal for minimizing the balanced group error.
  arXiv  Detail & Related papers  (2025-03-12T15:46:12Z)
• Multivariate Conformal Prediction using Optimal Transport [19.644272536912286]
  Conformal prediction (CP) quantifies the uncertainty of machine learning models. We propose a principled framework for constructing conformal prediction sets in multidimensional settings.
  arXiv  Detail & Related papers  (2025-02-05T20:56:41Z)
• Improved Diversity-Promoting Collaborative Metric Learning for Recommendation [127.08043409083687]
  Collaborative Metric Learning (CML) has recently emerged as a popular method in recommendation systems. This paper focuses on a challenging scenario where a user has multiple categories of interests. We propose a novel method called textitDiversity-Promoting Collaborative Metric Learning (DPCML)
  arXiv  Detail & Related papers  (2024-09-02T07:44:48Z)
• Adapting Conformal Prediction to Distribution Shifts Without Labels [16.478151550456804]
  Conformal prediction (CP) enables machine learning models to output prediction sets with guaranteed coverage rate. Our goal is to improve the quality of CP-generated prediction sets using only unlabeled data from the test domain. This is achieved by two new methods called ECP and EACP, that adjust the score function in CP according to the base model's uncertainty on the unlabeled test data.
  arXiv  Detail & Related papers  (2024-06-03T15:16:02Z)
• A structured regression approach for evaluating model performance across intersectional subgroups [53.91682617836498]
  Disaggregated evaluation is a central task in AI fairness assessment, where the goal is to measure an AI system's performance across different subgroups. We introduce a structured regression approach to disaggregated evaluation that we demonstrate can yield reliable system performance estimates even for very small subgroups.
  arXiv  Detail & Related papers  (2024-01-26T14:21:45Z)
• Well-calibrated Confidence Measures for Multi-label Text Classification with a Large Number of Labels [1.1833906227033337]
  We present a novel approach for addressing the computational inefficiency of the Label Powerset (LP) ICP, arrising when dealing with a high number of unique labels. We apply the LP-ICP on three deep Artificial Neural Network (ANN) classifiers of two types: one based on contextualised (bert) and two on non-contextualised (word2vec) word-embeddings. Our approach deals with the increased computational burden of LP by eliminating from consideration a significant number of label-sets that will surely have p-values below the specified significance level.
  arXiv  Detail & Related papers  (2023-12-14T19:17:42Z)
• A Universal Unbiased Method for Classification from Aggregate Observations [115.20235020903992]
  This paper presents a novel universal method of CFAO, which holds an unbiased estimator of the classification risk for arbitrary losses. Our proposed method not only guarantees the risk consistency due to the unbiased risk estimator but also can be compatible with arbitrary losses.
  arXiv  Detail & Related papers  (2023-06-20T07:22:01Z)
• Differentially Private Federated Clustering over Non-IID Data [59.611244450530315]
  clustering clusters (FedC) problem aims to accurately partition unlabeled data samples distributed over massive clients into finite clients under the orchestration of a server. We propose a novel FedC algorithm using differential privacy convergence technique, referred to as DP-Fed, in which partial participation and multiple clients are also considered. Various attributes of the proposed DP-Fed are obtained through theoretical analyses of privacy protection, especially for the case of non-identically and independently distributed (non-i.i.d.) data.
  arXiv  Detail & Related papers  (2023-01-03T05:38:43Z)
• Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction [76.26710990597498]
  We show that the class-imbalance of the grouped data from randomly selected clients can lead to significant performance degradation. Based on our key observation, we design an efficient client sampling mechanism, i.e., Federated Class-balanced Sampling (Fed-CBS) In particular, we propose a measure of class-imbalance and then employ homomorphic encryption to derive this measure in a privacy-preserving way.
  arXiv  Detail & Related papers  (2022-09-30T05:42:56Z)
• On-the-Fly Ensemble Pruning in Evolving Data Streams [4.137914981603379]
  CCRP is an on-the-fly ensemble prun-ing method for multi-class data stream classification. We show that different types of en-sembles that integrate CCRP consistently yield on par or superior performance with 20% to 90% less averagememory consumption.
  arXiv  Detail & Related papers  (2021-09-15T22:54:22Z)
• Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound [15.557653926558638]
  We investigate a counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. We instantiate it with Dirichlet distributions: this allows for a closed-form and differentiable expression for the expected risk. The resulting majority vote learning algorithm achieves state-of-the-art accuracy and benefits from (non-vacuous) tight bounds.
  arXiv  Detail & Related papers  (2021-06-23T16:57:23Z)
• Binary Classification from Multiple Unlabeled Datasets via Surrogate Set Classification [94.55805516167369]
  We propose a new approach for binary classification from m U-sets for $mge2$. Our key idea is to consider an auxiliary classification task called surrogate set classification (SSC)
  arXiv  Detail & Related papers  (2021-02-01T07:36:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.