AdaCC: Cumulative Cost-Sensitive Boosting for Imbalanced Classification

• URL: http://arxiv.org/abs/2209.08309v1
• Date: Sat, 17 Sep 2022 10:46:19 GMT
• Title: AdaCC: Cumulative Cost-Sensitive Boosting for Imbalanced Classification
• Abstract summary: We propose a novel cost-sensitive boosting approach for imbalanced data that dynamically adjusts the misclassification costs over the boosting rounds. Our method, called AdaCC, is parameter-free as it relies on the cumulative behavior of the boosting model.
• Score: 40.942008530628016
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Class imbalance poses a major challenge for machine learning as most supervised learning models might exhibit bias towards the majority class and under-perform in the minority class. Cost-sensitive learning tackles this problem by treating the classes differently, formulated typically via a user-defined fixed misclassification cost matrix provided as input to the learner. Such parameter tuning is a challenging task that requires domain knowledge and moreover, wrong adjustments might lead to overall predictive performance deterioration. In this work, we propose a novel cost-sensitive boosting approach for imbalanced data that dynamically adjusts the misclassification costs over the boosting rounds in response to model's performance instead of using a fixed misclassification cost matrix. Our method, called AdaCC, is parameter-free as it relies on the cumulative behavior of the boosting model in order to adjust the misclassification costs for the next boosting round and comes with theoretical guarantees regarding the training error. Experiments on 27 real-world datasets from different domains with high class imbalance demonstrate the superiority of our method over 12 state-of-the-art cost-sensitive boosting approaches exhibiting consistent improvements in different measures, for instance, in the range of [0.3%-28.56%] for AUC, [3.4%-21.4%] for balanced accuracy, [4.8%-45%] for gmean and [7.4%-85.5%] for recall.

Related papers

• Reducing Class-Wise Performance Disparity via Margin Regularization [82.81746960548382]
  Deep neural networks often exhibit substantial disparities in class-wise accuracy, even when trained on class-balanced data.<n>We present Margin Regularization for Performance Disparity Reduction (MR$2$), a theoretically principled regularization for classification.<n>Our analysis reveals how per-class feature variability contributes to error, motivating the use of larger margins for hard classes.
  arXiv  Detail & Related papers  (2026-01-30T12:56:08Z)
• Covariance-corrected Whitening Alleviates Network Degeneration on Imbalanced Classification [6.197116272789107]
  Class imbalance is a critical issue in image classification that significantly affects the performance of deep recognition models. We propose a novel framework called Whitening-Net to mitigate the degenerate solutions. In scenarios with extreme class imbalance, the batch covariance statistic exhibits significant fluctuations, impeding the convergence of the whitening operation.
  arXiv  Detail & Related papers  (2024-08-30T10:49:33Z)
• Electroencephalogram Emotion Recognition via AUC Maximization [0.0]
  Imbalanced datasets pose significant challenges in areas including neuroscience, cognitive science, and medical diagnostics. This study addresses the issue class imbalance, using the Liking' label in the DEAP dataset as an example.
  arXiv  Detail & Related papers  (2024-08-16T19:08:27Z)
• OCCAM: Towards Cost-Efficient and Accuracy-Aware Classification Inference [11.267210747162961]
  We propose a principled approach, OCCAM, to compute the best classifier assignment strategy over classification queries. On a variety of real-world datasets, OCCAM achieves 40% cost reduction with little to no accuracy drop.
  arXiv  Detail & Related papers  (2024-06-06T21:05:39Z)
• Optimizing Class-Level Probability Reweighting Coefficients for Equitable Prompting Accuracy [12.287692969438169]
  LLMs often uncover biases from pre-training data's statistical regularities.<n>This leads to persistent, uneven class accuracy in classification and QA.<n>We develop a post-hoc probability reweighting method that directly optimize for non-differentiable performance-driven metrics.
  arXiv  Detail & Related papers  (2024-05-13T10:30:33Z)
• Class-attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective [54.33066660817495]
  Modern classification problems exhibit heterogeneities across individual classes. We propose CAP: An effective and general method that generates a class-specific learning strategy. We show that CAP is competitive with prior art and its flexibility unlocks clear benefits for fairness objectives beyond balanced accuracy.
  arXiv  Detail & Related papers  (2024-01-25T17:43:39Z)
• Class Balanced Dynamic Acquisition for Domain Adaptive Semantic Segmentation using Active Learning [3.175227858236288]
  Domain adaptive active learning is leading the charge in label-efficient training of neural networks. For semantic segmentation, state-of-the-art models jointly use two criteria of uncertainty and diversity to select training labels. We show that such methods currently suffer from a class imbalance issue which degrades their performance for larger active learning budgets.
  arXiv  Detail & Related papers  (2023-11-23T18:35:26Z)
• Online Continual Learning via Logit Adjusted Softmax [24.327176079085703]
  Inter-class imbalance during training has been identified as a major cause of forgetting. We present a simple adjustment of model logits during training can effectively resist prior class bias. Our proposed method, Logit Adjusted Softmax, can mitigate the impact of inter-class imbalance not only in class-incremental but also in realistic general setups.
  arXiv  Detail & Related papers  (2023-11-11T03:03:33Z)
• Uncertainty-guided Boundary Learning for Imbalanced Social Event Detection [64.4350027428928]
  We propose a novel uncertainty-guided class imbalance learning framework for imbalanced social event detection tasks. Our model significantly improves social event representation and classification tasks in almost all classes, especially those uncertain ones.
  arXiv  Detail & Related papers  (2023-10-30T03:32:04Z)
• Class-Imbalanced Graph Learning without Class Rebalancing [62.1368829847041]
  Class imbalance is prevalent in real-world node classification tasks and poses great challenges for graph learning models. In this work, we approach the root cause of class-imbalance bias from an topological paradigm. We devise a lightweight topological augmentation framework BAT to mitigate the class-imbalance bias without class rebalancing.
  arXiv  Detail & Related papers  (2023-08-27T19:01:29Z)
• CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning [55.733193075728096]
  Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance. Sample re-weighting methods are popularly used to alleviate this data bias issue. We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
  arXiv  Detail & Related papers  (2022-02-11T13:49:51Z)
• Adaptive Cost-Sensitive Learning in Neural Networks for Misclassification Cost Problems [2.8935588665357077]
  We design a new adaptive learning algorithm for misclassification cost problems. Our algorithm bridges the difference between the class distributions between subgroups of samples in the training and test data sets with similar predicted probabilities. We present empirical evidence that a deep neural network used with the proposed algorithm yields better cost results on several binary classification data sets that have class-imbalanced and class-balanced distributions compared to other alternative approaches.
  arXiv  Detail & Related papers  (2021-11-14T16:19:11Z)
• Learning with Multiclass AUC: Theory and Algorithms [141.63211412386283]
  Area under the ROC curve (AUC) is a well-known ranking metric for problems such as imbalanced learning and recommender systems. In this paper, we start an early trial to consider the problem of learning multiclass scoring functions via optimizing multiclass AUC metrics.
  arXiv  Detail & Related papers  (2021-07-28T05:18:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.