Class-Balanced Softmax: A Bayes Theory-Based Method for Long-Tailed Recognition

• URL: http://arxiv.org/abs/2607.22258v1
• Date: Fri, 24 Jul 2026 12:46:03 GMT
• Title: Class-Balanced Softmax: A Bayes Theory-Based Method for Long-Tailed Recognition
• Abstract summary: Class-Balanced Softmax (CBS) is a simple logit adjustment that is computationally inexpensive and easily integrated into existing pipelines.<n>CBS is highly scalable and outperforms existing methods, including Balanced Softmax.
• Score: 16.988043430434065
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning models using traditional softmax classifiers have achieved remarkable success in various classification tasks. However, their performance degrades significantly on imbalanced datasets. Although Balanced Softmax is widely adopted as a state-of-the-art rebalancing method, it possesses inherent limitations, such as yielding disproportionately lower testing accuracy for tail classes. To mitigate these shortcomings, we propose the Class-Balanced Softmax (CBS). Rooted in a theoretical Bayesian framework and a heuristic power-law assumption, the CBS is a simple logit adjustment that is computationally inexpensive and easily integrated into existing pipelines. Furthermore, we characterise a fundamental phenomenon in models trained on imbalanced data, termed the preference issue, wherein models exhibit higher training error and a larger generalisation gap for classes with limited data. To quantify this issue, we introduce a novel metric and demonstrate that CBS effectively mitigates the preference issue. Extensive experiments on large-scale benchmarks show that CBS is highly scalable and outperforms existing methods, including Balanced Softmax.

Related papers

• Explaining Data Mixing Scaling Laws [2.1378946778058796]
  We propose a unified framework to explain the underlying mechanics of data mixing.<n>We identify two key factors that govern the domain losses of models trained on different data mixtures.<n>Our model successfully extrapolates across scales, predicting highly effective mixtures for large, unseen scales.
  arXiv  Detail & Related papers  (2026-06-06T13:31:38Z)
• Online Bayesian Imbalanced Learning with Bregman-Calibrated Deep Networks [0.7106986689736825]
  We present textitOnline Bayesian Imbalanced Learning (OBIL), a principled framework that decouples likelihood-ratio estimation from class-prior assumptions.<n>Our approach builds on the established connection between Bregman divergences and proper scoring rules to show that deep networks trained with such losses produce posterior probability estimates.<n>We prove that these likelihood-ratio estimates remain valid under arbitrary changes in class priors and cost structures, requiring only a threshold adjustment for optimal Bayes decisions.
  arXiv  Detail & Related papers  (2026-02-08T21:23:00Z)
• Reducing Class-Wise Performance Disparity via Margin Regularization [82.81746960548382]
  Deep neural networks often exhibit substantial disparities in class-wise accuracy, even when trained on class-balanced data.<n>We present Margin Regularization for Performance Disparity Reduction (MR$2$), a theoretically principled regularization for classification.<n>Our analysis reveals how per-class feature variability contributes to error, motivating the use of larger margins for hard classes.
  arXiv  Detail & Related papers  (2026-01-30T12:56:08Z)
• BAPE: Learning an Explicit Bayes Classifier for Long-tailed Visual Recognition [78.70453964041718]
  Current deep learning algorithms usually solve for the optimal classifier by emphimplicitly estimating the posterior probabilities.<n>This simple methodology has been proven effective for meticulously balanced academic benchmark datasets.<n>However, it is not applicable to the long-tailed data distributions in the real world.<n>This paper presents a novel approach (BAPE) that provides a more precise theoretical estimation of the data distributions.
  arXiv  Detail & Related papers  (2025-06-29T15:12:50Z)
• BMFT: Achieving Fairness via Bias-based Weight Masking Fine-tuning [17.857930204697983]
  Bias-based Weight Masking Fine-Tuning (BMFT) is a novel post-processing method that enhances the fairness of a trained model in significantly fewer epochs. BMFT produces a mask over model parameters, which efficiently identifies the weights contributing the most towards biased predictions. Experiments across four dermatological datasets and two sensitive attributes demonstrate that BMFT outperforms existing state-of-the-art (SOTA) techniques in both diagnostic accuracy and fairness metrics.
  arXiv  Detail & Related papers  (2024-08-13T13:36:48Z)
• Bias Mitigating Few-Shot Class-Incremental Learning [17.185744533050116]
  Few-shot class-incremental learning aims at recognizing novel classes continually with limited novel class samples. Recent methods somewhat alleviate the accuracy imbalance between base and incremental classes by fine-tuning the feature extractor in the incremental sessions. We propose a novel method to mitigate model bias of the FSCIL problem during training and inference processes.
  arXiv  Detail & Related papers  (2024-02-01T10:37:41Z)
• Online Continual Learning via Logit Adjusted Softmax [24.327176079085703]
  Inter-class imbalance during training has been identified as a major cause of forgetting. We present a simple adjustment of model logits during training can effectively resist prior class bias. Our proposed method, Logit Adjusted Softmax, can mitigate the impact of inter-class imbalance not only in class-incremental but also in realistic general setups.
  arXiv  Detail & Related papers  (2023-11-11T03:03:33Z)
• An Embarrassingly Simple Baseline for Imbalanced Semi-Supervised Learning [103.65758569417702]
  Semi-supervised learning (SSL) has shown great promise in leveraging unlabeled data to improve model performance. We consider a more realistic and challenging setting called imbalanced SSL, where imbalanced class distributions occur in both labeled and unlabeled data. We study a simple yet overlooked baseline -- SimiS -- which tackles data imbalance by simply supplementing labeled data with pseudo-labels.
  arXiv  Detail & Related papers  (2022-11-20T21:18:41Z)
• MaxMatch: Semi-Supervised Learning with Worst-Case Consistency [149.03760479533855]
  We propose a worst-case consistency regularization technique for semi-supervised learning (SSL) We present a generalization bound for SSL consisting of the empirical loss terms observed on labeled and unlabeled training data separately. Motivated by this bound, we derive an SSL objective that minimizes the largest inconsistency between an original unlabeled sample and its multiple augmented variants.
  arXiv  Detail & Related papers  (2022-09-26T12:04:49Z)
• Prototype-Anchored Learning for Learning with Imperfect Annotations [83.7763875464011]
  It is challenging to learn unbiased classification models from imperfectly annotated datasets. We propose a prototype-anchored learning (PAL) method, which can be easily incorporated into various learning-based classification schemes. We verify the effectiveness of PAL on class-imbalanced learning and noise-tolerant learning by extensive experiments on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-06-23T10:25:37Z)
• MixKD: Towards Efficient Distillation of Large-scale Language Models [129.73786264834894]
  We propose MixKD, a data-agnostic distillation framework, to endow the resulting model with stronger generalization ability. We prove from a theoretical perspective that under reasonable conditions MixKD gives rise to a smaller gap between the error and the empirical error. Experiments under a limited-data setting and ablation studies further demonstrate the advantages of the proposed approach.
  arXiv  Detail & Related papers  (2020-11-01T18:47:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.