Learning Dynamics of Logits Debiasing for Long-Tailed Semi-Supervised Learning

• URL: http://arxiv.org/abs/2608.30699v1
• Date: Mon, 31 Aug 2026 12:38:45 GMT
• Title: Learning Dynamics of Logits Debiasing for Long-Tailed Semi-Supervised Learning
• Abstract summary: Long-tailed distributions are prevalent in real-world semi-supervised learning (SSL), where pseudo-labels tend to favor majority classes.<n>We revisit LTSSL through the lens of learning dynamics and provide a theoretical characterization of logits debiasing.<n>We propose DyTrim, a principle-based dynamic pruning framework that reallocates gradient budget through class-aware pruning on labeled data and confidence-based soft pruning on unlabeled data.
• Score: 29.880846977690254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-tailed distributions are prevalent in real-world semi-supervised learning (SSL), where pseudo-labels tend to favor majority classes, leading to degraded generalization. While many long-tailed semi-supervised learning (LTSSL) methods have been proposed, the mechanisms by which they implicitly debias logits remain poorly understood. In this work, we revisit LTSSL through the lens of learning dynamics and provide a theoretical characterization of logits debiasing. Specifically, we derive a step-wise decomposition of the logits updates, showing that predictions are dominated by class-imbalance bias that reliably reflects label priors. To expose this effect, we use the logits of a task-irrelevant baseline image as an indicator of accumulated bias and prove that they converge to the class prior. This provides a unified view where LTSSL remedies such as logit adjustment, reweighting, and resampling correspond to reshaping gradient dynamics. Based on this insight, we propose DyTrim, a principle-based dynamic pruning framework that reallocates gradient budget through class-aware pruning on labeled data and confidence-based soft pruning on unlabeled data. We provide theoretical guarantees that DyTrim reduces class bias and improves generalization. Extensive experiments on standard LTSSL benchmarks show consistent gains across architectures and methods. Code available at: https://jiajun0425.github.io/DyTrim

Related papers

• Classification-Head Bias in Class-Level Machine Unlearning: Diagnosis, Mitigation, and Evaluation [0.7991370731695461]
  Class-level machine unlearning aims to remove the influence of specified classes while preserving model utility on retained classes.<n>Existing methods are commonly evaluated by retain-set accuracy, forget-set accuracy, and unlearning time.<n>We reveal a bias-dominated shortcut in class-level unlearning: the prediction of forgotten classes can be suppressed by decreasing the corresponding bias terms in the final classification head.
  arXiv  Detail & Related papers  (2026-05-09T06:33:42Z)
• On the Learning Dynamics of Two-layer Linear Networks with Label Noise SGD [93.70725920710208]
  We study the learning dynamics of gradient descent with label noise.<n>We highlight the critical role of label noise in driving the transition from the lazy to the rich regime.<n>We extend these insights to Sharpness-Aware Minimization (SAM), showing that the principles governing label noise SGD also apply to broader optimization algorithms.
  arXiv  Detail & Related papers  (2026-03-11T04:26:54Z)
• LCGC: Learning from Consistency Gradient Conflicting for Class-Imbalanced Semi-Supervised Debiasing [7.868824589618853]
  We theoretically analyze why exploiting a baseline image can refine pseudo-labels and prove that the black image is the best choice.<n>We propose a debiasing scheme dubbed LCGC, which Learning from Gradient Conflicting, by encouraging biased class predictions.<n> LCGC can significantly improve the prediction accuracy of existing CISSL models on public benchmarks.
  arXiv  Detail & Related papers  (2025-04-09T02:57:53Z)
• Sebra: Debiasing Through Self-Guided Bias Ranking [54.09529903433859]
  Ranking samples by fine-grained estimates of spuriosity has recently been shown to significantly benefit bias mitigation.<n>We propose a debiasing framework based on our novel ulSelf-Guided ulBias ulRanking (emphSebra)<n>Sebra mitigates biases via an automatic ranking of data points by spuriosity within their respective classes.
  arXiv  Detail & Related papers  (2025-01-30T11:31:38Z)
• Towards the Mitigation of Confirmation Bias in Semi-supervised Learning: a Debiased Training Perspective [6.164100243945264]
  Semi-supervised learning (SSL) commonly exhibits confirmation bias, where models disproportionately favor certain classes. We introduce TaMatch, a unified framework for debiased training in SSL. We show that TaMatch significantly outperforms existing state-of-the-art methods across a range of challenging image classification tasks.
  arXiv  Detail & Related papers  (2024-09-26T21:50:30Z)
• Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix It [6.19039575840278]
  Label smoothing (LS) is a popular regularisation method for training neural networks.<n>LS degrades uncertainty rank ordering of correct vs incorrect predictions.<n>We provide an explanation for this behaviour by analysing logit-level gradients.
  arXiv  Detail & Related papers  (2024-03-19T06:46:24Z)
• Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud Semantic Segmentation via Decoupling Optimization [64.36097398869774]
  Semi-supervised learning (SSL) has been an active research topic for large-scale 3D scene understanding. The existing SSL-based methods suffer from severe training bias due to class imbalance and long-tail distributions of the point cloud data. We introduce a new decoupling optimization framework, which disentangles feature representation learning and classifier in an alternative optimization manner to shift the bias decision boundary effectively.
  arXiv  Detail & Related papers  (2024-01-13T04:16:40Z)
• Relieving Long-tailed Instance Segmentation via Pairwise Class Balance [85.53585498649252]
  Long-tailed instance segmentation is a challenging task due to the extreme imbalance of training samples among classes. It causes severe biases of the head classes (with majority samples) against the tailed ones. We propose a novel Pairwise Class Balance (PCB) method, built upon a confusion matrix which is updated during training to accumulate the ongoing prediction preferences.
  arXiv  Detail & Related papers  (2022-01-08T07:48:36Z)
• Prototypical Classifier for Robust Class-Imbalanced Learning [64.96088324684683]
  We propose textitPrototypical, which does not require fitting additional parameters given the embedding network. Prototypical produces balanced and comparable predictions for all classes even though the training set is class-imbalanced. We test our method on CIFAR-10LT, CIFAR-100LT and Webvision datasets, observing that Prototypical obtains substaintial improvements compared with state of the arts.
  arXiv  Detail & Related papers  (2021-10-22T01:55:01Z)
• Self-supervised Learning is More Robust to Dataset Imbalance [65.84339596595383]
  We investigate self-supervised learning under dataset imbalance. Off-the-shelf self-supervised representations are already more robust to class imbalance than supervised representations. We devise a re-weighted regularization technique that consistently improves the SSL representation quality on imbalanced datasets.
  arXiv  Detail & Related papers  (2021-10-11T06:29:56Z)
• Class-incremental Learning with Rectified Feature-Graph Preservation [24.098892115785066]
  A central theme of this paper is to learn new classes that arrive in sequential phases over time. We propose a weighted-Euclidean regularization for old knowledge preservation. We show how it can work with binary cross-entropy to increase class separation for effective learning of new classes.
  arXiv  Detail & Related papers  (2020-12-15T07:26:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.