Classification-Head Bias in Class-Level Machine Unlearning: Diagnosis, Mitigation, and Evaluation

• URL: http://arxiv.org/abs/2605.08730v1
• Date: Sat, 09 May 2026 06:33:42 GMT
• Title: Classification-Head Bias in Class-Level Machine Unlearning: Diagnosis, Mitigation, and Evaluation
• Abstract summary: Class-level machine unlearning aims to remove the influence of specified classes while preserving model utility on retained classes.<n>Existing methods are commonly evaluated by retain-set accuracy, forget-set accuracy, and unlearning time.<n>We reveal a bias-dominated shortcut in class-level unlearning: the prediction of forgotten classes can be suppressed by decreasing the corresponding bias terms in the final classification head.
• Score: 0.7991370731695461
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Class-level machine unlearning aims to remove the influence of specified classes while preserving model utility on retained classes. Existing methods are commonly evaluated by retain-set accuracy, forget-set accuracy, and unlearning time, but these metrics provide limited insight into how forgetting is achieved internally. In this paper, we reveal a bias-dominated shortcut in class-level unlearning: the prediction of forgotten classes can be suppressed by decreasing the corresponding bias terms in the final classification head. We first analyze the gradient dynamics of classification-head biases under softmax cross-entropy training, explaining why retain-set-only optimization tends to reduce the biases of absent classes. Based on this observation, we introduce BiasShift as a diagnostic baseline, showing that simple bias manipulation can satisfy conventional unlearning metrics while leaving abnormal bias patterns that reveal forgotten labels. To mitigate excessive forgotten-class bias suppression, we propose two bias-aware mechanisms, namely Two-Stage Bias Gradient Reversal Mechanism (TS-BGRM) and Lower-Bound Hinge Regularization (LB-HR). We further introduce three bias-oriented metrics, including Bias Stability Coefficient (BSC), Median Bias Gap (MBG), and Minimal Bias Score (MBS), to quantify bias dependence and potential leakage. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that the proposed methods maintain competitive unlearning performance while producing more stable bias distributions. We have released our code at {https://github.com/zwd2024/Beyond-the-Shadow-of-Bias-From-Classification-Head-Bias-to-Parameter-Redi stribution}.

Related papers

• Learning Dynamics of Logits Debiasing for Long-Tailed Semi-Supervised Learning [29.880846977690254]
  Long-tailed distributions are prevalent in real-world semi-supervised learning (SSL), where pseudo-labels tend to favor majority classes.<n>We revisit LTSSL through the lens of learning dynamics and provide a theoretical characterization of logits debiasing.<n>We propose DyTrim, a principle-based dynamic pruning framework that reallocates gradient budget through class-aware pruning on labeled data and confidence-based soft pruning on unlabeled data.
  arXiv  Detail & Related papers  (2026-08-31T12:38:45Z)
• Neutral-Reference Prompting for Vision-Language Models [14.563492336625004]
  We propose NeRP, a plug-and-play prompting correction strategy that improves discrimination on unseen classes without modifying model parameters.<n>We show that NeRP substantially improves accuracy on unseen classes while preserving known-class prediction performance.
  arXiv  Detail & Related papers  (2026-05-15T04:54:08Z)
• Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration [76.08899010904652]
  CapCal is a training-free framework that mechanically decouples positional bias from ranking decisions.<n>It achieves superior performance among training-free methods while preserving single-pass efficiency.
  arXiv  Detail & Related papers  (2026-04-11T10:47:22Z)
• Multi Attribute Bias Mitigation via Representation Learning [5.155573439009767]
  Real world images frequently exhibit multiple overlapping biases, including textures, watermarks, gendered makeup, scene object pairings, etc.<n>We tackle this multi bias problem with Generalized Multi Bias Mitigation (GMBM), a lean two stage framework that needs group labels only while training and minimizes bias at test time.<n>We validate GMBM on FB CMNIST, CelebA, and COCO, where we boost worst group accuracy, halve multi attribute bias amplification, and set a new low in SBA.
  arXiv  Detail & Related papers  (2025-09-03T18:08:59Z)
• LCGC: Learning from Consistency Gradient Conflicting for Class-Imbalanced Semi-Supervised Debiasing [7.868824589618853]
  We theoretically analyze why exploiting a baseline image can refine pseudo-labels and prove that the black image is the best choice.<n>We propose a debiasing scheme dubbed LCGC, which Learning from Gradient Conflicting, by encouraging biased class predictions.<n> LCGC can significantly improve the prediction accuracy of existing CISSL models on public benchmarks.
  arXiv  Detail & Related papers  (2025-04-09T02:57:53Z)
• ALBAR: Adversarial Learning approach to mitigate Biases in Action Recognition [52.537021302246664]
  Action recognition models often suffer from background bias (i.e., inferring actions based on background cues) and foreground bias (i.e., relying on subject appearance)<n>We propose ALBAR, a novel adversarial training method that mitigates foreground and background biases without requiring specialized knowledge of the bias attributes.<n>We evaluate our method on established background and foreground bias protocols, setting a new state-of-the-art and strongly improving combined debiasing performance by over 12% absolute on HMDB51.
  arXiv  Detail & Related papers  (2025-01-31T20:47:06Z)
• Sebra: Debiasing Through Self-Guided Bias Ranking [54.09529903433859]
  Ranking samples by fine-grained estimates of spuriosity has recently been shown to significantly benefit bias mitigation.<n>We propose a debiasing framework based on our novel ulSelf-Guided ulBias ulRanking (emphSebra)<n>Sebra mitigates biases via an automatic ranking of data points by spuriosity within their respective classes.
  arXiv  Detail & Related papers  (2025-01-30T11:31:38Z)
• Towards the Mitigation of Confirmation Bias in Semi-supervised Learning: a Debiased Training Perspective [6.164100243945264]
  Semi-supervised learning (SSL) commonly exhibits confirmation bias, where models disproportionately favor certain classes. We introduce TaMatch, a unified framework for debiased training in SSL. We show that TaMatch significantly outperforms existing state-of-the-art methods across a range of challenging image classification tasks.
  arXiv  Detail & Related papers  (2024-09-26T21:50:30Z)
• Class-Imbalanced Graph Learning without Class Rebalancing [62.1368829847041]
  Class imbalance is prevalent in real-world node classification tasks and poses great challenges for graph learning models. In this work, we approach the root cause of class-imbalance bias from an topological paradigm. We devise a lightweight topological augmentation framework BAT to mitigate the class-imbalance bias without class rebalancing.
  arXiv  Detail & Related papers  (2023-08-27T19:01:29Z)
• Self-supervised debiasing using low rank regularization [59.84695042540525]
  Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability. We propose a self-supervised debiasing framework potentially compatible with unlabeled samples. Remarkably, the proposed debiasing framework significantly improves the generalization performance of self-supervised learning baselines.
  arXiv  Detail & Related papers  (2022-10-11T08:26:19Z)
• Ensembling over Classifiers: a Bias-Variance Perspective [13.006468721874372]
  We build upon the extension to the bias-variance decomposition by Pfau (2013) in order to gain crucial insights into the behavior of ensembles of classifiers. We show that conditional estimates necessarily incur an irreducible error. Empirically, standard ensembling reducesthe bias, leading us to hypothesize that ensembles of classifiers may perform well in part because of this unexpected reduction.
  arXiv  Detail & Related papers  (2022-06-21T17:46:35Z)
• Relieving Long-tailed Instance Segmentation via Pairwise Class Balance [85.53585498649252]
  Long-tailed instance segmentation is a challenging task due to the extreme imbalance of training samples among classes. It causes severe biases of the head classes (with majority samples) against the tailed ones. We propose a novel Pairwise Class Balance (PCB) method, built upon a confusion matrix which is updated during training to accumulate the ongoing prediction preferences.
  arXiv  Detail & Related papers  (2022-01-08T07:48:36Z)
• Prototypical Classifier for Robust Class-Imbalanced Learning [64.96088324684683]
  We propose textitPrototypical, which does not require fitting additional parameters given the embedding network. Prototypical produces balanced and comparable predictions for all classes even though the training set is class-imbalanced. We test our method on CIFAR-10LT, CIFAR-100LT and Webvision datasets, observing that Prototypical obtains substaintial improvements compared with state of the arts.
  arXiv  Detail & Related papers  (2021-10-22T01:55:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.