Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correction

• URL: http://arxiv.org/abs/2608.30046v1
• Date: Sun, 30 Aug 2026 21:16:29 GMT
• Title: Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correction
• Abstract summary: Noisy labels remain a critical challenge for training deep neural networks.<n>Standard solution is to retrain the model from scratch on the cleaned dataset.<n>Machine Unlearning has emerged as a computationally efficient alternative.
• Score: 0.5647577824219207
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Noisy labels remain a critical challenge for training deep neural networks, since memorizing incorrect labels degrades generalization. Once noisy samples are identified after training, the standard solution is to retrain the model from scratch on the cleaned dataset, which is increasingly expensive as datasets and models grow. Machine Unlearning (MU) has recently emerged as a computationally efficient alternative, but the relative effectiveness of different MU strategies for noisy-label correction remains poorly understood. In this work, we conduct a comparative empirical study of five MU methods (NegGrad, Fine-Tuning (FT), Random Labeling (RL), SalUn, and MUNBa) across symmetric, asymmetric, instance-dependent, and open-set noise on CIFAR-10, CIFAR-100, and the real-world noisy dataset Food-101N. Our central finding is that the appropriate unlearning strategy is conditioned on the noise structure. Simple FT is a strong baseline across most closed-set scenarios; RL and SalUn are the most consistently robust methods and, under instance-dependent noise, approach retraining accuracy at a fraction of the computational cost; MUNBa shows advantages mainly under extreme symmetric noise. Under open-set noise, in contrast, we show that retraining on the cleaned subset degrades accuracy relative to the noisy baseline, so approximating the retrained model is not an adequate objective in this regime. On Food-101N, all MU methods remain competitive and achieve accuracies close to retraining despite reducing runtime by an order of magnitude. These findings provide practical guidelines for selecting MU strategies for post-training noisy-label correction.

Related papers

• Pre-train to Gain: Robust Learning Without Clean Labels [1.1582652820340928]
  Training deep networks with noisy labels leads to poor generalization and degraded accuracy.<n>By pre-training a feature extractor backbone without labels, we can train a more noise robust model without requiring a subset with clean labels.<n>Our approach achieves comparable results to ImageNet pre-trained models at low noise levels, while substantially outperforming them under high noise conditions.
  arXiv  Detail & Related papers  (2025-11-25T20:48:07Z)
• Detect and Correct: A Selective Noise Correction Method for Learning with Noisy Labels [14.577138753507203]
  Falsely annotated samples, also known as noisy labels, can significantly harm the performance of deep learning models.<n>Two main approaches for learning with noisy labels are global noise estimation and data filtering.<n>Our method identifies potentially noisy samples based on their loss distribution.<n>We then apply a selection process to separate noisy and clean samples and learn a noise transition matrix to correct the loss for noisy samples while leaving the clean data unaffected.
  arXiv  Detail & Related papers  (2025-05-19T16:49:27Z)
• Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach [102.0769560460338]
  We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class. Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
  arXiv  Detail & Related papers  (2024-03-01T03:27:08Z)
• Combating Label Noise With A General Surrogate Model For Sample Selection [77.45468386115306]
  We propose to leverage the vision-language surrogate model CLIP to filter noisy samples automatically.<n>We validate the effectiveness of our proposed method on both real-world and synthetic noisy datasets.
  arXiv  Detail & Related papers  (2023-10-16T14:43:27Z)
• Latent Class-Conditional Noise Model [54.56899309997246]
  We introduce a Latent Class-Conditional Noise model (LCCN) to parameterize the noise transition under a Bayesian framework. We then deduce a dynamic label regression method for LCCN, whose Gibbs sampler allows us efficiently infer the latent true labels. Our approach safeguards the stable update of the noise transition, which avoids previous arbitrarily tuning from a mini-batch of samples.
  arXiv  Detail & Related papers  (2023-02-19T15:24:37Z)
• Class Prototype-based Cleaner for Label Noise Learning [73.007001454085]
  Semi-supervised learning methods are current SOTA solutions to the noisy-label learning problem. We propose a simple yet effective solution, named textbfClass textbfPrototype-based label noise textbfCleaner.
  arXiv  Detail & Related papers  (2022-12-21T04:56:41Z)
• UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning [89.56465237941013]
  We propose UNICON, a simple yet effective sample selection method which is robust to high label noise. We obtain an 11.4% improvement over the current state-of-the-art on CIFAR100 dataset with a 90% noise rate.
  arXiv  Detail & Related papers  (2022-03-28T07:36:36Z)
• Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model [80.91927573604438]
  This paper proposes a simple yet universal probabilistic model, which explicitly relates noisy labels to their instances. Experiments on datasets with both synthetic and real-world label noise verify that the proposed method yields significant improvements on robustness.
  arXiv  Detail & Related papers  (2021-01-14T05:43:51Z)
• Identifying Training Stop Point with Noisy Labeled Data [0.0]
  We develop an algorithm to find a training stop point (TSP) at or close to test accuracy (MOTA) We validated the robustness of our algorithm (AutoTSP) through several experiments on CIFAR-10, CIFAR-100, and a real-world noisy dataset.
  arXiv  Detail & Related papers  (2020-12-24T20:07:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.