Machine Unlearning under Retain-Forget Entanglement

• URL: http://arxiv.org/abs/2603.26569v1
• Date: Fri, 27 Mar 2026 16:32:09 GMT
• Title: Machine Unlearning under Retain-Forget Entanglement
• Abstract summary: We propose a novel framework to handle retai-forget entanglements in machine unlearning.<n>In the first phase, an augmented Lagrangian method increases the loss on the forget set while preserving accuracy on less-related retained samples.<n>The second phase applies a projection step, regularized by the Wasserstein-2 distance, to mitigate performance degradation on semantically related retained samples.
• Score: 30.3424578412571
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Forgetting a subset in machine unlearning is rarely an isolated task. Often, retained samples that are closely related to the forget set can be unintentionally affected, particularly when they share correlated features from pretraining or exhibit strong semantic similarities. To address this challenge, we propose a novel two-phase optimization framework specifically designed to handle such retai-forget entanglements. In the first phase, an augmented Lagrangian method increases the loss on the forget set while preserving accuracy on less-related retained samples. The second phase applies a gradient projection step, regularized by the Wasserstein-2 distance, to mitigate performance degradation on semantically related retained samples without compromising the unlearning objective. We validate our approach through comprehensive experiments on multiple unlearning tasks, standard benchmark datasets, and diverse neural architectures, demonstrating that it achieves effective and reliable unlearning while outperforming existing baselines in both accuracy retention and removal fidelity.

Related papers

• OFMU: Optimization-Driven Framework for Machine Unlearning [5.100622189286672]
  Large language models increasingly require the ability to unlearn specific knowledge, such as user requests, copyrighted materials, or outdated information.<n>We propose OFMU, a penalty-based bi-level optimization framework that explicitly prioritizes forgetting while preserving retention.<n>We show that OFMU consistently outperforms existing unlearning methods in both efficacy and retained utility.
  arXiv  Detail & Related papers  (2025-09-26T15:31:32Z)
• WSS-CL: Weight Saliency Soft-Guided Contrastive Learning for Efficient Machine Unlearning Image Classification [0.0]
  We introduce a new two-phase efficient machine unlearning method for image classification, in terms of weight saliency.<n>Our method is called weight saliency soft-guided contrastive learning for efficient machine unlearning image classification (WSS-CL)<n>Our proposed method yields much-improved unlearning efficacy with negligible performance loss compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-08-06T10:47:36Z)
• Zero-Shot Class Unlearning in CLIP with Synthetic Samples [0.0]
  We focus on unlearning within CLIP, a dual vision-language model trained on a massive dataset of image-text pairs.<n>We apply Lipschitz regularization to the multimodal context of CLIP.<n>Our forgetting procedure is iterative, where we track accuracy on a synthetic forget set and stop when accuracy falls below a chosen threshold.
  arXiv  Detail & Related papers  (2024-07-10T09:16:14Z)
• Certified Machine Unlearning via Noisy Stochastic Gradient Descent [20.546589699647416]
  Machine unlearning aims to efficiently remove the effect of certain data points on the trained model.<n>We propose to leverage noisy gradient descent for unlearning and establish its first approximate unlearning guarantee.
  arXiv  Detail & Related papers  (2024-03-25T18:43:58Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Primal Dual Continual Learning: Balancing Stability and Plasticity through Adaptive Memory Allocation [86.8475564814154]
  We show that it is both possible and beneficial to undertake the constrained optimization problem directly. We focus on memory-based methods, where a small subset of samples from previous tasks can be stored in a replay buffer. We show that dual variables indicate the sensitivity of the optimal value of the continual learning problem with respect to constraint perturbations.
  arXiv  Detail & Related papers  (2023-09-29T21:23:27Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning [59.44422468242455]
  We propose a novel method dubbed ShrinkMatch to learn uncertain samples. For each uncertain sample, it adaptively seeks a shrunk class space, which merely contains the original top-1 class. We then impose a consistency regularization between a pair of strongly and weakly augmented samples in the shrunk space to strive for discriminative representations.
  arXiv  Detail & Related papers  (2023-08-13T14:05:24Z)
• Adaptive Cross Batch Normalization for Metric Learning [75.91093210956116]
  Metric learning is a fundamental problem in computer vision. We show that it is equally important to ensure that the accumulated embeddings are up to date. In particular, it is necessary to circumvent the representational drift between the accumulated embeddings and the feature embeddings at the current training iteration.
  arXiv  Detail & Related papers  (2023-03-30T03:22:52Z)
• A Framework using Contrastive Learning for Classification with Noisy Labels [1.2891210250935146]
  We propose a framework using contrastive learning as a pre-training task to perform image classification in the presence of noisy labels. Recent strategies such as pseudo-labeling, sample selection with Gaussian Mixture models, weighted supervised contrastive learning have been combined into a fine-tuning phase following the pre-training.
  arXiv  Detail & Related papers  (2021-04-19T18:51:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.