Train Once, Forget Precisely: Anchored Optimization for Efficient Post-Hoc Unlearning

• URL: http://arxiv.org/abs/2506.14515v1
• Date: Tue, 17 Jun 2025 13:40:48 GMT
• Title: Train Once, Forget Precisely: Anchored Optimization for Efficient Post-Hoc Unlearning
• Authors: Prabhav Sanga, Jaskaran Singh, Arun K. Dubey,
• Abstract summary: We introduce textbfForget-Aligned Model Reconstruction (FAMR), a theoretically grounded and computationally efficient framework for post-hoc unlearning in deep image classifiers.<n>FAMR frames forgetting as a constrained optimization problem that minimizes a uniformprediction loss on the forget set while anchoring model parameters to their original values.<n> Empirical results on class forgetting tasks using CIFAR-10 and ImageNet-100 FAMR's effectiveness, with strong performance retention and minimal computational overhead.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As machine learning systems increasingly rely on data subject to privacy regulation, selectively unlearning specific information from trained models has become essential. In image classification, this involves removing the influence of particular training samples, semantic classes, or visual styles without full retraining. We introduce \textbf{Forget-Aligned Model Reconstruction (FAMR)}, a theoretically grounded and computationally efficient framework for post-hoc unlearning in deep image classifiers. FAMR frames forgetting as a constrained optimization problem that minimizes a uniform-prediction loss on the forget set while anchoring model parameters to their original values via an $\ell_2$ penalty. A theoretical analysis links FAMR's solution to influence-function-based retraining approximations, with bounds on parameter and output deviation. Empirical results on class forgetting tasks using CIFAR-10 and ImageNet-100 demonstrate FAMR's effectiveness, with strong performance retention and minimal computational overhead. The framework generalizes naturally to concept and style erasure, offering a scalable and certifiable route to efficient post-hoc forgetting in vision models.

Related papers

• Efficient Machine Unlearning via Influence Approximation [75.31015485113993]
  Influence-based unlearning has emerged as a prominent approach to estimate the impact of individual training samples on model parameters without retraining.<n>This paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning)<n>We introduce the Influence Approximation Unlearning algorithm for efficient machine unlearning from the incremental perspective.
  arXiv  Detail & Related papers  (2025-07-31T05:34:27Z)
• Pre-training for Recommendation Unlearning [14.514770044236375]
  UnlearnRec is a model-agnostic pre-training paradigm that prepares systems for efficient unlearning operations.<n>Our method delivers exceptional unlearning effectiveness while providing more than 10x speedup compared to retraining approaches.
  arXiv  Detail & Related papers  (2025-05-28T17:57:11Z)
• Enhancing Training Data Attribution with Representational Optimization [57.61977909113113]
  Training data attribution methods aim to measure how training data impacts a model's predictions.<n>We propose AirRep, a representation-based approach that closes this gap by learning task-specific and model-aligned representations explicitly for TDA.<n>AirRep introduces two key innovations: a trainable encoder tuned for attribution quality, and an attention-based pooling mechanism that enables accurate estimation of group-wise influence.
  arXiv  Detail & Related papers  (2025-05-24T05:17:53Z)
• Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing [58.52119063742121]
  Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance.<n>This paper addresses the question of how to optimally combine the model's predictions and the provided labels.<n>Our main contribution is the derivation of the Bayes optimal aggregator function to combine the current model's predictions and the given labels.
  arXiv  Detail & Related papers  (2025-05-21T07:16:44Z)
• The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models [69.798277882245]
  We introduce Unsupervised Prefix Fine-Tuning (UPFT) to enhance large language models' reasoning efficiency.<n>UPFT removes the need for labeled data or exhaustive sampling.<n> Experiments show that UPFT matches the performance of supervised methods.
  arXiv  Detail & Related papers  (2025-03-04T18:56:03Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• $\nabla τ$: Gradient-based and Task-Agnostic machine Unlearning [7.04736023670375]
  We introduce Gradient-based and Task-Agnostic machine Unlearning ($nabla tau$) $nabla tau$ applies adaptive gradient ascent to the data to be forgotten while using standard gradient descent for the remaining data. We evaluate our framework's effectiveness using a set of well-established Membership Inference Attack metrics.
  arXiv  Detail & Related papers  (2024-03-21T12:11:26Z)
• Recommendation Unlearning via Influence Function [42.4931807753579]
  We propose a new Influence Function-based Recommendation Unlearning (IFRU) framework, which efficiently updates the model without retraining. IFRU achieves more than 250 times acceleration compared to retraining-based methods with recommendation performance comparable to full retraining.
  arXiv  Detail & Related papers  (2023-07-05T09:42:51Z)
• Harnessing the Power of Explanations for Incremental Training: A LIME-Based Approach [6.244905619201076]
  In this work, model explanations are fed back to the feed-forward training to help the model generalize better. The framework incorporates the custom weighted loss with Elastic Weight Consolidation (EWC) to maintain performance in sequential testing sets. The proposed custom training procedure results in a consistent enhancement of accuracy ranging from 0.5% to 1.5% throughout all phases of the incremental learning setup.
  arXiv  Detail & Related papers  (2022-11-02T18:16:17Z)
• Fine-grained Retrieval Prompt Tuning [149.9071858259279]
  Fine-grained Retrieval Prompt Tuning steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompt and feature adaptation. Our FRPT with fewer learnable parameters achieves the state-of-the-art performance on three widely-used fine-grained datasets.
  arXiv  Detail & Related papers  (2022-07-29T04:10:04Z)
• AttentionLite: Towards Efficient Self-Attention Models for Vision [9.957033392865982]
  We propose a novel framework for producing a class of parameter and compute efficient models called AttentionLitesuitable for resource-constrained applications. We can simultaneously distill knowledge from a compute-heavy teacher while also pruning the student model in a single pass of training.
  arXiv  Detail & Related papers  (2020-12-21T17:54:09Z)
• Extrapolation for Large-batch Training in Deep Learning [72.61259487233214]
  We show that a host of variations can be covered in a unified framework that we propose. We prove the convergence of this novel scheme and rigorously evaluate its empirical performance on ResNet, LSTM, and Transformer.
  arXiv  Detail & Related papers  (2020-06-10T08:22:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.