Designing to Forget: Deep Semi-parametric Models for Unlearning

• URL: http://arxiv.org/abs/2603.22870v1
• Date: Tue, 24 Mar 2026 07:13:45 GMT
• Title: Designing to Forget: Deep Semi-parametric Models for Unlearning
• Abstract summary: We introduce a family of deep semi-parametric models (SPMs) that exhibit non-parametric behavior during unlearning.<n>SPMs achieve competitive task performance to parametric models in image classification and generation, while being significantly more efficient for unlearning.
• Score: 27.50875933694437
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in machine unlearning have focused on developing algorithms to remove specific training samples from a trained model. In contrast, we observe that not all models are equally easy to unlearn. Hence, we introduce a family of deep semi-parametric models (SPMs) that exhibit non-parametric behavior during unlearning. SPMs use a fusion module that aggregates information from each training sample, enabling explicit test-time deletion of selected samples without altering model parameters. Empirically, we demonstrate that SPMs achieve competitive task performance to parametric models in image classification and generation, while being significantly more efficient for unlearning. Notably, on ImageNet classification, SPMs reduce the prediction gap relative to a retrained (oracle) baseline by $11\%$ and achieve over $10\times$ faster unlearning compared to existing approaches on parametric models. The code is available at https://github.com/amberyzheng/spm_unlearning.

Related papers

• Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering [51.7496756448709]
  Language models (LMs) perform well on coding benchmarks but struggle with real-world software engineering tasks.<n>Existing approaches rely on supervised fine-tuning with high-quality data, which is expensive to curate at scale.<n>We propose Test-Time Scaling (EvoScale), a sample-efficient method that treats generation as an evolutionary process.
  arXiv  Detail & Related papers  (2025-05-29T16:15:36Z)
• AMUN: Adversarial Machine UNlearning [13.776549741449557]
  Adversarial Machine UNlearning (AMUN) outperforms prior state-of-the-art (SOTA) methods for image classification.<n>AMUN lowers the confidence of the model on the forget samples by fine-tuning the model on their corresponding adversarial examples.
  arXiv  Detail & Related papers  (2025-03-02T14:36:31Z)
• Realistic Image-to-Image Machine Unlearning via Decoupling and Knowledge Retention [1.795561427808824]
  We argue that the machine learning model performs fairly well on unseen data.<n>We propose a framework which decouples the model parameters with gradient ascent.<n>We also provide $(epsilon, delta)$-unlearning guarantee for model updates with gradient ascent.
  arXiv  Detail & Related papers  (2025-02-06T17:46:49Z)
• Attribute-to-Delete: Machine Unlearning via Datamodel Matching [65.13151619119782]
  Machine unlearning -- efficiently removing a small "forget set" training data on a pre-divertrained machine learning model -- has recently attracted interest. Recent research shows that machine unlearning techniques do not hold up in such a challenging setting.
  arXiv  Detail & Related papers  (2024-10-30T17:20:10Z)
• Truncated Consistency Models [57.50243901368328]
  Training consistency models requires learning to map all intermediate points along PF ODE trajectories to their corresponding endpoints.<n>We empirically find that this training paradigm limits the one-step generation performance of consistency models.<n>We propose a new parameterization of the consistency function and a two-stage training procedure that prevents the truncated-time training from collapsing to a trivial solution.
  arXiv  Detail & Related papers  (2024-10-18T22:38:08Z)
• MUSO: Achieving Exact Machine Unlearning in Over-Parameterized Regimes [19.664090734076712]
  Machine unlearning (MU) makes a well-trained model behave as if it had never been trained on specific data.<n>We propose an alternating optimization algorithm that unifies the tasks of unlearning and relabeling.<n>The algorithm's effectiveness, confirmed through numerical experiments, highlights its superior performance in unlearning across various scenarios.
  arXiv  Detail & Related papers  (2024-10-11T06:17:17Z)
• EMR-Merging: Tuning-Free High-Performance Model Merging [55.03509900949149]
  We show that Elect, Mask & Rescale-Merging (EMR-Merging) shows outstanding performance compared to existing merging methods. EMR-Merging is tuning-free, thus requiring no data availability or any additional training while showing impressive performance.
  arXiv  Detail & Related papers  (2024-05-23T05:25:45Z)
• Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection [56.292071534857946]
  Recent data-privacy laws have sparked interest in machine unlearning. Challenge is to discard information about the forget'' data without altering knowledge about remaining dataset. We adopt a projected-gradient based learning method, named as Projected-Gradient Unlearning (PGU) We provide empirically evidence to demonstrate that our unlearning method can produce models that behave similar to models retrained from scratch across various metrics even when the training dataset is no longer accessible.
  arXiv  Detail & Related papers  (2023-12-07T07:17:24Z)
• In-Context Unlearning: Language Models as Few Shot Unlearners [27.962361828354716]
  We propose a new class of unlearning methods for Large Language Models (LLMs) This method unlearns instances from the model by simply providing specific kinds of inputs in context, without the need to update model parameters. Our experimental results demonstrate that in-context unlearning performs on par with, or in some cases outperforms other state-of-the-art methods that require access to model parameters.
  arXiv  Detail & Related papers  (2023-10-11T15:19:31Z)
• Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models [76.410400238974]
  We propose TTA with feedback to rectify the model output and prevent the model from becoming blindly confident. A CLIP model is adopted as the reward model during TTA and provides feedback for the VLM. The proposed textitreinforcement learning with CLIP feedback(RLCF) framework is highly flexible and universal.
  arXiv  Detail & Related papers  (2023-05-29T11:03:59Z)
• Self-Damaging Contrastive Learning [92.34124578823977]
  Unlabeled data in reality is commonly imbalanced and shows a long-tail distribution. This paper proposes a principled framework called Self-Damaging Contrastive Learning to automatically balance the representation learning without knowing the classes. Our experiments show that SDCLR significantly improves not only overall accuracies but also balancedness.
  arXiv  Detail & Related papers  (2021-06-06T00:04:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.