Rethinking Machine Unlearning: Models Designed to Forget via Key Deletion

• URL: http://arxiv.org/abs/2603.15033v2
• Date: Tue, 24 Mar 2026 11:06:34 GMT
• Title: Rethinking Machine Unlearning: Models Designed to Forget via Key Deletion
• Abstract summary: We propose unlearning by design, a novel paradigm in which models are directly trained to support forgetting.<n>We instantiate this idea with Machine UNlearning via KEY deletion (MUNKEY), a memory augmented transformer that decouples instance-specific memorization from model weights.<n>Our results establish that unlearning by design enables fast, deployment-oriented unlearning while preserving predictive performance.
• Score: 18.166109788922153
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine unlearning is rapidly becoming a practical requirement, driven by privacy regulations, data errors, and the need to remove harmful or corrupted training samples. Despite this, most existing methods tackle the problem purely from a post-hoc perspective. They attempt to erase the influence of targeted training samples through parameter updates that typically require access to the full training data. This creates a mismatch with real deployment scenarios where unlearning requests can be anticipated, revealing a fundamental limitation of post-hoc approaches. We propose unlearning by design, a novel paradigm in which models are directly trained to support forgetting as an inherent capability. We instantiate this idea with Machine UNlearning via KEY deletion (MUNKEY), a memory augmented transformer that decouples instance-specific memorization from model weights. Here, unlearning corresponds to removing the instance-identifying key, enabling direct zero-shot forgetting without weight updates or access to the original samples or labels. Across natural image benchmarks, fine-grained recognition, and medical datasets, MUNKEY outperforms all post-hoc baselines. Our results establish that unlearning by design enables fast, deployment-oriented unlearning while preserving predictive performance.

Related papers

• Not Every Subject Should Stay: Machine Unlearning for Noisy Engagement Recognition [53.005382593686356]
  Engagement recognition datasets are typically subject-indexed and often contain noisy, subjective supervision.<n>We study this setting through subject-level machine unlearning as a post-hoc sanitization mechanism for engagement recognition.
  arXiv  Detail & Related papers  (2026-05-06T10:03:06Z)
• Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMs [54.167494079321465]
  Current unlearning methods for LLMs optimize on the private information they seek to remove by incorporating it into their fine-tuning data.<n>We propose a novel unlearning method-Partial Model Collapse (PMC), which does not require unlearning targets in the unlearning objective.
  arXiv  Detail & Related papers  (2025-07-06T03:08:49Z)
• NOVO: Unlearning-Compliant Vision Transformers [17.810044173023474]
  pname can perform unlearning for future unlearning requests without any fine-tuning over the requested set.<n>Forgetting is achieved by withdrawing keys, making unlearning on-the-fly and avoiding performance degradation.
  arXiv  Detail & Related papers  (2025-07-04T04:12:34Z)
• FUNU: Boosting Machine Unlearning Efficiency by Filtering Unnecessary Unlearning [9.472692023087223]
  We propose FUNU, a method to identify data points that lead to unnecessary unlearning.<n>We provide a theoretical analysis of FUNU and conduct extensive experiments to validate its efficacy.
  arXiv  Detail & Related papers  (2025-01-28T01:19:07Z)
• Single-weight Model Editing for Post-hoc Spurious Correlation Neutralization [54.8794775172033]
  Neural network training tends to exploit the simplest features as shortcuts to greedily minimize training loss.<n>Some of these features might be spuriously correlated with the target labels, leading to incorrect predictions by the model.<n>We propose a unique precise class removal technique that makes a single-weight modification, which entails negligible performance compromise.
  arXiv  Detail & Related papers  (2025-01-24T02:22:42Z)
• An Information Theoretic Approach to Machine Unlearning [43.423418819707784]
  To comply with AI and data regulations, the need to forget private or copyrighted information from trained machine learning models is increasingly important.<n>In this work, we address the zero-shot unlearning scenario, whereby an unlearning algorithm must be able to remove data given only a trained model and the data to be forgotten.<n>We derive a simple but principled zero-shot unlearning method based on the geometry of the model.
  arXiv  Detail & Related papers  (2024-02-02T13:33:30Z)
• Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening [51.34904967046097]
  Selective Synaptic Dampening (SSD) is a fast, performant, and does not require long-term storage of the training data. We present a novel two-step, post hoc, retrain-free approach to machine unlearning which is fast, performant, and does not require long-term storage of the training data.
  arXiv  Detail & Related papers  (2023-08-15T11:30:45Z)
• Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers [71.70205894168039]
  We consider instance-wise unlearning, of which the goal is to delete information on a set of instances from a pre-trained model. We propose two methods that reduce forgetting on the remaining data: 1) utilizing adversarial examples to overcome forgetting at the representation-level and 2) leveraging weight importance metrics to pinpoint network parameters guilty of propagating unwanted information.
  arXiv  Detail & Related papers  (2023-01-27T07:53:50Z)
• Zero-Shot Machine Unlearning [6.884272840652062]
  Modern privacy regulations grant citizens the right to be forgotten by products, services and companies. No data related to the training process or training samples may be accessible for the unlearning purpose. We propose two novel solutions for zero-shot machine unlearning based on (a) error minimizing-maximizing noise and (b) gated knowledge transfer.
  arXiv  Detail & Related papers  (2022-01-14T19:16:09Z)
• Machine Unlearning of Features and Labels [72.81914952849334]
  We propose first scenarios for unlearning and labels in machine learning models. Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters.
  arXiv  Detail & Related papers  (2021-08-26T04:42:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.