Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors

• URL: http://arxiv.org/abs/2608.16700v2
• Date: Mon, 24 Aug 2026 10:14:23 GMT
• Title: Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors
• Abstract summary: We introduce the first learning-based model-agnostic approach, Learning-to-UnLearn (L2UL)<n>Our core insight is to shift from manually designing $U$ to learning the unlearning behaviors from a distribution perspective.<n>Our experimental results demonstrate that the accuracy achieved by L2UL is comparable to that of retraining.
• Score: 17.009892363181567
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Various machine unlearning techniques have been developed in response to privacy legislation requirements, enabling individuals to exercise their legal right to have their data $D_f$ removed from a machine learning model. This process is typically accomplished via the use of an unlearning function denoted as $U$. Existing methods focus on designing an intricate $U$ to unlearn $D_f \subset D$ from a previous model $A(D)$, so that the unlearned model performs as closely as possible to the retrained model $A(D \setminus D_f)$. However, these methods often suffer from high computational costs when dealing with massive training data, as the complex structures of $U$ become a bottleneck even for models with fewer parameters. Inspired by Learning to Optimize, we introduce the first learning-based model-agnostic approach, Learning-to-UnLearn (L2UL). Our core insight is to shift from manually designing $U$ to learning the unlearning behaviors from a distribution perspective, thereby acquiring a simple and efficient $U$ via learning. Our experimental results demonstrate that the accuracy achieved by L2UL is comparable to that of retraining while exhibiting impressive efficiency, particularly in data-intensive scenarios. Furthermore, we validate the performance and scalability of our method on larger models ResNet.

Related papers

• Sample Complexity of Transfer Learning: An Optimal Transport Approach [18.537610792819176]
  In this work, we rigorously analyze the potential benefit of transfer learning in terms of sample efficiency.<n>We find that when the data dimension $d$ is higher than $3$, the sample complexity for transfer learning is $O(m-(+1)/d)$.<n>Our finding theoretically supports a better sample efficiency for transfer learning, when the target task is optimizing over a family of not-so-smooth models.
  arXiv  Detail & Related papers  (2026-05-19T22:39:03Z)
• Efficient Machine Unlearning via Influence Approximation [75.31015485113993]
  Influence-based unlearning has emerged as a prominent approach to estimate the impact of individual training samples on model parameters without retraining.<n>This paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning)<n>We introduce the Influence Approximation Unlearning algorithm for efficient machine unlearning from the incremental perspective.
  arXiv  Detail & Related papers  (2025-07-31T05:34:27Z)
• Intention-Conditioned Flow Occupancy Models [80.42634994902858]
  Large-scale pre-training has fundamentally changed how machine learning research is done today.<n>Applying this same framework to reinforcement learning is appealing because it offers compelling avenues for addressing core challenges in RL.<n>Recent advances in generative AI have provided new tools for modeling highly complex distributions.
  arXiv  Detail & Related papers  (2025-06-10T15:27:46Z)
• S$^2$R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning [51.84977135926156]
  We introduce S$2$R, an efficient framework that enhances LLM reasoning by teaching models to self-verify and self-correct during inference.<n>Our results demonstrate that Qwen2.5-math-7B achieves an accuracy improvement from 51.0% to 81.6%, outperforming models trained on an equivalent amount of long-CoT distilled data.
  arXiv  Detail & Related papers  (2025-02-18T13:40:22Z)
• $\ abla τ$: Gradient-based and Task-Agnostic machine Unlearning [7.04736023670375]
  We introduce Gradient-based and Task-Agnostic machine Unlearning ($nabla tau$) $nabla tau$ applies adaptive gradient ascent to the data to be forgotten while using standard gradient descent for the remaining data. We evaluate our framework's effectiveness using a set of well-established Membership Inference Attack metrics.
  arXiv  Detail & Related papers  (2024-03-21T12:11:26Z)
• Fast Exact Unlearning for In-Context Learning Data for LLMs [30.06631665962119]
  We show that for large language models we can efficiently unlearn "fine-tuning data"<n>We show that accurate in-context learning can be done with quantized k-means, which allows for effectively constant time unlearning operations.
  arXiv  Detail & Related papers  (2024-02-01T16:43:04Z)
• Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection [56.292071534857946]
  Recent data-privacy laws have sparked interest in machine unlearning. Challenge is to discard information about the forget'' data without altering knowledge about remaining dataset. We adopt a projected-gradient based learning method, named as Projected-Gradient Unlearning (PGU) We provide empirically evidence to demonstrate that our unlearning method can produce models that behave similar to models retrained from scratch across various metrics even when the training dataset is no longer accessible.
  arXiv  Detail & Related papers  (2023-12-07T07:17:24Z)
• Model Sparsity Can Simplify Machine Unlearning [33.18951938708467]
  In response to recent data regulation requirements, machine unlearning (MU) has emerged as a critical process. Our study introduces a novel model-based perspective: model sparsification via weight pruning. We show in both theory and practice that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner.
  arXiv  Detail & Related papers  (2023-04-11T02:12:02Z)
• Datamodels: Predicting Predictions from Training Data [86.66720175866415]
  We present a conceptual framework, datamodeling, for analyzing the behavior of a model class in terms of the training data. We show that even simple linear datamodels can successfully predict model outputs.
  arXiv  Detail & Related papers  (2022-02-01T18:15:24Z)
• LeanML: A Design Pattern To Slash Avoidable Wastes in Machine Learning Projects [0.5330240017302619]
  We introduce the first application of the lean methodology to machine learning projects. We argue that lean machine learning (LeanML) can drastically slash avoidable wastes in commercial machine learning projects.
  arXiv  Detail & Related papers  (2021-07-16T18:16:48Z)
• Improving Robustness and Generality of NLP Models Using Disentangled Representations [62.08794500431367]
  Supervised neural networks first map an input $x$ to a single representation $z$, and then map $z$ to the output label $y$. We present methods to improve robustness and generality of NLP models from the standpoint of disentangled representation learning. We show that models trained with the proposed criteria provide better robustness and domain adaptation ability in a wide range of supervised learning tasks.
  arXiv  Detail & Related papers  (2020-09-21T02:48:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.