System-Aware Unlearning Algorithms: Use Lesser, Forget Faster

• URL: http://arxiv.org/abs/2506.06073v1
• Date: Fri, 06 Jun 2025 13:30:40 GMT
• Title: System-Aware Unlearning Algorithms: Use Lesser, Forget Faster
• Authors: Linda Lu, Ayush Sekhari, Karthik Sridharan,
• Abstract summary: We present an exact system-aware unlearning algorithm for linear classification using a selective sampling-based approach.<n>We theoretically analyze the tradeoffs between deletion capacity, accuracy, memory, and computation time.
• Score: 15.783636887138904
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine unlearning addresses the problem of updating a machine learning model/system trained on a dataset $S$ so that the influence of a set of deletion requests $U \subseteq S$ on the unlearned model is minimized. The gold standard definition of unlearning demands that the updated model, after deletion, be nearly identical to the model obtained by retraining. This definition is designed for a worst-case attacker (one who can recover not only the unlearned model but also the remaining data samples, i.e., $S \setminus U$). Such a stringent definition has made developing efficient unlearning algorithms challenging. However, such strong attackers are also unrealistic. In this work, we propose a new definition, system-aware unlearning, which aims to provide unlearning guarantees against an attacker that can at best only gain access to the data stored in the system for learning/unlearning requests and not all of $S\setminus U$. With this new definition, we use the simple intuition that if a system can store less to make its learning/unlearning updates, it can be more secure and update more efficiently against a system-aware attacker. Towards that end, we present an exact system-aware unlearning algorithm for linear classification using a selective sampling-based approach, and we generalize the method for classification with general function classes. We theoretically analyze the tradeoffs between deletion capacity, accuracy, memory, and computation time.

Related papers

• Probably Approximately Precision and Recall Learning [62.912015491907994]
  Precision and Recall are foundational metrics in machine learning. One-sided feedback--where only positive examples are observed during training--is inherent in many practical problems. We introduce a PAC learning framework where each hypothesis is represented by a graph, with edges indicating positive interactions.
  arXiv  Detail & Related papers  (2024-11-20T04:21:07Z)
• LoRA Unlearns More and Retains More (Student Abstract) [0.0]
  PruneLoRA reduces the need for large-scale parameter updates by applying low-rank updates to the model. We leverage LoRA to selectively modify a subset of the pruned model's parameters, thereby reducing the computational cost, memory requirements and improving the model's ability to retain performance on the remaining classes.
  arXiv  Detail & Related papers  (2024-11-16T16:47:57Z)
• Attribute-to-Delete: Machine Unlearning via Datamodel Matching [65.13151619119782]
  Machine unlearning -- efficiently removing a small "forget set" training data on a pre-divertrained machine learning model -- has recently attracted interest. Recent research shows that machine unlearning techniques do not hold up in such a challenging setting.
  arXiv  Detail & Related papers  (2024-10-30T17:20:10Z)
• Hessian-Free Online Certified Unlearning [8.875278412741695]
  We develop an online unlearning algorithm that achieves near-instantaneous data removal.<n>We prove that our proposed method outperforms the state-of-the-art methods in terms of the unlearning and generalization guarantees.
  arXiv  Detail & Related papers  (2024-04-02T07:54:18Z)
• Loss-Free Machine Unlearning [51.34904967046097]
  We present a machine unlearning approach that is both retraining- and label-free. Retraining-free approaches often utilise Fisher information, which is derived from the loss and requires labelled data which may not be available. We present an extension to the Selective Synaptic Dampening algorithm, substituting the diagonal of the Fisher information matrix for the gradient of the l2 norm of the model output to approximate sensitivity.
  arXiv  Detail & Related papers  (2024-02-29T16:15:34Z)
• Discriminative Adversarial Unlearning [40.30974185546541]
  We introduce a novel machine unlearning framework founded upon the established principles of the min-max optimization paradigm. We capitalize on the capabilities of strong Membership Inference Attacks (MIA) to facilitate the unlearning of specific samples from a trained model. Our proposed algorithm closely approximates the ideal benchmark of retraining from scratch for both random sample forgetting and class-wise forgetting schemes.
  arXiv  Detail & Related papers  (2024-02-10T03:04:57Z)
• Deep Unlearning: Fast and Efficient Gradient-free Approach to Class Forgetting [9.91998873101083]
  We introduce a novel class unlearning algorithm designed to strategically eliminate specific classes from the learned model. Our algorithm exhibits competitive unlearning performance and resilience against Membership Inference Attacks (MIA)
  arXiv  Detail & Related papers  (2023-12-01T18:29:08Z)
• Verifiable and Provably Secure Machine Unlearning [44.142771334058715]
  Machine unlearning aims to remove points from the training dataset of a machine learning model after training.<n>We present the first cryptographic definition of verifiable unlearning to formally capture the guarantees of an unlearning system.<n>We implement the protocol for three different unlearning techniques and validate its feasibility for linear regression, logistic regression, and neural networks.
  arXiv  Detail & Related papers  (2022-10-17T14:19:52Z)
• Online Selective Classification with Limited Feedback [82.68009460301585]
  We study selective classification in the online learning model, wherein a predictor may abstain from classifying an instance. Two salient aspects of the setting we consider are that the data may be non-realisable, due to which abstention may be a valid long-term action. We construct simple versioning-based schemes for any $mu in (0,1],$ that make most $Tmu$ mistakes while incurring smash$tildeO(T1-mu)$ excess abstention against adaptive adversaries.
  arXiv  Detail & Related papers  (2021-10-27T08:00:53Z)
• Machine Unlearning of Features and Labels [72.81914952849334]
  We propose first scenarios for unlearning and labels in machine learning models. Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters.
  arXiv  Detail & Related papers  (2021-08-26T04:42:24Z)
• Fast Few-Shot Classification by Few-Iteration Meta-Learning [173.32497326674775]
  We introduce a fast optimization-based meta-learning method for few-shot classification. Our strategy enables important aspects of the base learner objective to be learned during meta-training. We perform a comprehensive experimental analysis, demonstrating the speed and effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-10-01T15:59:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.