A Robust Certified Machine Unlearning Method Under Distribution Shift

• URL: http://arxiv.org/abs/2601.06967v1
• Date: Sun, 11 Jan 2026 15:53:19 GMT
• Title: A Robust Certified Machine Unlearning Method Under Distribution Shift
• Authors: Jinduo Guo, Yinzhi Cao,
• Abstract summary: We show that certified unlearning with the Newton method becomes inefficient and ineffective under non-i.i.d. unlearning sets.<n>We propose a better certified unlearning approach by performing a distribution-aware certified unlearning framework.
• Score: 14.904880015049208
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Newton method has been widely adopted to achieve certified unlearning. A critical assumption in existing approaches is that the data requested for unlearning are selected i.i.d.(independent and identically distributed). However,the problem of certified unlearning under non-i.i.d. deletions remains largely unexplored. In practice, unlearning requests are inherently biased, leading to non-i.i.d. deletions and causing distribution shifts between the original and retained datasets. In this paper, we show that certified unlearning with the Newton method becomes inefficient and ineffective under non-i.i.d. unlearning sets. We then propose a better certified unlearning approach by performing a distribution-aware certified unlearning framework based on iterative Newton updates constrained by a trust region. Our method provides a closer approximation to the retrained model and yields a tighter pre-run bound on the gradient residual, thereby ensuring efficient (epsilon, delta)-certified unlearning. To demonstrate its practical effectiveness under distribution shift, we also conduct extensive experiments across multiple evaluation metrics, providing a comprehensive assessment of our approach.

Related papers

• Certified Unlearning in Decentralized Federated Learning [24.229643475639293]
  In decentralized federated learning (DFL), clients exchange local updates only with neighbors, causing model information to propagate and mix across the network.<n>We propose a novel certified unlearning framework for DFL based on Newton-style updates.
  arXiv  Detail & Related papers  (2026-01-10T05:39:24Z)
• Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check [60.77691669644931]
  We propose Functional Alignment for Distributional Equivalence (FADE), a novel metric that measures distributional similarity between unlearned and reference models.<n>We show that FADE captures functional alignment across the entire output distribution, providing a principled assessment of genuine unlearning.<n>These findings expose fundamental gaps in current evaluation practices and demonstrate that FADE provides a more robust foundation for developing and assessing truly effective unlearning methods.
  arXiv  Detail & Related papers  (2025-10-14T20:50:30Z)
• PRUNE: A Patching Based Repair Framework for Certifiable Unlearning of Neural Networks [6.251501206858703]
  It is desirable to remove (a.k.a. unlearn) a specific part of the training data from a trained neural network model.<n>Existing unlearning methods involve training alternative models with remaining data.<n>We propose a novel unlearning approach by imposing carefully crafted "patch" on the original neural network to achieve targeted "forgetting" of the requested data to delete.
  arXiv  Detail & Related papers  (2025-05-10T05:35:08Z)
• Redefining Machine Unlearning: A Conformal Prediction-Motivated Approach [11.609354498110358]
  Machine unlearning seeks to remove the influence of specified data from a trained model.<n>In this paper, we find that the data misclassified across UA and MIA still have their ground truth labels included in the prediction set.<n>We propose two novel metrics inspired by conformal prediction that more reliably evaluate forgetting quality.
  arXiv  Detail & Related papers  (2025-01-31T18:58:43Z)
• Towards Effective Evaluations and Comparisons for LLM Unlearning Methods [97.2995389188179]
  This paper seeks to refine the evaluation of machine unlearning for large language models.<n>It addresses two key challenges -- the robustness of evaluation metrics and the trade-offs between competing goals.
  arXiv  Detail & Related papers  (2024-06-13T14:41:00Z)
• Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach [102.0769560460338]
  We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class. Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
  arXiv  Detail & Related papers  (2024-03-01T03:27:08Z)
• Class-wise Federated Unlearning: Harnessing Active Forgetting with Teacher-Student Memory Generation [11.638683787598817]
  We propose a neuro-inspired federated unlearning framework based on active forgetting.<n>Our framework distinguishes itself from existing methods by utilizing new memories to overwrite old ones.<n>Our method achieves satisfactory unlearning completeness against backdoor attacks.
  arXiv  Detail & Related papers  (2023-07-07T03:07:26Z)
• Model Sparsity Can Simplify Machine Unlearning [33.18951938708467]
  In response to recent data regulation requirements, machine unlearning (MU) has emerged as a critical process. Our study introduces a novel model-based perspective: model sparsification via weight pruning. We show in both theory and practice that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner.
  arXiv  Detail & Related papers  (2023-04-11T02:12:02Z)
• Post-hoc Uncertainty Learning using a Dirichlet Meta-Model [28.522673618527417]
  We propose a novel Bayesian meta-model to augment pre-trained models with better uncertainty quantification abilities. Our proposed method requires no additional training data and is flexible enough to quantify different uncertainties. We demonstrate our proposed meta-model approach's flexibility and superior empirical performance on these applications.
  arXiv  Detail & Related papers  (2022-12-14T17:34:11Z)
• MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation [132.77005365032468]
  We propose a novel framework of Model-Agnostic Counterfactual Explanation (MACE) In our MACE approach, we propose a novel RL-based method for finding good counterfactual examples and a gradient-less descent method for improving proximity. Experiments on public datasets validate the effectiveness with better validity, sparsity and proximity.
  arXiv  Detail & Related papers  (2022-05-31T04:57:06Z)
• Agree to Disagree: Diversity through Disagreement for Better Transferability [54.308327969778155]
  We propose D-BAT (Diversity-By-disAgreement Training), which enforces agreement among the models on the training data. We show how D-BAT naturally emerges from the notion of generalized discrepancy.
  arXiv  Detail & Related papers  (2022-02-09T12:03:02Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)
• Learning the Truth From Only One Side of the Story [58.65439277460011]
  We focus on generalized linear models and show that without adjusting for this sampling bias, the model may converge suboptimally or even fail to converge to the optimal solution. We propose an adaptive approach that comes with theoretical guarantees and show that it outperforms several existing methods empirically.
  arXiv  Detail & Related papers  (2020-06-08T18:20:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.