Exact Unlearning from Proxies Induces Closeness Guarantees on Approximate Unlearning

• URL: http://arxiv.org/abs/2605.10680v1
• Date: Mon, 11 May 2026 14:57:31 GMT
• Title: Exact Unlearning from Proxies Induces Closeness Guarantees on Approximate Unlearning
• Abstract summary: This paper proposes a paradigm shift linking machine unlearning directly to the structure of the data distributions.<n>We show that inferring these distributions with precision enables distilling the exact unlearning signal induced by the modeling.
• Score: 21.011990634407834
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes a paradigm shift linking machine unlearning directly to the structure of the data distributions rather than a mere update of the neural network parameters. We show that inferring these distributions with precision enables distilling the exact unlearning signal induced by the modeling. Theoretical bounds on the Kullback-Leibler divergence from the ideal retrained model to our unlearned model, under verifiable admissibility criterion, reveal the soundness of our framework. This method is experimentally validated over three forgetting scenarios as reaching the closest classifier to the ideal retrained model when compared to competitors.

Related papers

• Temper-Then-Tilt: Principled Unlearning for Generative Models through Tempering and Classifier Guidance [51.532841645285835]
  We study machine unlearning in large generative models by framing the task as density ratio estimation to a target distribution.<n>We show it can fail to faithfully unlearn with finite samples when the forget set represents a sharp, concentrated data distribution.<n>We introduce Temper-Then-Tilt Unlearning (T3-Unlearning), which freezes the base model and applies a two-step inference procedure.
  arXiv  Detail & Related papers  (2026-02-10T19:08:40Z)
• Toward Reliable Machine Unlearning: Theory, Algorithms, and Evaluation [1.7767466724342065]
  We introduce Adrial Machine UNlearning (AMUN), which surpasses prior state-of-the-art methods for image classification based on SOTA MIA scores.<n>We show that existing methods fail in replicating a retrained model's behavior by introducing a nearest-neighbor membership inference attack (MIA-NN)<n>We then propose a fine-tuning objective that mitigates this leakage by approximating, for forget-class inputs, the distribution over remaining classes that a model retrained from scratch would produce.
  arXiv  Detail & Related papers  (2025-12-07T20:57:25Z)
• Improving Local Fidelity Through Sampling and Modeling Nonlinearity [3.7080015862513847]
  Local Interpretable Model-agnostic Explanation (LIME) assumes that the local decision boundary is linear and fails to capture the non-linear relationships.<n>We propose a novel method that can generate high-fidelity explanations.
  arXiv  Detail & Related papers  (2025-12-05T09:26:18Z)
• Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check [60.77691669644931]
  We propose Functional Alignment for Distributional Equivalence (FADE), a novel metric that measures distributional similarity between unlearned and reference models.<n>We show that FADE captures functional alignment across the entire output distribution, providing a principled assessment of genuine unlearning.<n>These findings expose fundamental gaps in current evaluation practices and demonstrate that FADE provides a more robust foundation for developing and assessing truly effective unlearning methods.
  arXiv  Detail & Related papers  (2025-10-14T20:50:30Z)
• A recursive Bayesian neural network for constitutive modeling of sands under monotonic and cyclic loading [0.0]
  In engineering, models are central to capturing soil behavior across diverse drainage conditions, stress paths,and loading histories.<n>This study introduces a recursive Bayesian neural network (rBNN) framework that unifies temporal sequence learning with generalized inference.<n>The framework is validated against four datasets spanning both simulated and experimental triaxial tests.
  arXiv  Detail & Related papers  (2025-01-17T10:15:03Z)
• A Diffusion Model Framework for Unsupervised Neural Combinatorial Optimization [23.972397132797116]
  Current deep learning approaches rely on generative models that yield exact sample likelihoods.<n>This work introduces a method that lifts this restriction and opens the possibility to employ highly expressive latent variable models.<n>We experimentally validate our approach in data-free Combinatorial Optimization and demonstrate that our method achieves a new state-of-the-art on a wide range of benchmark problems.
  arXiv  Detail & Related papers  (2024-06-03T17:55:02Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning [123.3472310767721]
  Prediction credibility measures are fundamental in statistics and machine learning. These measures should account for the wide variety of models used in practice. The framework developed in this work expresses the credibility as a risk-fit trade-off.
  arXiv  Detail & Related papers  (2020-11-24T19:52:38Z)
• Adversarial Robustness of Supervised Sparse Coding [34.94566482399662]
  We consider a model that involves learning a representation while at the same time giving a precise generalization bound and a robustness certificate. We focus on the hypothesis class obtained by combining a sparsity-promoting encoder coupled with a linear encoder. We provide a robustness certificate for end-to-end classification.
  arXiv  Detail & Related papers  (2020-10-22T22:05:21Z)
• Control as Hybrid Inference [62.997667081978825]
  We present an implementation of CHI which naturally mediates the balance between iterative and amortised inference. We verify the scalability of our algorithm on a continuous control benchmark, demonstrating that it outperforms strong model-free and model-based baselines.
  arXiv  Detail & Related papers  (2020-07-11T19:44:09Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.