Soft Weighted Machine Unlearning

• URL: http://arxiv.org/abs/2505.18783v1
• Date: Sat, 24 May 2025 16:40:14 GMT
• Title: Soft Weighted Machine Unlearning
• Authors: Xinbao Qiao, Ningning Ding, Yushi Cheng, Meng Zhang,
• Abstract summary: We introduce a weighted influence function that assigns tailored weights to each sample by solving a convex quadratic programming problem analytically.<n>We demonstrate that the proposed soft-weighted scheme is versatile and can be seamlessly integrated into most existing unlearning algorithms.<n>In fairness- and robustness-driven tasks, the soft-weighted scheme significantly outperforms hard-weighted schemes in fairness/robustness metrics.
• Score: 7.696293975773435
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine unlearning, as a post-hoc processing technique, has gained widespread adoption in addressing challenges like bias mitigation and robustness enhancement, colloquially, machine unlearning for fairness and robustness. However, existing non-privacy unlearning-based solutions persist in using binary data removal framework designed for privacy-driven motivation, leading to significant information loss, a phenomenon known as over-unlearning. While over-unlearning has been largely described in many studies as primarily causing utility degradation, we investigate its fundamental causes and provide deeper insights in this work through counterfactual leave-one-out analysis. In this paper, we introduce a weighted influence function that assigns tailored weights to each sample by solving a convex quadratic programming problem analytically. Building on this, we propose a soft-weighted framework enabling fine-grained model adjustments to address the over-unlearning challenge. We demonstrate that the proposed soft-weighted scheme is versatile and can be seamlessly integrated into most existing unlearning algorithms. Extensive experiments show that in fairness- and robustness-driven tasks, the soft-weighted scheme significantly outperforms hard-weighted schemes in fairness/robustness metrics and alleviates the decline in utility metric, thereby enhancing machine unlearning algorithm as an effective correction solution.

Related papers

• WSS-CL: Weight Saliency Soft-Guided Contrastive Learning for Efficient Machine Unlearning Image Classification [0.0]
  We introduce a new two-phase efficient machine unlearning method for image classification, in terms of weight saliency.<n>Our method is called weight saliency soft-guided contrastive learning for efficient machine unlearning image classification (WSS-CL)<n>Our proposed method yields much-improved unlearning efficacy with negligible performance loss compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-08-06T10:47:36Z)
• Efficient Machine Unlearning via Influence Approximation [75.31015485113993]
  Influence-based unlearning has emerged as a prominent approach to estimate the impact of individual training samples on model parameters without retraining.<n>This paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning)<n>We introduce the Influence Approximation Unlearning algorithm for efficient machine unlearning from the incremental perspective.
  arXiv  Detail & Related papers  (2025-07-31T05:34:27Z)
• Does Machine Unlearning Truly Remove Model Knowledge? A Framework for Auditing Unlearning in LLMs [58.24692529185971]
  We introduce a comprehensive auditing framework for unlearning evaluation comprising three benchmark datasets, six unlearning algorithms, and five prompt-based auditing methods.<n>We evaluate the effectiveness and robustness of different unlearning strategies.
  arXiv  Detail & Related papers  (2025-05-29T09:19:07Z)
• Unlocking the Potential of Difficulty Prior in RL-based Multimodal Reasoning [69.64809103333839]
  We investigate how explicitly modeling problem's difficulty prior information shapes the effectiveness of reinforcement learning based fine-tuning for multimodal reasoning.<n>Our approach demonstrates significant performances across various multi-modal mathematical reasoning benchmarks with only 2K+0.6K two-stage training data.
  arXiv  Detail & Related papers  (2025-05-19T15:43:10Z)
• Instance-Level Difficulty: A Missing Perspective in Machine Unlearning [13.052520843129363]
  We study the cruxes that make machine unlearning difficult through a thorough instance-level unlearning performance analysis.<n>In particular, we summarize four factors that make unlearning a data point difficult.<n>We argue that machine unlearning research should pay attention to the instance-level difficulty of unlearning.
  arXiv  Detail & Related papers  (2024-10-03T23:41:42Z)
• Silver Linings in the Shadows: Harnessing Membership Inference for Machine Unlearning [7.557226714828334]
  We present a novel unlearning mechanism designed to remove the impact of specific data samples from a neural network. In achieving this goal, we crafted a novel loss function tailored to eliminate privacy-sensitive information from weights and activation values of the target model. Our results showcase the superior performance of our approach in terms of unlearning efficacy and latency as well as the fidelity of the primary task.
  arXiv  Detail & Related papers  (2024-07-01T00:20:26Z)
• Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEE [68.6018458996143]
  We propose a more general dynamic network that can combine both quantization and early exit dynamic network: QuEE. Our algorithm can be seen as a form of soft early exiting or input-dependent compression. The crucial factor of our approach is accurate prediction of the potential accuracy improvement achievable through further computation.
  arXiv  Detail & Related papers  (2024-06-20T15:25:13Z)
• Towards Effective Evaluations and Comparisons for LLM Unlearning Methods [97.2995389188179]
  This paper seeks to refine the evaluation of machine unlearning for large language models.<n>It addresses two key challenges -- the robustness of evaluation metrics and the trade-offs between competing goals.
  arXiv  Detail & Related papers  (2024-06-13T14:41:00Z)
• Goldfish: An Efficient Federated Unlearning Framework [3.956103498302838]
  Goldfish is a new framework for machine unlearning algorithms. It comprises four modules: basic model, loss function, optimization, and extension. To address the challenge of low validity in existing machine unlearning algorithms, we propose a novel loss function.
  arXiv  Detail & Related papers  (2024-04-04T03:29:41Z)
• Dataset Condensation Driven Machine Unlearning [0.0]
  Current trend in data regulation requirements and privacy-preserving machine learning has emphasized the importance of machine unlearning. We propose new dataset condensation techniques and an innovative unlearning scheme that strikes a balance between machine unlearning privacy, utility, and efficiency. We present a novel and effective approach to instrumenting machine unlearning and propose its application in defending against membership inference and model inversion attacks.
  arXiv  Detail & Related papers  (2024-01-31T21:48:25Z)
• Resilient Constrained Learning [94.27081585149836]
  This paper presents a constrained learning approach that adapts the requirements while simultaneously solving the learning task. We call this approach resilient constrained learning after the term used to describe ecological systems that adapt to disruptions by modifying their operation.
  arXiv  Detail & Related papers  (2023-06-04T18:14:18Z)
• Robust Deep Reinforcement Learning Scheduling via Weight Anchoring [7.570246812206769]
  We use weight anchoring to cultivate and fixate desired behavior in Neural Networks. Weight anchoring may be used to find a solution to a learning problem that is nearby the solution of another learning problem. Results show that this method provides performance comparable to the state of the art of augmenting a simulation environment.
  arXiv  Detail & Related papers  (2023-04-20T09:30:23Z)
• AttNS: Attention-Inspired Numerical Solving For Limited Data Scenarios [51.94807626839365]
  We propose the attention-inspired numerical solver (AttNS) to solve differential equations due to limited data.<n>AttNS is inspired by the effectiveness of attention modules in Residual Neural Networks (ResNet) in enhancing model generalization and robustness.
  arXiv  Detail & Related papers  (2023-02-05T01:39:21Z)
• A Novel Plug-and-Play Approach for Adversarially Robust Generalization [38.72514422694518]
  We propose a robust framework that employs adversarially robust training to safeguard the ML models against perturbed testing data. Our contributions can be seen from both computational and statistical perspectives.
  arXiv  Detail & Related papers  (2022-08-19T17:02:55Z)
• Unsupervised Learning for Robust Fitting:A Reinforcement Learning Approach [25.851792661168698]
  We introduce a novel framework that learns to solve robust model fitting. Unlike other methods, our work is agnostic to the underlying input features. We empirically show that our method outperforms existing learning approaches.
  arXiv  Detail & Related papers  (2021-03-05T07:14:00Z)
• Leveraging Semi-Supervised Learning for Fairness using Neural Networks [49.604038072384995]
  There has been a growing concern about the fairness of decision-making systems based on machine learning. In this paper, we propose a semi-supervised algorithm using neural networks benefiting from unlabeled data. The proposed model, called SSFair, exploits the information in the unlabeled data to mitigate the bias in the training data.
  arXiv  Detail & Related papers  (2019-12-31T09:11:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.