Multi-Objective Reference-Aligned Machine Unlearning

• URL: http://arxiv.org/abs/2606.00399v1
• Date: Fri, 29 May 2026 22:32:40 GMT
• Title: Multi-Objective Reference-Aligned Machine Unlearning
• Abstract summary: Machine unlearning aims to remove the influence of specific training samples while preserving the model's utility.<n>Existing single-objective approaches, such as ascent or random relabeling, often induce forgetting due to conflicting optimization dynamics and unbounded forgetting objectives.<n>We propose a multi-objective framework that jointly optimize forgetting and retention by replacing loss with a bounded KL alignment of predictions on forgotten samples.
• Score: 0.6299766708197883
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine unlearning aims to remove the influence of specific training samples while preserving the model's utility. Existing single-objective approaches, such as gradient ascent or random relabeling, often induce catastrophic forgetting due to conflicting optimization dynamics and unbounded forgetting objectives that cause the model to drift from its pre-trained knowledge. We propose Reference-Aligned UnLearning (RAUL), a multi-objective framework that jointly optimizes forgetting and retention by replacing unbounded loss maximization with a bounded KL alignment of predictions on forgotten samples toward a reference distribution representing unseen data, instantiated either as a uniform distribution or an empirical distribution from a held-out reference set, which constrains the forgetting objective and reduces gradient conflict with retention. The resulting multi-objective optimization (MOO) problem is solved via Jacobian descent, which aggregates multiple gradients into a direction that does not conflict. Our results demonstrate that RAUL achieves the closest gap compared to full retraining.

Related papers

• A Better Spur Should Start From Each Objective [12.849739663589316]
  Real-world Multi-Objective Reinforcement Learning (MORL) often suffers from sparse rewards, reward conflicts, and late-stage reward tug-of-war.<n>We propose Multi-Marginal Preference Optimization (MMPO), a fine-grained framework that intervenes at the data, gradient, and constraint levels.<n>Experiments on real-world e-commerce datasets show that MMPO improves training stability and consistently mitigates better performance across conflicting metrics.
  arXiv  Detail & Related papers  (2026-09-08T03:50:21Z)
• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Revitalizing the Beginning: Avoiding Storage Dependency for Model Merging in Continual Learning [57.10440766103372]
  Trajectory Regularized Merging (TRM) is a framework that reformulates the merging phase as an optimization process within an augmented trajectory subspace.<n>Our framework integrates three synergistic objectives including task alignment, prediction consistency, and gradient responsiveness to concurrently preserve merged model's historical stability and re-activate optimization dynamics.
  arXiv  Detail & Related papers  (2026-05-08T14:07:32Z)
• Step-level Denoising-time Diffusion Alignment with Multiple Objectives [26.715620800855962]
  Reinforcement learning (RL) has emerged as a powerful tool for aligning diffusion models with human preferences.<n>In practice, however, human preferences are inherently pluralistic, and aligned models must balance multiple downstream objectives.<n>We propose Multi-objective Step-level Denoising-time Diffusion Alignment (MSDDA), a retraining-free framework for aligning diffusion models with multiple objectives.
  arXiv  Detail & Related papers  (2026-04-15T19:52:05Z)
• SetPO: Set-Level Policy Optimization for Diversity-Preserving LLM Reasoning [50.93295951454092]
  We introduce a set level diversity objective defined over sampled trajectories using kernelized similarity.<n>Our approach derives a leave-one-out marginal contribution for each sampled trajectory and integrates this objective as a plug-in advantage shaping term for policy optimization.<n>Experiments across a range of model scales demonstrate the effectiveness of our proposed algorithm, consistently outperforming strong baselines in both Pass@1 and Pass@K across various benchmarks.
  arXiv  Detail & Related papers  (2026-02-01T07:13:20Z)
• Distribution-Guided and Constrained Quantum Machine Unlearning [5.518378568494161]
  Machine unlearning aims to remove the influence of specific training data from a learned model without full retraining.<n>We propose a distribution-guided framework for class-level quantum machine unlearning that treats unlearning as a constrained optimization problem.
  arXiv  Detail & Related papers  (2026-01-07T21:44:20Z)
• OFMU: Optimization-Driven Framework for Machine Unlearning [5.100622189286672]
  Large language models increasingly require the ability to unlearn specific knowledge, such as user requests, copyrighted materials, or outdated information.<n>We propose OFMU, a penalty-based bi-level optimization framework that explicitly prioritizes forgetting while preserving retention.<n>We show that OFMU consistently outperforms existing unlearning methods in both efficacy and retained utility.
  arXiv  Detail & Related papers  (2025-09-26T15:31:32Z)
• Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing [58.52119063742121]
  Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance.<n>This paper addresses the question of how to optimally combine the model's predictions and the provided labels.<n>Our main contribution is the derivation of the Bayes optimal aggregator function to combine the current model's predictions and the given labels.
  arXiv  Detail & Related papers  (2025-05-21T07:16:44Z)
• MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment [14.541973333460149]
  Mixing Preference Optimization (MPO) is a post-processing framework for aggregating single-objective policies.<n>MPO achieves balanced performance across diverse preferences, outperforming existing models with significantly reduced computational costs.
  arXiv  Detail & Related papers  (2025-02-25T23:22:12Z)
• UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning [57.081646768835704]
  User specifications or legal frameworks often require information to be removed from pretrained models, including large language models (LLMs)<n>This requires deleting or "forgetting" a set of data points from an already-trained model, which typically degrades its performance on other data points.<n>We propose UPCORE, a method-agnostic data selection framework for mitigating collateral damage during unlearning.
  arXiv  Detail & Related papers  (2025-02-20T22:51:10Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.