MCU: Improving Machine Unlearning through Mode Connectivity

• URL: http://arxiv.org/abs/2505.10859v1
• Date: Fri, 16 May 2025 04:56:47 GMT
• Title: MCU: Improving Machine Unlearning through Mode Connectivity
• Abstract summary: We propose a novel MU framework called Mode Connectivity Unlearning (MCU) that leverages mode connectivity to find an unlearning pathway in a nonlinear manner.<n>To further enhance performance and efficiency, we introduce a parameter mask strategy that not only improves unlearning effectiveness but also reduces computational overhead.
• Score: 1.3731623617634434
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine Unlearning (MU) aims to remove the information of specific training data from a trained model, ensuring compliance with privacy regulations and user requests. While one line of existing MU methods relies on linear parameter updates via task arithmetic, they suffer from weight entanglement. In this work, we propose a novel MU framework called Mode Connectivity Unlearning (MCU) that leverages mode connectivity to find an unlearning pathway in a nonlinear manner. To further enhance performance and efficiency, we introduce a parameter mask strategy that not only improves unlearning effectiveness but also reduces computational overhead. Moreover, we propose an adaptive adjustment strategy for our unlearning penalty coefficient to adaptively balance forgetting quality and predictive performance during training, eliminating the need for empirical hyperparameter tuning. Unlike traditional MU methods that identify only a single unlearning model, MCU uncovers a spectrum of unlearning models along the pathway. Overall, MCU serves as a plug-and-play framework that seamlessly integrates with any existing MU methods, consistently improving unlearning efficacy. Extensive experiments on the image classification task demonstrate that MCU achieves superior performance.

Related papers

• A Model Merging Approach for Continual MLLM Unlearning [22.83414107472909]
  Merging for Continual Unlearning (MCU) is an approach that dynamically merges multiple one-shot unlearning adapters into a unified adapter upon receiving each new unlearning request.<n>MCU achieves superior unlearning effectiveness while preserving both retained knowledge and general multimodal utility.
  arXiv  Detail & Related papers  (2026-08-05T07:37:23Z)
• Understanding Machine Unlearning Through the Lens of Mode Connectivity [20.167582259240763]
  We study machine unlearning through the lens of mode connectivity.<n>We find that unlearned models lie in connected basins with smooth retain/forget behavior.<n>This is the first study of machine unlearning through the lens of mode connectivity.
  arXiv  Detail & Related papers  (2026-07-27T03:39:03Z)
• Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning [57.237835833636154]
  We propose MetrIc Learning with Expandable Subspace (Miles) to harness the prior information within pre-trained knowledge.<n>Miles achieves state-of-the-art performance in various CIL settings.
  arXiv  Detail & Related papers  (2026-07-20T06:22:02Z)
• Efficient Machine Unlearning via Influence Approximation [75.31015485113993]
  Influence-based unlearning has emerged as a prominent approach to estimate the impact of individual training samples on model parameters without retraining.<n>This paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning)<n>We introduce the Influence Approximation Unlearning algorithm for efficient machine unlearning from the incremental perspective.
  arXiv  Detail & Related papers  (2025-07-31T05:34:27Z)
• Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate [105.86576388991713]
  We introduce a normalized gradient difference (NGDiff) algorithm, enabling us to have better control over the trade-off between the objectives.<n>We provide a theoretical analysis and empirically demonstrate the superior performance of NGDiff among state-of-the-art unlearning methods on the TOFU and MUSE datasets.
  arXiv  Detail & Related papers  (2024-10-29T14:41:44Z)
• Machine Unlearning with Minimal Gradient Dependence for High Unlearning Ratios [18.73206066109299]
  Mini-Unlearning is a novel approach that capitalizes on a critical observation: unlearned parameters correlate with retrained parameters through contraction mapping. This lightweight, scalable method significantly enhances model accuracy and strengthens resistance to membership inference attacks. Our experiments demonstrate that Mini-Unlearning not only works under higher unlearning ratios but also outperforms existing techniques in both accuracy and security.
  arXiv  Detail & Related papers  (2024-06-24T01:43:30Z)
• Alignment Calibration: Machine Unlearning for Contrastive Learning under Auditing [33.418062986773606]
  We first propose the framework of Machine Unlearning for Contrastive learning (MUC) and adapting existing methods. We observe that several methods are mediocre unlearners and existing auditing tools may not be sufficient for data owners to validate the unlearning effects in contrastive learning. We propose a novel method called Alignment (AC) by explicitly considering the properties of contrastive learning and optimizing towards novel metrics to easily verify unlearning.
  arXiv  Detail & Related papers  (2024-06-05T19:55:45Z)
• Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining [50.00291020618743]
  This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU) Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems.
  arXiv  Detail & Related papers  (2024-04-08T20:02:19Z)
• Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters [65.15700861265432]
  We present a parameter-efficient continual learning framework to alleviate long-term forgetting in incremental learning with vision-language models. Our approach involves the dynamic expansion of a pre-trained CLIP model, through the integration of Mixture-of-Experts (MoE) adapters. To preserve the zero-shot recognition capability of vision-language models, we introduce a Distribution Discriminative Auto-Selector.
  arXiv  Detail & Related papers  (2024-03-18T08:00:23Z)
• A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading [62.34538208323411]
  We propose a multi-head ensemble multi-task learning (MEMTL) approach with a shared backbone and multiple prediction heads (PHs) MEMTL outperforms benchmark methods in both the inference accuracy and mean square error without requiring additional training data.
  arXiv  Detail & Related papers  (2023-09-02T11:01:16Z)
• Model Sparsity Can Simplify Machine Unlearning [33.18951938708467]
  In response to recent data regulation requirements, machine unlearning (MU) has emerged as a critical process. Our study introduces a novel model-based perspective: model sparsification via weight pruning. We show in both theory and practice that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner.
  arXiv  Detail & Related papers  (2023-04-11T02:12:02Z)
• Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resources [78.72922528736011]
  We propose a novel approach, black-box adversarial reprogramming (BAR), that repurposes a well-trained black-box machine learning model. Using zeroth order optimization and multi-label mapping techniques, BAR can reprogram a black-box ML model solely based on its input-output responses. BAR outperforms state-of-the-art methods and yields comparable performance to the vanilla adversarial reprogramming method.
  arXiv  Detail & Related papers  (2020-07-17T01:52:34Z)
• Gradient Monitored Reinforcement Learning [0.0]
  We focus on the enhancement of training and evaluation performance in reinforcement learning algorithms. We propose an approach to steer the learning in the weight parameters of a neural network based on the dynamic development and feedback from the training process itself.
  arXiv  Detail & Related papers  (2020-05-25T13:45:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.