SAGE: Retain-Aware Post-Hoc Sanitization of Final Unlearning Vector

• URL: http://arxiv.org/abs/2606.18309v1
• Date: Tue, 16 Jun 2026 08:29:43 GMT
• Title: SAGE: Retain-Aware Post-Hoc Sanitization of Final Unlearning Vector
• Abstract summary: Current unlearning methods all involve a trade-off between unlearning and retention.<n>We have found that the retention activation bias can also be used to quantify the damage an unlearning method inflicts on retention.<n>This allows us to restore retention performance for any unlearning method using a post-hoc approach.
• Score: 32.30497427199285
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Model (LLM) unlearning aims to remove undesirable knowledge or behaviors while preserving retained capabilities. Current unlearning methods all involve a trade-off between unlearning and retention. We have found that the retention activation bias can also be used to quantify the damage an unlearning method inflicts on retention, without considering the specific implementation of the unlearning process. This allows us to restore retention performance for any unlearning method using a post-hoc approach. Therefore, we propose a complementary post-hoc setting to sanitize the final update vector without rerunning the original unlearning pipeline. In this setting, we design SAGE, Spectral Activation-GEometry Sanitization, a source-agnostic correction for final unlearning updates. SAGE collects real module inputs from a small retain proxy, extracts their dominant activation geometry, and solves a source-anchored optimization objective in closed form, which suppresses update components aligned with high-energy retained directions while preserving the source method's forgetting carrier. Across multiple unlearning methods, model scales, and benchmarks, SAGE consistently relieves the retain-forget trade-off, identifying post-hoc sanitization of final vectors as a practical and underexplored axis for machine unlearning.

Related papers

• Test-Time Unlearning via Sparse Autoencoder [55.4290003355764]
  We propose ARIA, a test-time unlearning method that leaves model weights intact and gates access to unwanted knowledge only when generation enters a forget-related state.<n>ARIA improves the forget-retain trade-off over weight-based baselines across both a thinking model and an instruction model.
  arXiv  Detail & Related papers  (2026-09-14T18:58:09Z)
• On-the-go Forgetting without Explicit Unlearning via ERASE [2.6426981014417046]
  ERASE is a framework for on-the-go forgetting that suppresses the observable influence of private data without modifying model weights.<n>We rigorously characterize sufficient conditions when ERASE provably achieves functional forgetting of designated subclasses.<n> ERASE maintains the best observed balance between forgetting efficacy, computational efficiency, and retention fidelity over recent unlearning-based methods.
  arXiv  Detail & Related papers  (2026-09-05T08:09:27Z)
• Source-Free Class Relearning: Diagnosing Forgetting in Class Unlearning [22.455121713091245]
  Class unlearning aims to remove a model's ability to recognize designated forget classes while preserving performance on retain classes.<n>We study class relearning in a strictly source-free setting, asking whether a forget class can be recovered using only the unlearned model.
  arXiv  Detail & Related papers  (2026-09-02T02:45:56Z)
• Inference-Time Machine Unlearning via Gated Activation Redirection [1.1699531043716684]
  We introduce Inference-Time Unlearning via Gated Activation Redirection (GUARD-IT)<n>GUARD-IT unlearns via input-dependent activation steering at inference time, leaving model weights untouched.<n>Experiments on TOFU and MUSE show that GUARD-IT matches or exceeds 12 gradient-based baselines across three model scales.
  arXiv  Detail & Related papers  (2026-05-12T21:26:25Z)
• Geometric-Disentangelment Unlearning [106.99160454669902]
  gradient ascent on forget samples often harms retained knowledge.<n>We propose the Geometric-disment Unlearning (GU) that decomposes any candidate forget gradient update into tangential and normal components to retain space and executes only the normal component.<n>Our method is plug-and-play and can be attached to existing gradient-based unlearning procedures to mitigate side effects.
  arXiv  Detail & Related papers  (2025-11-21T09:58:25Z)
• Rotation Control Unlearning: Quantifying and Controlling Continuous Unlearning for LLM with The Cognitive Rotation Space [66.51378598755933]
  We propose a novel method, called Rotation Control Unlearning (RCU), to quantify and control the unlearning degree in the continuous unlearning process.<n>The skew symmetric loss is designed to construct the existence of the cognitive rotation space, where the changes of rotational angle can simulate the continuous unlearning process.<n> Experiments on multiple datasets confirm that our method without retained dataset achieves SOTA performance.
  arXiv  Detail & Related papers  (2025-09-30T03:59:29Z)
• ResAD++: Towards Class Agnostic Anomaly Detection via Residual Feature Learning [52.11294707895649]
  This paper explores the problem of class-agnostic anomaly detection (AD)<n>The objective is to train one class-agnostic AD model that can generalize to detect anomalies in diverse new classes from different domains without any retraining or fine-tuning on the target data.<n> Comprehensive experiments on eight real-world AD datasets demonstrate that our ResAD++ can achieve remarkable AD results when directly used in new classes.
  arXiv  Detail & Related papers  (2025-09-28T08:41:05Z)
• Restoring Forgotten Knowledge in Non-Exemplar Class Incremental Learning through Test-Time Semantic Evolution [11.50324946279326]
  In Non-exemplar Class Incremental Learning (NECIL), forgetting arises because old classes are inaccessible.<n>We propose RoSE, which is a test-time semantic drift compensation framework.<n>We evaluate RoSE on CIFAR-100, TinyImageNet, and ImageNet100 datasets, under both cold-start and warm-start settings.
  arXiv  Detail & Related papers  (2025-03-21T02:02:35Z)
• GRU: Mitigating the Trade-off between Unlearning and Retention for LLMs [34.90826139012299]
  We propose gradient Rectified Unlearning (GRU), an improved framework that regulates the directions of updates during the unlearning procedure.<n>GRU is easy and general to implement, demonstrating practical effectiveness across a variety of well-established unlearning benchmarks.
  arXiv  Detail & Related papers  (2025-03-12T07:08:54Z)
• Adaptive Retention & Correction: Test-Time Training for Continual Learning [114.5656325514408]
  A common problem in continual learning is the classification layer's bias towards the most recent task.<n>We name our approach Adaptive Retention & Correction (ARC)<n>ARC achieves an average performance increase of 2.7% and 2.6% on the CIFAR-100 and Imagenet-R datasets.
  arXiv  Detail & Related papers  (2024-05-23T08:43:09Z)
• Is Retain Set All You Need in Machine Unlearning? Restoring Performance of Unlearned Models with Out-Of-Distribution Images [0.0]
  We introduce Selective-distillation for Class and Architecture-agnostic unleaRning (SCAR) SCAR efficiently eliminates specific information while preserving the model's test accuracy without using a retain set. We experimentally verified the effectiveness of our method, on three public datasets.
  arXiv  Detail & Related papers  (2024-04-19T14:45:27Z)
• Incremental Object Detection via Meta-Learning [77.55310507917012]
  We propose a meta-learning approach that learns to reshape model gradients, such that information across incremental tasks is optimally shared. In comparison to existing meta-learning methods, our approach is task-agnostic, allows incremental addition of new-classes and scales to high-capacity models for object detection.
  arXiv  Detail & Related papers  (2020-03-17T13:40:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.