Measuring the Depth of LLM Unlearning via Activation Patching

• URL: http://arxiv.org/abs/2605.24614v1
• Date: Sat, 23 May 2026 14:52:28 GMT
• Title: Measuring the Depth of LLM Unlearning via Activation Patching
• Abstract summary: Large language model (LLM) unlearning has emerged as a crucial post-hoc mechanism for privacy protection and AI safety.<n>Existing output-level metrics fail to detect when this knowledge remains recoverable from internal representations.<n>We propose the Unlearning Depth Score (UDS), a metric that quantifies the mechanistic depth of unlearning via activation patching.
• Score: 3.590648050085134
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) unlearning has emerged as a crucial post-hoc mechanism for privacy protection and AI safety, yet auditing whether target knowledge is truly erased remains challenging. Existing output-level metrics fail to detect when this knowledge remains recoverable from internal representations. Recent white-box studies reveal such residual knowledge but often rely on auxiliary training or dataset-specific adaptations, leaving no generalizable metric. To address these limitations, we propose the Unlearning Depth Score (UDS), a metric that quantifies the mechanistic depth of unlearning via activation patching. UDS first identifies layers that encode the target knowledge using a retain model baseline, then measures how much of it is erased in the unlearned model on a 0-1 scale. In a meta-evaluation across 20 metrics on 150 unlearned models spanning 8 methods, UDS achieves the highest faithfulness and robustness, confirming our causal approach as the most reliable for unlearning evaluation. Case studies further reveal that white-box metrics can disagree at the layer level and that erasure depth varies across examples. We provide guidelines for integrating UDS into existing benchmarking frameworks and streamlining the evaluation pipeline. Code and data are available at https://github.com/gnueaj/unlearning-depth-score

Related papers

• Test-Time Unlearning via Sparse Autoencoder [55.4290003355764]
  We propose ARIA, a test-time unlearning method that leaves model weights intact and gates access to unwanted knowledge only when generation enters a forget-related state.<n>ARIA improves the forget-retain trade-off over weight-based baselines across both a thinking model and an instruction model.
  arXiv  Detail & Related papers  (2026-09-14T18:58:09Z)
• Exploring and Bridging Knowledge Holes in Unlearned Multimodal Large Language Models [23.51933427959403]
  We propose a benchmark that captures unintended degradation on benign inputs sharing generic patterns with the forget set.<n>We also propose Selective Protection with Anchored Regularization, which protects generic patterns via anchored activation filtering.<n>Our experiments on SafeEraser demonstrate that SPAR recovers over 98% of vanilla response quality compared to below 50% for standard baselines.
  arXiv  Detail & Related papers  (2026-08-03T07:57:34Z)
• LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning [30.67510673327925]
  We introduce LACUNA: the first unlearning testbed with ground-truth parameter-level localization.<n>We show that LACUNA achieves strong erasure and robustness to resurfacing attacks, highlighting the importance of precise unlearning.
  arXiv  Detail & Related papers  (2026-07-02T17:59:52Z)
• REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loop [11.291998503454854]
  Machine unlearning for LLMs aims to remove sensitive or copyrighted data from trained models.<n>Standard evaluation metrics rely on benign queries that often mistake superficial information suppression for genuine knowledge removal.<n>We introduce REBEL, an evolutionary approach for adversarial prompt generation designed to probe whether unlearned data can still be recovered.
  arXiv  Detail & Related papers  (2026-02-05T22:54:56Z)
• Leak@$k$: Unlearning Does Not Make LLMs Forget Under Probabilistic Decoding [18.830386174815583]
  We show that textitalmost all existing unlearning methods fail to achieve true forgetting in practice.<n>We introduce textttleak@$k$, a new meta-evaluation metric that quantifies the likelihood of forgotten knowledge reappearing.
  arXiv  Detail & Related papers  (2025-11-07T02:30:05Z)
• Exploring Structural Degradation in Dense Representations for Self-supervised Learning [84.52554180480037]
  We observe a counterintuitive phenomenon in self-supervised learning (SSL): longer training may impair the performance of dense prediction tasks.<n>We refer to this phenomenon as Self-supervised Dense Degradation (SDD) and demonstrate its consistent presence across sixteen state-of-the-art SSL methods.<n>We introduce a Dense representation Structure Estimator (DSE) composed of a class-relevance measure and an effective dimensionality measure.
  arXiv  Detail & Related papers  (2025-10-20T08:40:16Z)
• LLM Unlearning on Noisy Forget Sets: A Study of Incomplete, Rewritten, and Watermarked Data [69.5099112089508]
  Large language models (LLMs) exhibit remarkable generative capabilities but raise ethical and security concerns by memorizing sensitive data.<n>This work presents the first study of unlearning under perturbed or low-fidelity forget data, referred to as noisy forget sets.<n>We find that unlearning remains surprisingly robust to perturbations, provided that core semantic signals are preserved.
  arXiv  Detail & Related papers  (2025-10-10T05:10:49Z)
• Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputs [23.538087984484207]
  Machine unlearning (MU) for large language models (LLMs) seeks to remove specific undesirable data or knowledge from a trained model.<n>We identify a new vulnerability post-unlearning: unlearning trace detection.<n>We demonstrate that unlearning traces can be detected with over 90% accuracy even under forget-irrelevant inputs.
  arXiv  Detail & Related papers  (2025-06-16T21:03:51Z)
• Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs [38.837810490068556]
  Unlearning in large language models (LLMs) aims to remove specified data, but its efficacy is typically assessed with task-level metrics like accuracy and perplexity.<n>We demonstrate that models can appear to forget while their original behavior is easily restored through minimal fine-tuning.<n>This phenomenon of emphreversibility suggests that information is merely suppressed, not genuinely erased.
  arXiv  Detail & Related papers  (2025-05-22T16:02:10Z)
• Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models [81.62767292169225]
  We investigate knowledge forgetting in large language models with a focus on its generalisation.<n>We propose PerMU, a novel probability perturbation-based unlearning paradigm.<n>Experiments are conducted on a diverse range of datasets, including TOFU, Harry Potter, ZsRE, WMDP, and MUSE.
  arXiv  Detail & Related papers  (2025-02-27T11:03:33Z)
• Unsupervised Continual Anomaly Detection with Contrastively-learned Prompt [80.43623986759691]
  We introduce a novel Unsupervised Continual Anomaly Detection framework called UCAD. The framework equips the UAD with continual learning capability through contrastively-learned prompts. We conduct comprehensive experiments and set the benchmark on unsupervised continual anomaly detection and segmentation.
  arXiv  Detail & Related papers  (2024-01-02T03:37:11Z)
• Incremental Object Detection via Meta-Learning [77.55310507917012]
  We propose a meta-learning approach that learns to reshape model gradients, such that information across incremental tasks is optimally shared. In comparison to existing meta-learning methods, our approach is task-agnostic, allows incremental addition of new-classes and scales to high-capacity models for object detection.
  arXiv  Detail & Related papers  (2020-03-17T13:40:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.