Rethinking Federated Unlearning via the Lens of Memorization

• URL: http://arxiv.org/abs/2605.24545v1
• Date: Sat, 23 May 2026 12:25:50 GMT
• Title: Rethinking Federated Unlearning via the Lens of Memorization
• Abstract summary: We argue that unlearning should mainly remove the unique memorized information attributable to the data to be forgotten.<n>FedMemPrune is a pruning-based unlearning approach that resets redundant parameters responsible for memorization.
• Score: 43.29764212924278
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning (FL) increasingly needs machine unlearning to comply with privacy regulations. However, existing federated unlearning approaches may overlook the overlapping information between the unlearning and remaining data, leading to ineffective unlearning and unfairness between clients. In this work, we revisit federated unlearning through the lens of memorization. We argue that unlearning should mainly remove the unique memorized information attributable to the data to be forgotten, while preserving overlapping patterns that are also supported by the remaining data. Specifically, we propose Grouped Memorization Evaluation, an example-level metric that separates memorized knowledge from overlapping knowledge. Building on this metric, we introduce Federated Memorization Pruning (FedMemPrune), a pruning-based unlearning approach that resets redundant parameters responsible for memorization. Extensive experiments show that FedMemPrune closely matches retraining-based unlearning baselines while more effectively eliminating memorization than existing federated unlearning algorithms, yielding strong unlearning performance without sacrificing the utility of retained knowledge.

Related papers

• Confess What You Know: Forget-Set Misalignment with Model Knowledge in LLM Unlearning [9.140696856930457]
  We term this gap forget-set misalignment and identify two cases.<n>In Under Unlearning, the forget set omits memorized information and leakage persists.<n>In Out-of-Knowledge Unlearning, the algorithm is driven to "forget" knowledge the model never learned, perturbing parameters and degrading utility.<n>We propose a data-blind framework that constructs model-aligned forget sets by eliciting and formalizing the model's memorized knowledge.
  arXiv  Detail & Related papers  (2026-09-01T02:47:43Z)
• Certifying the Right to Be Forgotten: Primal-Dual Optimization for Sample and Label Unlearning in Vertical Federated Learning [31.54643729002375]
  Federated unlearning enables the removal of specific data influences from trained models.<n>Federated Optimization for data Removal via primal-dual Algorithm proposed.<n>New unlearning loss function promotes classification uncertainty rather than misclassification.
  arXiv  Detail & Related papers  (2025-12-29T03:25:52Z)
• Retrieval-augmented Prompt Learning for Pre-trained Foundation Models [101.13972024610733]
  We present RetroPrompt, which aims to achieve a balance between memorization and generalization.<n>Unlike traditional prompting methods, RetroPrompt incorporates a retrieval mechanism throughout the input, training, and inference stages.<n>We conduct comprehensive experiments on a variety of datasets across natural language processing and computer vision tasks to demonstrate the superior performance of our proposed approach.
  arXiv  Detail & Related papers  (2025-12-23T08:15:34Z)
• LLM Unlearning on Noisy Forget Sets: A Study of Incomplete, Rewritten, and Watermarked Data [69.5099112089508]
  Large language models (LLMs) exhibit remarkable generative capabilities but raise ethical and security concerns by memorizing sensitive data.<n>This work presents the first study of unlearning under perturbed or low-fidelity forget data, referred to as noisy forget sets.<n>We find that unlearning remains surprisingly robust to perturbations, provided that core semantic signals are preserved.
  arXiv  Detail & Related papers  (2025-10-10T05:10:49Z)
• Understanding the Dilemma of Unlearning for Large Language Models [50.54260066313032]
  Unlearning seeks to remove specific knowledge from large language models (LLMs)<n>We propose unPact, an interpretable framework for unlearning via prompt attribution and contribution tracking.
  arXiv  Detail & Related papers  (2025-09-29T12:15:19Z)
• Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter [35.65566527544619]
  Federated learning is a promising paradigm for privacy-preserving collaborative model training.<n>We propose FUSED, which first identifies critical layers by analyzing each layer's sensitivity to knowledge.<n> adapters are trained without altering the original parameters, overwriting the unlearning knowledge with the remaining knowledge.
  arXiv  Detail & Related papers  (2025-02-28T04:35:26Z)
• Siamese Machine Unlearning with Knowledge Vaporization and Concentration [23.796344455232227]
  We propose the concepts of knowledge vaporization and concentration to selectively erase learned knowledge from specific data points.<n>Our proposed Siamese unlearning method does not require additional memory overhead and full access to the remaining dataset.
  arXiv  Detail & Related papers  (2024-12-02T07:19:49Z)
• TOFU: A Task of Fictitious Unlearning for LLMs [99.92305790945507]
  Large language models trained on massive corpora of data from the web can reproduce sensitive or private data raising both legal and ethical concerns. Unlearning, or tuning models to forget information present in their training data, provides us with a way to protect private data after training. We present TOFU, a benchmark aimed at helping deepen our understanding of unlearning.
  arXiv  Detail & Related papers  (2024-01-11T18:57:12Z)
• Class-wise Federated Unlearning: Harnessing Active Forgetting with Teacher-Student Memory Generation [11.638683787598817]
  We propose a neuro-inspired federated unlearning framework based on active forgetting.<n>Our framework distinguishes itself from existing methods by utilizing new memories to overwrite old ones.<n>Our method achieves satisfactory unlearning completeness against backdoor attacks.
  arXiv  Detail & Related papers  (2023-07-07T03:07:26Z)
• Learning with Recoverable Forgetting [77.56338597012927]
  Learning wIth Recoverable Forgetting explicitly handles the task- or sample-specific knowledge removal and recovery. Specifically, LIRF brings in two innovative schemes, namely knowledge deposit and withdrawal. We conduct experiments on several datasets, and demonstrate that the proposed LIRF strategy yields encouraging results with gratifying generalization capability.
  arXiv  Detail & Related papers  (2022-07-17T16:42:31Z)
• Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning [113.58691755215663]
  We develop RetroPrompt to help a model strike a balance between generalization and memorization. In contrast with vanilla prompt learning, RetroPrompt constructs an open-book knowledge-store from training instances. Extensive experiments demonstrate that RetroPrompt can obtain better performance in both few-shot and zero-shot settings.
  arXiv  Detail & Related papers  (2022-05-29T16:07:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.