CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge

• URL: http://arxiv.org/abs/2604.14644v1
• Date: Thu, 16 Apr 2026 05:49:10 GMT
• Title: CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge
• Abstract summary: We propose Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge (CURaTE)<n>Our method begins by training a sentence embedding model on a dataset designed to enable the formation of sharp decision boundaries.<n>We show that even with such a simple approach, CURaTE achieve more effective forgetting than existing methods.
• Score: 39.26757541186985
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The inability to filter out in advance all potentially problematic data from the pre-training of large language models has given rise to the need for methods for unlearning specific pieces of knowledge after training. Existing techniques overlook the need for continuous and immediate action, causing them to suffer from degraded utility as updates accumulate and protracted exposure of sensitive information. To address these issues, we propose Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge (CURaTE). Our method begins by training a sentence embedding model on a dataset designed to enable the formation of sharp decision boundaries for determining whether a given input prompt corresponds to any stored forget requests. The similarity of a given input to the forget requests is then used to determine whether to answer or return a refusal response. We show that even with such a simple approach, not only does CURaTE achieve more effective forgetting than existing methods, but by avoiding modification of the language model parameters, it also maintains near perfect knowledge preservation over any number of updates and is the only method capable of continual unlearning in real-time.

Related papers

• DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning [15.58340591381191]
  Unlearning in Large Language Models (LLMs) is crucial for protecting private data and removing harmful knowledge.<n>Most existing approaches rely on fine-tuning to balance unlearning efficiency with general language capabilities.<n>We propose Detect-Reasoning Augmented GeneratiON (DRAGON) to overcome these limitations.
  arXiv  Detail & Related papers  (2025-11-08T01:13:28Z)
• Scalable and Robust LLM Unlearning by Correcting Responses with Retrieved Exclusions [49.55618517046225]
  Language models trained on web-scale corpora risk memorizing and exposing sensitive information.<n>We propose Corrective Unlearning with Retrieved Exclusions (CURE), a novel unlearning framework.<n>CURE verifies model outputs for leakage and revises them into safe responses.
  arXiv  Detail & Related papers  (2025-09-30T09:07:45Z)
• GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection [36.38245533018162]
  Large Language Models (LLMs) have demonstrated strong capabilities in memorizing vast amounts of knowledge across diverse domains.<n>Existing unlearning efforts typically fine-tune the model with resources such as forget data, retain data, and a calibration model.<n>We propose Generation-time Unlearning via Adaptive Restriction and Detection (GUARD), a framework that enables dynamic unlearning during LLM generation.
  arXiv  Detail & Related papers  (2025-05-19T16:26:58Z)
• ACU: Analytic Continual Unlearning for Efficient and Exact Forgetting with Privacy Preservation [39.0731790601695]
  Continual Unlearning (CU) aims to sequentially forget particular knowledge acquired during the Continual Learning phase.<n>Most existing unlearning methods require access to the retained dataset for re-training or fine-tuning.<n>We propose a novel gradient-free method for CU, named Analytic Continual Unlearning (ACU), for efficient and exact forgetting with historical data privacy preservation.
  arXiv  Detail & Related papers  (2025-05-18T05:28:18Z)
• Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models [52.40798352740857]
  We introduce the Iterative Contrastive Unlearning (ICU) framework, which consists of three core components.<n>A Knowledge Unlearning Induction module targets specific knowledge for removal using an unlearning loss.<n>A Contrastive Learning Enhancement module preserves the model's expressive capabilities against the pure unlearning goal.<n>An Iterative Unlearning Refinement module dynamically adjusts the unlearning process through ongoing evaluation and updates.
  arXiv  Detail & Related papers  (2024-07-25T07:09:35Z)
• Adaptive Retention & Correction: Test-Time Training for Continual Learning [114.5656325514408]
  A common problem in continual learning is the classification layer's bias towards the most recent task.<n>We name our approach Adaptive Retention & Correction (ARC)<n>ARC achieves an average performance increase of 2.7% and 2.6% on the CIFAR-100 and Imagenet-R datasets.
  arXiv  Detail & Related papers  (2024-05-23T08:43:09Z)
• Embedding-Informed Adaptive Retrieval-Augmented Generation of Large Language Models [37.02290559379761]
  Retrieval-augmented large language models (LLMs) have been remarkably competent in various NLP tasks.<n>Motivated by this, Adaptive Retrieval-Augmented Generation (ARAG) studies retrieving only when the knowledge asked by the query is absent in the LLM.
  arXiv  Detail & Related papers  (2024-04-04T15:21:22Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• Learning with Recoverable Forgetting [77.56338597012927]
  Learning wIth Recoverable Forgetting explicitly handles the task- or sample-specific knowledge removal and recovery. Specifically, LIRF brings in two innovative schemes, namely knowledge deposit and withdrawal. We conduct experiments on several datasets, and demonstrate that the proposed LIRF strategy yields encouraging results with gratifying generalization capability.
  arXiv  Detail & Related papers  (2022-07-17T16:42:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.