Erasing CLIP Memories: Non-Destructive, Data-Free Zero-Shot class Unlearning in CLIP Models

• URL: http://arxiv.org/abs/2512.14137v1
• Date: Tue, 16 Dec 2025 06:37:41 GMT
• Title: Erasing CLIP Memories: Non-Destructive, Data-Free Zero-Shot class Unlearning in CLIP Models
• Authors: Ashish Mishra, Tarun Kumar, Gyanaranjan Nayak, Arpit Shah, Suparna Bhattacharya, Martin Foltin,
• Abstract summary: We introduce a novel, closed-form approach for selective unlearning in multimodal models.<n>Our method leverages nullspace projection to erase the target class information embedded in the final projection layer.<n>Our experiments demonstrate that even a partial projection can balance between complete unlearning and retaining useful information.
• Score: 4.820351122363815
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We introduce a novel, closed-form approach for selective unlearning in multimodal models, specifically targeting pretrained models such as CLIP. Our method leverages nullspace projection to erase the target class information embedded in the final projection layer, without requiring any retraining or the use of images from the forget set. By computing an orthonormal basis for the subspace spanned by target text embeddings and projecting these directions, we dramatically reduce the alignment between image features and undesired classes. Unlike traditional unlearning techniques that rely on iterative fine-tuning and extensive data curation, our approach is both computationally efficient and surgically precise. This leads to a pronounced drop in zero-shot performance for the target classes while preserving the overall multimodal knowledge of the model. Our experiments demonstrate that even a partial projection can balance between complete unlearning and retaining useful information, addressing key challenges in model decontamination and privacy preservation.

Related papers

• Selective, Controlled and Domain-Agnostic Unlearning in Pretrained CLIP: A Training- and Data-Free Approach [4.820351122363815]
  Real-world applications often demand the removal (or "unlearning") of specific object classes without requiring additional data or retraining.<n>We propose a novel training- and data-free unlearning framework that enables three distinct forgetting paradigms.<n>By leveraging a multimodal nullspace through synergistic integration of text prompts and synthesized visual prototypes, our method efficiently removes undesired class information while preserving the remaining knowledge.
  arXiv  Detail & Related papers  (2025-12-16T05:54:13Z)
• Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMs [54.167494079321465]
  Current unlearning methods for LLMs optimize on the private information they seek to remove by incorporating it into their fine-tuning data.<n>We propose a novel unlearning method-Partial Model Collapse (PMC), which does not require unlearning targets in the unlearning objective.
  arXiv  Detail & Related papers  (2025-07-06T03:08:49Z)
• Targeted Forgetting of Image Subgroups in CLIP Models [30.78624907082701]
  Foundation models (FMs) such as CLIP have demonstrated impressive zero-shot performance across various tasks.<n>They often inherit harmful or unwanted knowledge from noisy internet-sourced datasets.<n>Existing model unlearning methods either rely on access to pre-trained datasets or focus on coarse-grained unlearning.<n>We propose a novel three-stage approach that progressively unlearns targeted knowledge while mitigating over-forgetting.
  arXiv  Detail & Related papers  (2025-06-03T17:50:03Z)
• UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning [57.081646768835704]
  User specifications or legal frameworks often require information to be removed from pretrained models, including large language models (LLMs)<n>This requires deleting or "forgetting" a set of data points from an already-trained model, which typically degrades its performance on other data points.<n>We propose UPCORE, a method-agnostic data selection framework for mitigating collateral damage during unlearning.
  arXiv  Detail & Related papers  (2025-02-20T22:51:10Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• Zero-Shot Class Unlearning in CLIP with Synthetic Samples [0.0]
  We focus on unlearning within CLIP, a dual vision-language model trained on a massive dataset of image-text pairs.<n>We apply Lipschitz regularization to the multimodal context of CLIP.<n>Our forgetting procedure is iterative, where we track accuracy on a synthetic forget set and stop when accuracy falls below a chosen threshold.
  arXiv  Detail & Related papers  (2024-07-10T09:16:14Z)
• Partially Blinded Unlearning: Class Unlearning for Deep Networks a Bayesian Perspective [4.31734012105466]
  Machine Unlearning is the process of selectively discarding information designated to specific sets or classes of data from a pre-trained model. We propose a methodology tailored for the purposeful elimination of information linked to a specific class of data from a pre-trained classification network. Our novel approach, termed textbfPartially-Blinded Unlearning (PBU), surpasses existing state-of-the-art class unlearning methods, demonstrating superior effectiveness.
  arXiv  Detail & Related papers  (2024-03-24T17:33:22Z)
• An Information Theoretic Approach to Machine Unlearning [43.423418819707784]
  To comply with AI and data regulations, the need to forget private or copyrighted information from trained machine learning models is increasingly important.<n>In this work, we address the zero-shot unlearning scenario, whereby an unlearning algorithm must be able to remove data given only a trained model and the data to be forgotten.<n>We derive a simple but principled zero-shot unlearning method based on the geometry of the model.
  arXiv  Detail & Related papers  (2024-02-02T13:33:30Z)
• RanPAC: Random Projections and Pre-trained Models for Continual Learning [59.07316955610658]
  Continual learning (CL) aims to learn different tasks (such as classification) in a non-stationary data stream without forgetting old ones. We propose a concise and effective approach for CL with pre-trained models.
  arXiv  Detail & Related papers  (2023-07-05T12:49:02Z)
• Retrieval-Enhanced Contrastive Vision-Text Models [61.783728119255365]
  We propose to equip vision-text models with the ability to refine their embedding with cross-modal retrieved information from a memory at inference time. Remarkably, we show that this can be done with a light-weight, single-layer, fusion transformer on top of a frozen CLIP. Our experiments validate that our retrieval-enhanced contrastive (RECO) training improves CLIP performance substantially on several challenging fine-grained tasks.
  arXiv  Detail & Related papers  (2023-06-12T15:52:02Z)
• No Token Left Behind: Explainability-Aided Image Classification and Generation [79.4957965474334]
  We present a novel explainability-based approach, which adds a loss term to ensure that CLIP focuses on all relevant semantic parts of the input. Our method yields an improvement in the recognition rate, without additional training or fine-tuning.
  arXiv  Detail & Related papers  (2022-04-11T07:16:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.