Approximate Domain Unlearning for Vision-Language Models

• URL: http://arxiv.org/abs/2510.08132v1
• Date: Thu, 09 Oct 2025 12:17:59 GMT
• Title: Approximate Domain Unlearning for Vision-Language Models
• Authors: Kodai Kawamura, Yuta Goto, Rintaro Yanagi, Hirokatsu Kataoka, Go Irie,
• Abstract summary: We introduce Approximate Domain Unlearning (ADU), a novel problem setting that requires reducing recognition accuracy for images from specified domains.<n>ADU presents new technical challenges due to the strong domain generalization capability of pre-trained Vision-Language Models.<n>We propose a novel approach that explicitly disentangles domain distributions and adaptively captures instance-specific domain information.
• Score: 22.8689039971533
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pre-trained Vision-Language Models (VLMs) exhibit strong generalization capabilities, enabling them to recognize a wide range of objects across diverse domains without additional training. However, they often retain irrelevant information beyond the requirements of specific downstream tasks, raising concerns about computational efficiency and potential information leakage. This has motivated growing interest in approximate unlearning, which aims to selectively remove unnecessary knowledge while preserving overall model performance. Existing approaches to approximate unlearning have primarily focused on class unlearning, where a VLM is retrained to fail to recognize specified object classes while maintaining accuracy for others. However, merely forgetting object classes is often insufficient in practical applications. For instance, an autonomous driving system should accurately recognize real cars while avoiding misrecognition of illustrated cars depicted in roadside advertisements as real cars, which could be hazardous. In this paper, we introduce Approximate Domain Unlearning (ADU), a novel problem setting that requires reducing recognition accuracy for images from specified domains (e.g., illustration) while preserving accuracy for other domains (e.g., real). ADU presents new technical challenges: due to the strong domain generalization capability of pre-trained VLMs, domain distributions are highly entangled in the feature space, making naive approaches based on penalizing target domains ineffective. To tackle this limitation, we propose a novel approach that explicitly disentangles domain distributions and adaptively captures instance-specific domain information. Extensive experiments show that our approach outperforms baselines built upon VLM tuning techniques, paving the way for practical and fine-grained unlearning in VLMs. Code: https://kodaikawamura.github.io/Domain_Unlearning/.

Related papers

• GRAIL: Gradient-Based Adaptive Unlearning for Privacy and Copyright in LLMs [26.13653211674955]
  Large Language Models (LLMs) trained on extensive datasets often learn sensitive information.<n>Retraining entire models from scratch to remove undesired information is both costly and impractical.<n>We propose GRAIL (GRadient-based AdaptIve unLearning), a novel multi-domain unlearning framework.
  arXiv  Detail & Related papers  (2025-04-17T06:16:32Z)
• Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Models [79.28821338925947]
  Domain-Class Incremental Learning is a realistic but challenging continual learning scenario. To handle these diverse tasks, pre-trained Vision-Language Models (VLMs) are introduced for their strong generalizability. This incurs a new problem: the knowledge encoded in the pre-trained VLMs may be disturbed when adapting to new tasks, compromising their inherent zero-shot ability. Existing methods tackle it by tuning VLMs with knowledge distillation on extra datasets, which demands heavy overhead. We propose the Distribution-aware Interference-free Knowledge Integration (DIKI) framework, retaining pre-trained knowledge of
  arXiv  Detail & Related papers  (2024-07-07T12:19:37Z)
• Towards Federated Domain Unlearning: Verification Methodologies and Challenges [34.9987941096371]
  We present the first comprehensive empirical study on Federated Domain Unlearning. Our findings reveal that unlearning disproportionately affects the model's deeper layers. We propose novel evaluation methodologies tailored for Federated Domain Unlearning.
  arXiv  Detail & Related papers  (2024-06-05T09:05:55Z)
• Self-Guided Masked Autoencoders for Domain-Agnostic Self-Supervised Learning [58.93724285214628]
  We present Self-guided Masked Autoencoders (SMA), a fully domain-agnostic masked modeling method. SMA trains an attention based model using a masked modeling objective, by learning masks to sample without any domain-specific assumptions. We evaluate SMA on three self-supervised learning benchmarks in protein biology, chemical property prediction, and particle physics.
  arXiv  Detail & Related papers  (2024-02-22T18:46:22Z)
• Domain-Controlled Prompt Learning [49.45309818782329]
  Existing prompt learning methods often lack domain-awareness or domain-transfer mechanisms. We propose a textbfDomain-Controlled Prompt Learning for the specific domains. Our method achieves state-of-the-art performance in specific domain image recognition datasets.
  arXiv  Detail & Related papers  (2023-09-30T02:59:49Z)
• CDFSL-V: Cross-Domain Few-Shot Learning for Videos [58.37446811360741]
  Few-shot video action recognition is an effective approach to recognizing new categories with only a few labeled examples. Existing methods in video action recognition rely on large labeled datasets from the same domain. We propose a novel cross-domain few-shot video action recognition method that leverages self-supervised learning and curriculum learning.
  arXiv  Detail & Related papers  (2023-09-07T19:44:27Z)
• Maximizing Model Generalization for Machine Condition Monitoring with Self-Supervised Learning and Federated Learning [4.214064911004321]
  Deep Learning can diagnose faults and assess machine health from raw condition monitoring data without manually designed statistical features. Traditional supervised learning may struggle to learn compact, discriminative representations that generalize to unseen target domains. This study proposes focusing on maximizing the feature generality on the source domain and applying TL via weight transfer to copy the model to the target domain.
  arXiv  Detail & Related papers  (2023-04-27T17:57:54Z)
• Domain Adaptive Semantic Segmentation without Source Data [50.18389578589789]
  We investigate domain adaptive semantic segmentation without source data, which assumes that the model is pre-trained on the source domain. We propose an effective framework for this challenging problem with two components: positive learning and negative learning. Our framework can be easily implemented and incorporated with other methods to further enhance the performance.
  arXiv  Detail & Related papers  (2021-10-13T04:12:27Z)
• Learning Task-oriented Disentangled Representations for Unsupervised Domain Adaptation [165.61511788237485]
  Unsupervised domain adaptation (UDA) aims to address the domain-shift problem between a labeled source domain and an unlabeled target domain. We propose a dynamic task-oriented disentangling network (DTDN) to learn disentangled representations in an end-to-end fashion for UDA.
  arXiv  Detail & Related papers  (2020-07-27T01:21:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.