AD-CLIP: Adapting Domains in Prompt Space Using CLIP
- URL: http://arxiv.org/abs/2308.05659v1
- Date: Thu, 10 Aug 2023 15:58:28 GMT
- Title: AD-CLIP: Adapting Domains in Prompt Space Using CLIP
- Authors: Mainak Singha, Harsh Pal, Ankit Jha, Biplab Banerjee
- Abstract summary: We introduce AD-CLIP, a domain-agnostic prompt learning strategy for CLIP that aims to solve the DA problem in the prompt space.
Our prompts are designed to be domain-invariant and class-generalizable, by conditioning prompt learning on image style and content features simultaneously.
Our experiments on three benchmark DA datasets demonstrate the effectiveness of AD-CLIP compared to existing literature.
- Score: 13.915653907503463
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Although deep learning models have shown impressive performance on supervised
learning tasks, they often struggle to generalize well when the training
(source) and test (target) domains differ. Unsupervised domain adaptation (DA)
has emerged as a popular solution to this problem. However, current DA
techniques rely on visual backbones, which may lack semantic richness. Despite
the potential of large-scale vision-language foundation models like CLIP, their
effectiveness for DA has yet to be fully explored. To address this gap, we
introduce AD-CLIP, a domain-agnostic prompt learning strategy for CLIP that
aims to solve the DA problem in the prompt space. We leverage the frozen vision
backbone of CLIP to extract both image style (domain) and content information,
which we apply to learn prompt tokens. Our prompts are designed to be
domain-invariant and class-generalizable, by conditioning prompt learning on
image style and content features simultaneously. We use standard supervised
contrastive learning in the source domain, while proposing an entropy
minimization strategy to align domains in the embedding space given the target
domain data. We also consider a scenario where only target domain samples are
available during testing, without any source domain data, and propose a
cross-domain style mapping network to hallucinate domain-agnostic tokens. Our
extensive experiments on three benchmark DA datasets demonstrate the
effectiveness of AD-CLIP compared to existing literature.
Related papers
- Domain-Controlled Prompt Learning [49.45309818782329]
Existing prompt learning methods often lack domain-awareness or domain-transfer mechanisms.
We propose a textbfDomain-Controlled Prompt Learning for the specific domains.
Our method achieves state-of-the-art performance in specific domain image recognition datasets.
arXiv Detail & Related papers (2023-09-30T02:59:49Z) - Domain Adaptive Few-Shot Open-Set Learning [36.39622440120531]
We propose Domain Adaptive Few-Shot Open Set Recognition (DA-FSOS) and introduce a meta-learning-based architecture named DAFOSNET.
Our training approach ensures that DAFOS-NET can generalize well to new scenarios in the target domain.
We present three benchmarks for DA-FSOS based on the Office-Home, mini-ImageNet/CUB, and DomainNet datasets.
arXiv Detail & Related papers (2023-09-22T12:04:47Z) - MoP-CLIP: A Mixture of Prompt-Tuned CLIP Models for Domain Incremental
Learning [12.737883740101438]
We present a novel DIL approach based on a mixture of prompt-tuned CLIP models (MoP-CLIP)
At the training stage we model the features distribution of every class in each domain, learning individual text and visual prompts to adapt to a given domain.
At inference, the learned distributions allow us to identify whether a given test sample belongs to a known domain, selecting the correct prompt for the classification task, or from an unseen domain.
arXiv Detail & Related papers (2023-07-11T18:17:50Z) - StyLIP: Multi-Scale Style-Conditioned Prompt Learning for CLIP-based
Domain Generalization [26.08922351077744]
StyLIP is a novel approach for Domain Generalization that enhances CLIP's classification performance across domains.
Our method focuses on a domain-agnostic prompt learning strategy, aiming to disentangle the visual style and content information embedded in CLIP's pre-trained vision encoder.
arXiv Detail & Related papers (2023-02-18T07:36:16Z) - Structured Latent Embeddings for Recognizing Unseen Classes in Unseen
Domains [108.11746235308046]
We propose a novel approach that learns domain-agnostic structured latent embeddings by projecting images from different domains.
Our experiments on the challenging DomainNet and DomainNet-LS benchmarks show the superiority of our approach over existing methods.
arXiv Detail & Related papers (2021-07-12T17:57:46Z) - Cross-domain Contrastive Learning for Unsupervised Domain Adaptation [108.63914324182984]
Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a fully-labeled source domain to a different unlabeled target domain.
We build upon contrastive self-supervised learning to align features so as to reduce the domain discrepancy between training and testing sets.
arXiv Detail & Related papers (2021-06-10T06:32:30Z) - Classes Matter: A Fine-grained Adversarial Approach to Cross-domain
Semantic Segmentation [95.10255219396109]
We propose a fine-grained adversarial learning strategy for class-level feature alignment.
We adopt a fine-grained domain discriminator that not only plays as a domain distinguisher, but also differentiates domains at class level.
An analysis with Class Center Distance (CCD) validates that our fine-grained adversarial strategy achieves better class-level alignment.
arXiv Detail & Related papers (2020-07-17T20:50:59Z) - Spatial Attention Pyramid Network for Unsupervised Domain Adaptation [66.75008386980869]
Unsupervised domain adaptation is critical in various computer vision tasks.
We design a new spatial attention pyramid network for unsupervised domain adaptation.
Our method performs favorably against the state-of-the-art methods by a large margin.
arXiv Detail & Related papers (2020-03-29T09:03:23Z) - Mind the Gap: Enlarging the Domain Gap in Open Set Domain Adaptation [65.38975706997088]
Open set domain adaptation (OSDA) assumes the presence of unknown classes in the target domain.
We show that existing state-of-the-art methods suffer a considerable performance drop in the presence of larger domain gaps.
We propose a novel framework to specifically address the larger domain gaps.
arXiv Detail & Related papers (2020-03-08T14:20:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.