UPRE: Zero-Shot Domain Adaptation for Object Detection via Unified Prompt and Representation Enhancement

• URL: http://arxiv.org/abs/2507.00721v2
• Date: Mon, 21 Jul 2025 11:56:13 GMT
• Title: UPRE: Zero-Shot Domain Adaptation for Object Detection via Unified Prompt and Representation Enhancement
• Authors: Xiao Zhang, Fei Wei, Yong Wang, Wenda Zhao, Feiyi Li, Xiangxiang Chu,
• Abstract summary: Zero-shot domain adaptation (ZSDA) presents substantial challenges due to the lack of images in the target domain.<n>Previous approaches leverage Vision-Language Models (VLMs) to tackle this challenge.<n>We propose the unified prompt and representation enhancement (UPRE) framework, which jointly optimize both textual prompts and visual representations.
• Score: 25.139037597606233
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-shot domain adaptation (ZSDA) presents substantial challenges due to the lack of images in the target domain. Previous approaches leverage Vision-Language Models (VLMs) to tackle this challenge, exploiting their zero-shot learning capabilities. However, these methods primarily address domain distribution shifts and overlook the misalignment between the detection task and VLMs, which rely on manually crafted prompts. To overcome these limitations, we propose the unified prompt and representation enhancement (UPRE) framework, which jointly optimizes both textual prompts and visual representations. Specifically, our approach introduces a multi-view domain prompt that combines linguistic domain priors with detection-specific knowledge, and a visual representation enhancement module that produces domain style variations. Furthermore, we introduce multi-level enhancement strategies, including relative domain distance and positive-negative separation, which align multi-modal representations at the image level and capture diverse visual representations at the instance level, respectively. Extensive experiments conducted on nine benchmark datasets demonstrate the superior performance of our framework in ZSDA detection scenarios. Code is available at https://github.com/AMAP-ML/UPRE.

Related papers

• Prompt Disentanglement via Language Guidance and Representation Alignment for Domain Generalization [75.88719716002014]
  Domain Generalization (DG) seeks to develop a versatile model capable of performing effectively on unseen target domains.<n>Recent advances in pre-trained Visual Foundation Models (VFMs) have demonstrated considerable potential in enhancing the generalization capabilities of deep learning models.<n>We propose addressing this challenge by leveraging the controllable and flexible language prompt of the VFM.
  arXiv  Detail & Related papers  (2025-07-03T03:52:37Z)
• Boosting Single-domain Generalized Object Detection via Vision-Language Knowledge Interaction [4.692621855184482]
  Single-Domain Generalized Object Detection(S-DGOD) aims to train an object detector on a single source domain.<n>Recent S-DGOD approaches exploit pre-trained vision-language knowledge to guide invariant feature learning across visual domains.<n>We propose a new cross-modal feature learning method, which can capture generalized and discriminative regional features for S-DGOD tasks.
  arXiv  Detail & Related papers  (2025-04-27T02:55:54Z)
• Cross-domain Few-shot Object Detection with Multi-modal Textual Enrichment [21.36633828492347]
  Cross-Domain Multi-Modal Few-Shot Object Detection (CDMM-FSOD)<n>We introduce a meta-learning-based framework designed to leverage rich textual semantics as an auxiliary modality to achieve effective domain adaptation.<n>We evaluate the proposed method on common cross-domain object detection benchmarks and demonstrate that it significantly surpasses existing few-shot object detection approaches.
  arXiv  Detail & Related papers  (2025-02-23T06:59:22Z)
• GlocalCLIP: Object-agnostic Global-Local Prompt Learning for Zero-shot Anomaly Detection [5.530212768657544]
  We introduce glocal contrastive learning to improve the complementary learning of global and local prompts.<n>The generalization performance of GlocalCLIP in ZSAD was demonstrated on 15 real-world datasets.
  arXiv  Detail & Related papers  (2024-11-09T05:22:13Z)
• SSPA: Split-and-Synthesize Prompting with Gated Alignments for Multi-Label Image Recognition [71.90536979421093]
  We propose a Split-and-Synthesize Prompting with Gated Alignments (SSPA) framework to amplify the potential of Vision-Language Models (VLMs) We develop an in-context learning approach to associate the inherent knowledge from LLMs. Then we propose a novel Split-and-Synthesize Prompting (SSP) strategy to first model the generic knowledge and downstream label semantics individually.
  arXiv  Detail & Related papers  (2024-07-30T15:58:25Z)
• WIDIn: Wording Image for Domain-Invariant Representation in Single-Source Domain Generalization [63.98650220772378]
  We present WIDIn, Wording Images for Domain-Invariant representation, to disentangle discriminative visual representation. We first estimate the language embedding with fine-grained alignment, which can be used to adaptively identify and then remove domain-specific counterpart. We show that WIDIn can be applied to both pretrained vision-language models like CLIP, and separately trained uni-modal models like MoCo and BERT.
  arXiv  Detail & Related papers  (2024-05-28T17:46:27Z)
• Progressive Multi-modal Conditional Prompt Tuning [92.50645776024624]
  Pre-trained vision-language models (VLMs) have shown remarkable generalization capabilities via prompting. We propose a novel method, Progressive Multi-modal conditional Prompt Tuning (ProMPT) ProMPT exploits a recurrent structure, optimizing and aligning V-L features by iteratively utilizing image and current encoding information.
  arXiv  Detail & Related papers  (2024-04-18T02:40:31Z)
• Language Guided Domain Generalized Medical Image Segmentation [68.93124785575739]
  Single source domain generalization holds promise for more reliable and consistent image segmentation across real-world clinical settings. We propose an approach that explicitly leverages textual information by incorporating a contrastive learning mechanism guided by the text encoder features. Our approach achieves favorable performance against existing methods in literature.
  arXiv  Detail & Related papers  (2024-04-01T17:48:15Z)
• C-SAW: Self-Supervised Prompt Learning for Image Generalization in Remote Sensing [12.930814370829893]
  We focus on domain and class generalization problems in analyzing optical remote sensing images, using the large-scale pre-trained vision-language model (VLM), CLIP. Existing prompt learning techniques overlook the importance of incorporating domain and content information into the prompts. We propose a solution that ensures domain-invariant prompt learning while enhancing the expressiveness of visual features.
  arXiv  Detail & Related papers  (2023-11-27T13:35:20Z)
• Domain-Controlled Prompt Learning [49.45309818782329]
  Existing prompt learning methods often lack domain-awareness or domain-transfer mechanisms. We propose a textbfDomain-Controlled Prompt Learning for the specific domains. Our method achieves state-of-the-art performance in specific domain image recognition datasets.
  arXiv  Detail & Related papers  (2023-09-30T02:59:49Z)
• Domain-Aware Continual Zero-Shot Learning [52.349332188116975]
  Domain-Aware Continual Zero-Shot Learning (DACZSL) is a task to recognize images of unseen categories in continuously changing domains. We propose a Domain-Invariant Network (DIN) to learn factorized features for shifting domains and improved textual representation for unseen classes. Our results show that DIN significantly outperforms existing baselines by over 5% in harmonic accuracy and over 1% in backward transfer.
  arXiv  Detail & Related papers  (2021-12-24T08:17:18Z)
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection [90.18752912204778]
  Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
  arXiv  Detail & Related papers  (2021-06-10T05:01:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.