TDG: Text-guided Domain Generalization

• URL: http://arxiv.org/abs/2308.09931v1
• Date: Sat, 19 Aug 2023 07:21:02 GMT
• Title: TDG: Text-guided Domain Generalization
• Authors: Geng Liu and Yuxi Wang
• Abstract summary: We develop a Text-guided Domain Generalization (TDG) paradigm for domain generalization. We first devise an automatic words generation method to extend the description of current domains with novel domain-relevant words. Then, we embed the generated domain information into the text feature space, by the proposed prompt learning-based text feature generation method. Finally, we utilize both input image features and generated text features to train a specially designed classifier that generalizes well on unseen target domains.
• Score: 10.322052096998728
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Domain generalization (DG) attempts to generalize a model trained on single or multiple source domains to the unseen target domain. Benefiting from the success of Visual-and-Language Pre-trained models in recent years, we argue that it is crucial for domain generalization by introducing extra text information. In this paper, we develop a novel Text-guided Domain Generalization (TDG) paradigm for domain generalization, which includes three following aspects. Specifically, we first devise an automatic words generation method to extend the description of current domains with novel domain-relevant words. Then, we embed the generated domain information into the text feature space, by the proposed prompt learning-based text feature generation method, which shares a common representation space with the image feature. Finally, we utilize both input image features and generated text features to train a specially designed classifier that generalizes well on unseen target domains, while the image encoder is also updated under the supervision of gradients back propagated from the classifier. Our experimental results show that the techniques incorporated by TDG contribute to the performance in an easy implementation manner. Experimental results on several domain generalization benchmarks show that our proposed framework achieves superior performance by effectively leveraging generated text information in domain generalization.

Related papers

• Learning to Generalize Unseen Domains via Multi-Source Meta Learning for Text Classification [71.08024880298613]
  We study the multi-source Domain Generalization of text classification. We propose a framework to use multiple seen domains to train a model that can achieve high accuracy in an unseen domain.
  arXiv  Detail & Related papers  (2024-09-20T07:46:21Z)
• A Unified Data Augmentation Framework for Low-Resource Multi-Domain Dialogue Generation [52.0964459842176]
  Current state-of-the-art dialogue systems heavily rely on extensive training datasets. We propose a novel data textbfAugmentation framework for textbfMulti-textbfDomain textbfDialogue textbfGeneration, referred to as textbfAMD$2$G. The AMD$2$G framework consists of a data augmentation process and a two-stage training approach: domain-agnostic training and domain adaptation training.
  arXiv  Detail & Related papers  (2024-06-14T09:52:27Z)
• TeG-DG: Textually Guided Domain Generalization for Face Anti-Spoofing [8.830873674673828]
  Existing methods are dedicated to extracting domain-invariant features from various training domains. The extracted features inevitably contain residual style feature bias, resulting in inferior generalization performance. We propose the Textually Guided Domain Generalization framework, which can effectively leverage text information for cross-domain alignment.
  arXiv  Detail & Related papers  (2023-11-30T10:13:46Z)
• Prompting Diffusion Representations for Cross-Domain Semantic Segmentation [101.04326113360342]
  diffusion-pretraining achieves extraordinary domain generalization results for semantic segmentation. We introduce a scene prompt and a prompt randomization strategy to help further disentangle the domain-invariant information when training the segmentation head.
  arXiv  Detail & Related papers  (2023-07-05T09:28:25Z)
• Language-aware Domain Generalization Network for Cross-Scene Hyperspectral Image Classification [15.842081807249416]
  It is necessary to explore the effectiveness of linguistic mode in assisting hyperspectral image classification. Large-scale pre-training image-text foundation models have demonstrated great performance in a variety of downstream applications. A Language-aware Domain Generalization Network (LDGnet) is proposed to learn cross-domain invariant representation.
  arXiv  Detail & Related papers  (2022-09-06T10:06:10Z)
• Grounding Visual Representations with Texts for Domain Generalization [9.554646174100123]
  Cross-modality supervision can be successfully used to ground domain-invariant visual representations. Our proposed method achieves state-of-the-art results and ranks 1st in average performance for five multi-domain datasets.
  arXiv  Detail & Related papers  (2022-07-21T03:43:38Z)
• INDIGO: Intrinsic Multimodality for Domain Generalization [26.344372409315177]
  We study how multimodal information can be leveraged in an "intrinsic" way to make systems generalize under unseen domains. We propose IntriNsic multimodality for DomaIn GeneralizatiOn (INDIGO)
  arXiv  Detail & Related papers  (2022-06-13T05:41:09Z)
• Compound Domain Generalization via Meta-Knowledge Encoding [55.22920476224671]
  We introduce Style-induced Domain-specific Normalization (SDNorm) to re-normalize the multi-modal underlying distributions. We harness the prototype representations, the centroids of classes, to perform relational modeling in the embedding space. Experiments on four standard Domain Generalization benchmarks reveal that COMEN exceeds the state-of-the-art performance without the need of domain supervision.
  arXiv  Detail & Related papers  (2022-03-24T11:54:59Z)
• Unsupervised Domain Generalization by Learning a Bridge Across Domains [78.855606355957]
  Unsupervised Domain Generalization (UDG) setup has no training supervision in neither source nor target domains. Our approach is based on self-supervised learning of a Bridge Across Domains (BrAD) - an auxiliary bridge domain accompanied by a set of semantics preserving visual (image-to-image) mappings to BrAD from each of the training domains. We show how using an edge-regularized BrAD our approach achieves significant gains across multiple benchmarks and a range of tasks, including UDG, Few-shot UDA, and unsupervised generalization across multi-domain datasets.
  arXiv  Detail & Related papers  (2021-12-04T10:25:45Z)
• Exploiting Domain-Specific Features to Enhance Domain Generalization [10.774902700296249]
  Domain Generalization (DG) aims to train a model, from multiple observed source domains, in order to perform well on unseen target domains. Prior DG approaches have focused on extracting domain-invariant information across sources to generalize on target domains. We propose meta-Domain Specific-Domain Invariant (mD) - a novel theoretically sound framework.
  arXiv  Detail & Related papers  (2021-10-18T15:42:39Z)
• Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains [108.11746235308046]
  We propose a novel approach that learns domain-agnostic structured latent embeddings by projecting images from different domains. Our experiments on the challenging DomainNet and DomainNet-LS benchmarks show the superiority of our approach over existing methods.
  arXiv  Detail & Related papers  (2021-07-12T17:57:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.