INSURE: An Information Theory Inspired Disentanglement and Purification Model for Domain Generalization

• URL: http://arxiv.org/abs/2309.04063v1
• Date: Fri, 8 Sep 2023 01:41:35 GMT
• Title: INSURE: An Information Theory Inspired Disentanglement and Purification Model for Domain Generalization
• Authors: Xi Yu, Huan-Hsin Tseng, Shinjae Yoo, Haibin Ling, Yuewei Lin
• Abstract summary: Domain Generalization (DG) aims to learn a generalizable model on the unseen target domain by only training on the multiple observed source domains. We propose an Information theory iNspired diSentanglement and pURification modEl (INSURE) to explicitly disentangle the latent features. We conduct experiments on four widely used DG benchmark datasets including PACS, OfficeHome, TerraIncognita, and DomainNet.
• Score: 55.86299081580768
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Domain Generalization (DG) aims to learn a generalizable model on the unseen target domain by only training on the multiple observed source domains. Although a variety of DG methods have focused on extracting domain-invariant features, the domain-specific class-relevant features have attracted attention and been argued to benefit generalization to the unseen target domain. To take into account the class-relevant domain-specific information, in this paper we propose an Information theory iNspired diSentanglement and pURification modEl (INSURE) to explicitly disentangle the latent features to obtain sufficient and compact (necessary) class-relevant feature for generalization to the unseen domain. Specifically, we first propose an information theory inspired loss function to ensure the disentangled class-relevant features contain sufficient class label information and the other disentangled auxiliary feature has sufficient domain information. We further propose a paired purification loss function to let the auxiliary feature discard all the class-relevant information and thus the class-relevant feature will contain sufficient and compact (necessary) class-relevant information. Moreover, instead of using multiple encoders, we propose to use a learnable binary mask as our disentangler to make the disentanglement more efficient and make the disentangled features complementary to each other. We conduct extensive experiments on four widely used DG benchmark datasets including PACS, OfficeHome, TerraIncognita, and DomainNet. The proposed INSURE outperforms the state-of-art methods. We also empirically show that domain-specific class-relevant features are beneficial for domain generalization.

Related papers

• Learning to Discover Knowledge: A Weakly-Supervised Partial Domain Adaptation Approach [20.899013563493202]
  Domain adaptation has shown appealing performance by leveraging knowledge from a source domain with rich annotations. For a specific target task, it is cumbersome to collect related and high-quality source domains. In this paper, we propose a simple yet effective domain adaptation approach, termed as self-paced transfer classifier learning (SP-TCL)
  arXiv  Detail & Related papers  (2024-06-20T12:54:07Z)
• Domain-aware Triplet loss in Domain Generalization [0.0]
  Domain shift is caused by discrepancies in the distributions of the testing and training data. We design a domainaware triplet loss for domain generalization to help the model to cluster similar semantic features. Our algorithm is designed to disperse domain information in the embedding space.
  arXiv  Detail & Related papers  (2023-03-01T14:02:01Z)
• Unsupervised Domain Adaptation via Style-Aware Self-intermediate Domain [52.783709712318405]
  Unsupervised domain adaptation (UDA) has attracted considerable attention, which transfers knowledge from a label-rich source domain to a related but unlabeled target domain. We propose a novel style-aware feature fusion method (SAFF) to bridge the large domain gap and transfer knowledge while alleviating the loss of class-discnative information.
  arXiv  Detail & Related papers  (2022-09-05T10:06:03Z)
• Compound Domain Generalization via Meta-Knowledge Encoding [55.22920476224671]
  We introduce Style-induced Domain-specific Normalization (SDNorm) to re-normalize the multi-modal underlying distributions. We harness the prototype representations, the centroids of classes, to perform relational modeling in the embedding space. Experiments on four standard Domain Generalization benchmarks reveal that COMEN exceeds the state-of-the-art performance without the need of domain supervision.
  arXiv  Detail & Related papers  (2022-03-24T11:54:59Z)
• Exploiting Domain-Specific Features to Enhance Domain Generalization [10.774902700296249]
  Domain Generalization (DG) aims to train a model, from multiple observed source domains, in order to perform well on unseen target domains. Prior DG approaches have focused on extracting domain-invariant information across sources to generalize on target domains. We propose meta-Domain Specific-Domain Invariant (mD) - a novel theoretically sound framework.
  arXiv  Detail & Related papers  (2021-10-18T15:42:39Z)
• COLUMBUS: Automated Discovery of New Multi-Level Features for Domain Generalization via Knowledge Corruption [12.555885317622131]
  We address the challenging domain generalization problem, where a model trained on a set of source domains is expected to generalize well in unseen domains without exposure to their data. We propose Columbus, a method that enforces new feature discovery via a targeted corruption of the most relevant input and multi-level representations of the data.
  arXiv  Detail & Related papers  (2021-09-09T14:52:05Z)
• Open Set Domain Recognition via Attention-Based GCN and Semantic Matching Optimization [8.831857715361624]
  This work presents an end-to-end model based on attention-based GCN and semantic matching optimization. Experimental results validate that the proposed model not only has superiority on recognizing the images of known and unknown classes, but also can adapt to various openness of the target domain.
  arXiv  Detail & Related papers  (2021-05-11T12:05:36Z)
• Inferring Latent Domains for Unsupervised Deep Domain Adaptation [54.963823285456925]
  Unsupervised Domain Adaptation (UDA) refers to the problem of learning a model in a target domain where labeled data are not available. This paper introduces a novel deep architecture which addresses the problem of UDA by automatically discovering latent domains in visual datasets. We evaluate our approach on publicly available benchmarks, showing that it outperforms state-of-the-art domain adaptation methods.
  arXiv  Detail & Related papers  (2021-03-25T14:33:33Z)
• Adaptively-Accumulated Knowledge Transfer for Partial Domain Adaptation [66.74638960925854]
  Partial domain adaptation (PDA) deals with a realistic and challenging problem when the source domain label space substitutes the target domain. We propose an Adaptively-Accumulated Knowledge Transfer framework (A$2$KT) to align the relevant categories across two domains.
  arXiv  Detail & Related papers  (2020-08-27T00:53:43Z)
• Mind the Gap: Enlarging the Domain Gap in Open Set Domain Adaptation [65.38975706997088]
  Open set domain adaptation (OSDA) assumes the presence of unknown classes in the target domain. We show that existing state-of-the-art methods suffer a considerable performance drop in the presence of larger domain gaps. We propose a novel framework to specifically address the larger domain gaps.
  arXiv  Detail & Related papers  (2020-03-08T14:20:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.