DomCLP: Domain-wise Contrastive Learning with Prototype Mixup for Unsupervised Domain Generalization

• URL: http://arxiv.org/abs/2412.09074v1
• Date: Thu, 12 Dec 2024 08:59:08 GMT
• Title: DomCLP: Domain-wise Contrastive Learning with Prototype Mixup for Unsupervised Domain Generalization
• Authors: Jin-Seop Lee, Noo-ri Kim, Jee-Hyong Lee,
• Abstract summary: Self-supervised learning (SSL) methods based on the instance discrimination tasks with InfoNCE have achieved remarkable success. UDG approaches utilize contrastive learning with InfoNCE to generate representations. We propose DomCLP, Domain-wise Contrastive Learning with Prototype Mixup.
• Score: 10.343546104340962
• License:
• Abstract: Self-supervised learning (SSL) methods based on the instance discrimination tasks with InfoNCE have achieved remarkable success. Despite their success, SSL models often struggle to generate effective representations for unseen-domain data. To address this issue, research on unsupervised domain generalization (UDG), which aims to develop SSL models that can generate domain-irrelevant features, has been conducted. Most UDG approaches utilize contrastive learning with InfoNCE to generate representations, and perform feature alignment based on strong assumptions to generalize domain-irrelevant common features from multi-source domains. However, existing methods that rely on instance discrimination tasks are not effective at extracting domain-irrelevant common features. This leads to the suppression of domain-irrelevant common features and the amplification of domain-relevant features, thereby hindering domain generalization. Furthermore, strong assumptions underlying feature alignment can lead to biased feature learning, reducing the diversity of common features. In this paper, we propose a novel approach, DomCLP, Domain-wise Contrastive Learning with Prototype Mixup. We explore how InfoNCE suppresses domain-irrelevant common features and amplifies domain-relevant features. Based on this analysis, we propose Domain-wise Contrastive Learning (DCon) to enhance domain-irrelevant common features. We also propose Prototype Mixup Learning (PMix) to generalize domain-irrelevant common features across multiple domains without relying on strong assumptions. The proposed method consistently outperforms state-of-the-art methods on the PACS and DomainNet datasets across various label fractions, showing significant improvements. Our code will be released. Our project page is available at https://github.com/jinsuby/DomCLP.

Related papers

• Mitigating Domain Shift in Federated Learning via Intra- and Inter-Domain Prototypes [28.507105940018704]
  We introduce a novel federated prototype learning method, which incorporates $textbfI$ntra-domain and $textbfI$nter-domain. We propose feature alignment with MixUp-based augmented prototypes to capture the diversity of local domains and enhance the generalization of local features.
  arXiv  Detail & Related papers  (2025-01-15T02:17:38Z)
• DIGIC: Domain Generalizable Imitation Learning by Causal Discovery [69.13526582209165]
  Causality has been combined with machine learning to produce robust representations for domain generalization. We make a different attempt by leveraging the demonstration data distribution to discover causal features for a domain generalizable policy. We design a novel framework, called DIGIC, to identify the causal features by finding the direct cause of the expert action from the demonstration data distribution.
  arXiv  Detail & Related papers  (2024-02-29T07:09:01Z)
• Domain Generalization for Domain-Linked Classes [8.738092015092207]
  In the real-world, classes may often be domain-linked, i.e. expressed only in a specific domain. We propose a Fair and cONtrastive feature-space regularization algorithm for Domain-linked DG, FOND.
  arXiv  Detail & Related papers  (2023-06-01T16:39:50Z)
• Aggregation of Disentanglement: Reconsidering Domain Variations in Domain Generalization [9.577254317971933]
  We argue that the domain variantions also contain useful information, ie, classification-aware information, for downstream tasks. We propose a novel paradigm called Domain Disentanglement Network (DDN) to disentangle the domain expert features from the source domain images. We also propound a new contrastive learning method to guide the domain expert features to form a more balanced and separable feature space.
  arXiv  Detail & Related papers  (2023-02-05T09:48:57Z)
• Generalizable Representation Learning for Mixture Domain Face Anti-Spoofing [53.82826073959756]
  Face anti-spoofing approach based on domain generalization(DG) has drawn growing attention due to its robustness forunseen scenarios. We propose domain dy-namic adjustment meta-learning (D2AM) without using do-main labels. To overcome the limitation, we propose domain dy-namic adjustment meta-learning (D2AM) without using do-main labels.
  arXiv  Detail & Related papers  (2021-05-06T06:04:59Z)
• Batch Normalization Embeddings for Deep Domain Generalization [50.51405390150066]
  Domain generalization aims at training machine learning models to perform robustly across different and unseen domains. We show a significant increase in classification accuracy over current state-of-the-art techniques on popular domain generalization benchmarks.
  arXiv  Detail & Related papers  (2020-11-25T12:02:57Z)
• Learning to Balance Specificity and Invariance for In and Out of Domain Generalization [27.338573739304604]
  We introduce Domain-specific Masks for Generalization, a model for improving both in-domain and out-of-domain generalization performance. For domain generalization, the goal is to learn from a set of source domains to produce a single model that will best generalize to an unseen target domain. We demonstrate competitive performance compared to naive baselines and state-of-the-art methods on both PACS and DomainNet.
  arXiv  Detail & Related papers  (2020-08-28T20:39:51Z)
• Discrepancy Minimization in Domain Generalization with Generative Nearest Neighbors [13.047289562445242]
  Domain generalization (DG) deals with the problem of domain shift where a machine learning model trained on multiple-source domains fail to generalize well on a target domain with different statistics. Multiple approaches have been proposed to solve the problem of domain generalization by learning domain invariant representations across the source domains that fail to guarantee generalization on the shifted target domain. We propose a Generative Nearest Neighbor based Discrepancy Minimization (GNNDM) method which provides a theoretical guarantee that is upper bounded by the error in the labeling process of the target.
  arXiv  Detail & Related papers  (2020-07-28T14:54:25Z)
• Learning to Combine: Knowledge Aggregation for Multi-Source Domain Adaptation [56.694330303488435]
  We propose a Learning to Combine for Multi-Source Domain Adaptation (LtC-MSDA) framework. In the nutshell, a knowledge graph is constructed on the prototypes of various domains to realize the information propagation among semantically adjacent representations. Our approach outperforms existing methods with a remarkable margin.
  arXiv  Detail & Related papers  (2020-07-17T07:52:44Z)
• Domain Conditioned Adaptation Network [90.63261870610211]
  We propose a Domain Conditioned Adaptation Network (DCAN) to excite distinct convolutional channels with a domain conditioned channel attention mechanism. This is the first work to explore the domain-wise convolutional channel activation for deep DA networks.
  arXiv  Detail & Related papers  (2020-05-14T04:23:24Z)
• Mind the Gap: Enlarging the Domain Gap in Open Set Domain Adaptation [65.38975706997088]
  Open set domain adaptation (OSDA) assumes the presence of unknown classes in the target domain. We show that existing state-of-the-art methods suffer a considerable performance drop in the presence of larger domain gaps. We propose a novel framework to specifically address the larger domain gaps.
  arXiv  Detail & Related papers  (2020-03-08T14:20:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.