Consistency Regularization for Domain Generalization with Logit Attribution Matching

• URL: http://arxiv.org/abs/2305.07888v2
• Date: Wed, 12 Jun 2024 13:14:07 GMT
• Title: Consistency Regularization for Domain Generalization with Logit Attribution Matching
• Authors: Han Gao, Kaican Li, Weiyan Xie, Zhi Lin, Yongxiang Huang, Luning Wang, Caleb Chen Cao, Nevin L. Zhang,
• Abstract summary: Domain generalization (DG) is about training models that generalize well under domain shift. We consider a third, lesser-known setting where a training domain is endowed with a collection of pairs of examples that share the same semantic information. We present a theory showing consistency regularization is conducive to DG and propose a novel CR method called Logit Matching.
• Score: 14.98337914353095
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Domain generalization (DG) is about training models that generalize well under domain shift. Previous research on DG has been conducted mostly in single-source or multi-source settings. In this paper, we consider a third, lesser-known setting where a training domain is endowed with a collection of pairs of examples that share the same semantic information. Such semantic sharing (SS) pairs can be created via data augmentation and then utilized for consistency regularization (CR). We present a theory showing CR is conducive to DG and propose a novel CR method called Logit Attribution Matching (LAM). We conduct experiments on five DG benchmarks and four pretrained models with SS pairs created by both generic and targeted data augmentation methods. LAM outperforms representative single/multi-source DG methods and various CR methods that leverage SS pairs. The code and data of this project are available at https://github.com/Gaohan123/LAM

Related papers

• Generative Classifier for Domain Generalization [84.92088101715116]
  Domain generalization aims to the generalizability of computer vision models toward distribution shifts. We propose Generative-driven Domain Generalization (GCDG) GCDG consists of three key modules: Heterogeneity Learning(HLC), Spurious Correlation(SCB), and Diverse Component Balancing(DCB)
  arXiv  Detail & Related papers  (2025-04-03T04:38:33Z)
• PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization [24.413415998529754]
  We propose a new benchmark Hybrid Domain Generalization (HDG) and a novel metric $H2$-CV, which construct various splits to assess the robustness of algorithms. Our method outperforms state-of-the-art algorithms on multiple datasets, especially improving the robustness when confronting data scarcity.
  arXiv  Detail & Related papers  (2024-04-13T13:41:13Z)
• Collaborating Foundation Models for Domain Generalized Semantic Segmentation [23.359941294938142]
  Domain Generalized Semantic (DGSS) deals with training a model on a labeled source domain. We take an approach to DGSS and propose to use an assembly of CoLlaborative FOUndation models for Domain Generalized Semantic (CLOUDS)
  arXiv  Detail & Related papers  (2023-12-15T13:43:24Z)
• Towards Reliable Domain Generalization: A New Dataset and Evaluations [45.68339440942477]
  We propose a new domain generalization task for handwritten Chinese character recognition (HCCR) We evaluate eighteen DG methods on the proposed PaHCC dataset and show that the performance of existing methods is still unsatisfactory. Our dataset and evaluations bring new perspectives to the community for more substantial progress.
  arXiv  Detail & Related papers  (2023-09-12T11:29:12Z)
• NormAUG: Normalization-guided Augmentation for Domain Generalization [60.159546669021346]
  We propose a simple yet effective method called NormAUG (Normalization-guided Augmentation) for deep learning. Our method introduces diverse information at the feature level and improves the generalization of the main path. In the test stage, we leverage an ensemble strategy to combine the predictions from the auxiliary path of our model, further boosting performance.
  arXiv  Detail & Related papers  (2023-07-25T13:35:45Z)
• Meta Adaptive Task Sampling for Few-Domain Generalization [43.2043988610497]
  Few-domain generalization (FDG) aims to learn a generalizable model from very few domains of novel tasks. We propose a Meta Adaptive Task Sampling (MATS) procedure to differentiate base tasks according to their semantic and domain-shift similarity to the novel task.
  arXiv  Detail & Related papers  (2023-05-25T01:44:09Z)
• Federated Domain Generalization for Image Recognition via Cross-Client Style Transfer [60.70102634957392]
  Domain generalization (DG) has been a hot topic in image recognition, with a goal to train a general model that can perform well on unseen domains. In this paper, we propose a novel domain generalization method for image recognition through cross-client style transfer (CCST) without exchanging data samples. Our method outperforms recent SOTA DG methods on two DG benchmarks (PACS, OfficeHome) and a large-scale medical image dataset (Camelyon17) in the FL setting.
  arXiv  Detail & Related papers  (2022-10-03T13:15:55Z)
• Compound Domain Generalization via Meta-Knowledge Encoding [55.22920476224671]
  We introduce Style-induced Domain-specific Normalization (SDNorm) to re-normalize the multi-modal underlying distributions. We harness the prototype representations, the centroids of classes, to perform relational modeling in the embedding space. Experiments on four standard Domain Generalization benchmarks reveal that COMEN exceeds the state-of-the-art performance without the need of domain supervision.
  arXiv  Detail & Related papers  (2022-03-24T11:54:59Z)
• Semi-Supervised Domain Generalization with Stochastic StyleMatch [90.98288822165482]
  In real-world applications, we might have only a few labels available from each source domain due to high annotation cost. In this work, we investigate semi-supervised domain generalization, a more realistic and practical setting. Our proposed approach, StyleMatch, is inspired by FixMatch, a state-of-the-art semi-supervised learning method based on pseudo-labeling.
  arXiv  Detail & Related papers  (2021-06-01T16:00:08Z)
• Dual Distribution Alignment Network for Generalizable Person Re-Identification [174.36157174951603]
  Domain generalization (DG) serves as a promising solution to handle person Re-Identification (Re-ID) We present a Dual Distribution Alignment Network (DDAN) which handles this challenge by selectively aligning distributions of multiple source domains. We evaluate our DDAN on a large-scale Domain Generalization Re-ID (DG Re-ID) benchmark.
  arXiv  Detail & Related papers  (2020-07-27T00:08:07Z)
• Learning to Generate Novel Domains for Domain Generalization [115.21519842245752]
  This paper focuses on the task of learning from multiple source domains a model that generalizes well to unseen domains. We employ a data generator to synthesize data from pseudo-novel domains to augment the source domains. Our method, L2A-OT, outperforms current state-of-the-art DG methods on four benchmark datasets.
  arXiv  Detail & Related papers  (2020-07-07T09:34:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.