Encouraging Intra-Class Diversity Through a Reverse Contrastive Loss for
Better Single-Source Domain Generalization
- URL: http://arxiv.org/abs/2106.07916v1
- Date: Tue, 15 Jun 2021 07:04:39 GMT
- Title: Encouraging Intra-Class Diversity Through a Reverse Contrastive Loss for
Better Single-Source Domain Generalization
- Authors: Thomas Duboudin (imagine), Emmanuel Dellandr\'ea, Corentin Abgrall,
Gilles H\'enaff, Liming Chen
- Abstract summary: Traditional deep learning algorithms often fail to generalize when they are tested outside of the domain of training data.
This paper aims to develop deep learning algorithms able to generalize from a single training domain where no information about the test domain is available at training time.
- Score: 4.298298881279499
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Traditional deep learning algorithms often fail to generalize when they are
tested outside of the domain of training data. Because data distributions can
change dynamically in real-life applications once a learned model is deployed,
in this paper we are interested in single-source domain generalization (SDG)
which aims to develop deep learning algorithms able to generalize from a single
training domain where no information about the test domain is available at
training time. Firstly, we design two simple MNISTbased SDG benchmarks, namely
MNIST Color SDG-MP and MNIST Color SDG-UP, which highlight the two different
fundamental SDG issues of increasing difficulties: 1) a class-correlated
pattern in the training domain is missing (SDG-MP), or 2) uncorrelated with the
class (SDG-UP), in the testing data domain. This is in sharp contrast with the
current domain generalization (DG) benchmarks which mix up different
correlation and variation factors and thereby make hard to disentangle success
or failure factors when benchmarking DG algorithms. We further evaluate several
state-of-the-art SDG algorithms through our simple benchmark, namely MNIST
Color SDG-MP, and show that the issue SDG-MP is largely unsolved despite of a
decade of efforts in developing DG algorithms. Finally, we also propose a
partially reversed contrastive loss to encourage intra-class diversity and find
less strongly correlated patterns, to deal with SDG-MP and show that the
proposed approach is very effective on our MNIST Color SDG-MP benchmark.
Related papers
- Disentangling Masked Autoencoders for Unsupervised Domain Generalization [57.56744870106124]
Unsupervised domain generalization is fast gaining attention but is still far from well-studied.
Disentangled Masked Auto (DisMAE) aims to discover the disentangled representations that faithfully reveal intrinsic features.
DisMAE co-trains the asymmetric dual-branch architecture with semantic and lightweight variation encoders.
arXiv Detail & Related papers (2024-07-10T11:11:36Z) - Grounding Stylistic Domain Generalization with Quantitative Domain Shift Measures and Synthetic Scene Images [63.58800688320182]
Domain Generalization is a challenging task in machine learning.
Current methodology lacks quantitative understanding about shifts in stylistic domain.
We introduce a new DG paradigm to address these risks.
arXiv Detail & Related papers (2024-05-24T22:13:31Z) - PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization [24.413415998529754]
We propose a new benchmark Hybrid Domain Generalization (HDG) and a novel metric $H2$-CV, which construct various splits to assess the robustness of algorithms.
Our method outperforms state-of-the-art algorithms on multiple datasets, especially improving the robustness when confronting data scarcity.
arXiv Detail & Related papers (2024-04-13T13:41:13Z) - Activate and Reject: Towards Safe Domain Generalization under Category
Shift [71.95548187205736]
We study a practical problem of Domain Generalization under Category Shift (DGCS)
It aims to simultaneously detect unknown-class samples and classify known-class samples in the target domains.
Compared to prior DG works, we face two new challenges: 1) how to learn the concept of unknown'' during training with only source known-class samples, and 2) how to adapt the source-trained model to unseen environments.
arXiv Detail & Related papers (2023-10-07T07:53:12Z) - FIXED: Frustratingly Easy Domain Generalization with Mixup [53.782029033068675]
Domain generalization (DG) aims to learn a generalizable model from multiple training domains such that it can perform well on unseen target domains.
A popular strategy is to augment training data to benefit generalization through methods such as Mixupcitezhang 2018mixup.
We propose a simple yet effective enhancement for Mixup-based DG, namely domain-invariant Feature mIXup (FIX)
Our approach significantly outperforms nine state-of-the-art related methods, beating the best performing baseline by 6.5% on average in terms of test accuracy.
arXiv Detail & Related papers (2022-11-07T09:38:34Z) - Compound Domain Generalization via Meta-Knowledge Encoding [55.22920476224671]
We introduce Style-induced Domain-specific Normalization (SDNorm) to re-normalize the multi-modal underlying distributions.
We harness the prototype representations, the centroids of classes, to perform relational modeling in the embedding space.
Experiments on four standard Domain Generalization benchmarks reveal that COMEN exceeds the state-of-the-art performance without the need of domain supervision.
arXiv Detail & Related papers (2022-03-24T11:54:59Z) - Improving Multi-Domain Generalization through Domain Re-labeling [31.636953426159224]
We study the important link between pre-specified domain labels and the generalization performance.
We introduce a general approach for multi-domain generalization, MulDEns, that uses an ERM-based deep ensembling backbone.
We show that MulDEns does not require tailoring the augmentation strategy or the training process specific to a dataset.
arXiv Detail & Related papers (2021-12-17T23:21:50Z) - Reappraising Domain Generalization in Neural Networks [8.06370138649329]
Domain generalization (DG) of machine learning algorithms is defined as their ability to learn a domain agnostic hypothesis from multiple training distributions.
We find that a straightforward Empirical Risk Minimization (ERM) baseline consistently outperforms existing DG methods.
We propose a classwise-DG formulation, where for each class, we randomly select one of the domains and keep it aside for testing.
arXiv Detail & Related papers (2021-10-15T10:06:40Z) - HCDG: A Hierarchical Consistency Framework for Domain Generalization on
Medical Image Segmentation [33.623948922908184]
We present a novel Hierarchical Consistency framework for Domain Generalization (HCDG)
For the Extrinsic Consistency, we leverage the knowledge across multiple source domains to enforce data-level consistency.
For the Intrinsic Consistency, we perform task-level consistency for the same instance under the dual-task scenario.
arXiv Detail & Related papers (2021-09-13T07:07:23Z) - Robust Domain-Free Domain Generalization with Class-aware Alignment [4.442096198968069]
Domain-Free Domain Generalization (DFDG) is a model-agnostic method to achieve better generalization performance on the unseen test domain.
DFDG uses novel strategies to learn domain-invariant class-discriminative features.
It obtains competitive performance on both time series sensor and image classification public datasets.
arXiv Detail & Related papers (2021-02-17T17:46:06Z) - Cross-Domain Facial Expression Recognition: A Unified Evaluation
Benchmark and Adversarial Graph Learning [85.6386289476598]
We develop a novel adversarial graph representation adaptation (AGRA) framework for cross-domain holistic-local feature co-adaptation.
We conduct extensive and fair evaluations on several popular benchmarks and show that the proposed AGRA framework outperforms previous state-of-the-art methods.
arXiv Detail & Related papers (2020-08-03T15:00:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.