Gated Domain Units for Multi-source Domain Generalization
- URL: http://arxiv.org/abs/2206.12444v2
- Date: Tue, 16 May 2023 14:32:42 GMT
- Title: Gated Domain Units for Multi-source Domain Generalization
- Authors: Simon F\"oll, Alina Dubatovka, Eugen Ernst, Siu Lun Chau, Martin
Maritsch, Patrik Okanovic, Gudrun Th\"ater, Joachim M. Buhmann, Felix
Wortmann, Krikamol Muandet
- Abstract summary: Distribution shift (DS) occurs when a dataset at test time differs from the dataset at training time.
We introduce a modular neural network layer consisting of Gated Domain Units (GDUs) that learn a representation for each latent elementary distribution.
During inference, a weighted ensemble of learning machines can be created by comparing new observations with the representations of each elementary distribution.
- Score: 14.643490853965385
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The phenomenon of distribution shift (DS) occurs when a dataset at test time
differs from the dataset at training time, which can significantly impair the
performance of a machine learning model in practical settings due to a lack of
knowledge about the data's distribution at test time. To address this problem,
we postulate that real-world distributions are composed of latent Invariant
Elementary Distributions (I.E.D) across different domains. This assumption
implies an invariant structure in the solution space that enables knowledge
transfer to unseen domains. To exploit this property for domain generalization,
we introduce a modular neural network layer consisting of Gated Domain Units
(GDUs) that learn a representation for each latent elementary distribution.
During inference, a weighted ensemble of learning machines can be created by
comparing new observations with the representations of each elementary
distribution. Our flexible framework also accommodates scenarios where explicit
domain information is not present. Extensive experiments on image, text, and
graph data show consistent performance improvement on out-of-training target
domains. These findings support the practicality of the I.E.D assumption and
the effectiveness of GDUs for domain generalisation.
Related papers
- Non-stationary Domain Generalization: Theory and Algorithm [11.781050299571692]
In this paper, we study domain generalization in non-stationary environment.
We first examine the impact of environmental non-stationarity on model performance.
Then, we propose a novel algorithm based on adaptive invariant representation learning.
arXiv Detail & Related papers (2024-05-10T21:32:43Z) - DIGIC: Domain Generalizable Imitation Learning by Causal Discovery [69.13526582209165]
Causality has been combined with machine learning to produce robust representations for domain generalization.
We make a different attempt by leveraging the demonstration data distribution to discover causal features for a domain generalizable policy.
We design a novel framework, called DIGIC, to identify the causal features by finding the direct cause of the expert action from the demonstration data distribution.
arXiv Detail & Related papers (2024-02-29T07:09:01Z) - Multi-Domain Causal Representation Learning via Weak Distributional
Invariances [27.72497122405241]
Causal representation learning has emerged as the center of action in causal machine learning research.
We show that autoencoders that incorporate such invariances can provably identify the stable set of latents from the rest across different settings.
arXiv Detail & Related papers (2023-10-04T14:41:41Z) - Improving Domain Generalization with Domain Relations [77.63345406973097]
This paper focuses on domain shifts, which occur when the model is applied to new domains that are different from the ones it was trained on.
We propose a new approach called D$3$G to learn domain-specific models.
Our results show that D$3$G consistently outperforms state-of-the-art methods.
arXiv Detail & Related papers (2023-02-06T08:11:16Z) - Towards Generalization on Real Domain for Single Image Dehazing via
Meta-Learning [41.99615673136883]
Internal information learned from synthesized images is usually sub-optimal in real domains.
We present a domain generalization framework based on meta-learning to dig out representative internal properties of real hazy domains.
Our proposed method has superior generalization ability than the state-of-the-art competitors.
arXiv Detail & Related papers (2022-11-14T07:04:00Z) - AFAN: Augmented Feature Alignment Network for Cross-Domain Object
Detection [90.18752912204778]
Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications.
We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training.
Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
arXiv Detail & Related papers (2021-06-10T05:01:20Z) - Inferring Latent Domains for Unsupervised Deep Domain Adaptation [54.963823285456925]
Unsupervised Domain Adaptation (UDA) refers to the problem of learning a model in a target domain where labeled data are not available.
This paper introduces a novel deep architecture which addresses the problem of UDA by automatically discovering latent domains in visual datasets.
We evaluate our approach on publicly available benchmarks, showing that it outperforms state-of-the-art domain adaptation methods.
arXiv Detail & Related papers (2021-03-25T14:33:33Z) - Robust Domain-Free Domain Generalization with Class-aware Alignment [4.442096198968069]
Domain-Free Domain Generalization (DFDG) is a model-agnostic method to achieve better generalization performance on the unseen test domain.
DFDG uses novel strategies to learn domain-invariant class-discriminative features.
It obtains competitive performance on both time series sensor and image classification public datasets.
arXiv Detail & Related papers (2021-02-17T17:46:06Z) - Learning Invariant Representations and Risks for Semi-supervised Domain
Adaptation [109.73983088432364]
We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA)
We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
arXiv Detail & Related papers (2020-10-09T15:42:35Z) - Dual Distribution Alignment Network for Generalizable Person
Re-Identification [174.36157174951603]
Domain generalization (DG) serves as a promising solution to handle person Re-Identification (Re-ID)
We present a Dual Distribution Alignment Network (DDAN) which handles this challenge by selectively aligning distributions of multiple source domains.
We evaluate our DDAN on a large-scale Domain Generalization Re-ID (DG Re-ID) benchmark.
arXiv Detail & Related papers (2020-07-27T00:08:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.