SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of
Invariances in Domain Generalization
- URL: http://arxiv.org/abs/2106.02266v1
- Date: Fri, 4 Jun 2021 05:20:54 GMT
- Title: SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of
Invariances in Domain Generalization
- Authors: Soroosh Shahtalebi, Jean-Christophe Gagnon-Audet, Touraj Laleh,
Mojtaba Faramarzi, Kartik Ahuja, Irina Rish
- Abstract summary: We propose a masking strategy, which determines a continuous weight based on the agreement of gradients that flow in each edge of network.
SAND-mask is validated over the Domainbed benchmark for domain generalization.
- Score: 7.253255826783766
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: A major bottleneck in the real-world applications of machine learning models
is their failure in generalizing to unseen domains whose data distribution is
not i.i.d to the training domains. This failure often stems from learning
non-generalizable features in the training domains that are spuriously
correlated with the label of data. To address this shortcoming, there has been
a growing surge of interest in learning good explanations that are hard to
vary, which is studied under the notion of Out-of-Distribution (OOD)
Generalization. The search for good explanations that are \textit{invariant}
across different domains can be seen as finding local (global) minimas in the
loss landscape that hold true across all of the training domains. In this
paper, we propose a masking strategy, which determines a continuous weight
based on the agreement of gradients that flow in each edge of network, in order
to control the amount of update received by the edge in each step of
optimization. Particularly, our proposed technique referred to as "Smoothed-AND
(SAND)-masking", not only validates the agreement in the direction of gradients
but also promotes the agreement among their magnitudes to further ensure the
discovery of invariances across training domains. SAND-mask is validated over
the Domainbed benchmark for domain generalization and significantly improves
the state-of-the-art accuracy on the Colored MNIST dataset while providing
competitive results on other domain generalization datasets.
Related papers
- Disentangling Masked Autoencoders for Unsupervised Domain Generalization [57.56744870106124]
Unsupervised domain generalization is fast gaining attention but is still far from well-studied.
Disentangled Masked Auto (DisMAE) aims to discover the disentangled representations that faithfully reveal intrinsic features.
DisMAE co-trains the asymmetric dual-branch architecture with semantic and lightweight variation encoders.
arXiv Detail & Related papers (2024-07-10T11:11:36Z) - META: Mimicking Embedding via oThers' Aggregation for Generalizable
Person Re-identification [68.39849081353704]
Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time.
This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
arXiv Detail & Related papers (2021-12-16T08:06:50Z) - Fishr: Invariant Gradient Variances for Out-of-distribution
Generalization [98.40583494166314]
Fishr is a learning scheme to enforce domain invariance in the space of the gradients of the loss function.
Fishr exhibits close relations with the Fisher Information and the Hessian of the loss.
In particular, Fishr improves the state of the art on the DomainBed benchmark and performs significantly better than Empirical Risk Minimization.
arXiv Detail & Related papers (2021-09-07T08:36:09Z) - Discriminative Domain-Invariant Adversarial Network for Deep Domain
Generalization [33.84004077585957]
We propose a discriminative domain-invariant adversarial network (DDIAN) for domain generalization.
DDIAN achieves better prediction on unseen target data during training compared to state-of-the-art domain generalization approaches.
arXiv Detail & Related papers (2021-08-20T04:24:12Z) - Generalizable Representation Learning for Mixture Domain Face
Anti-Spoofing [53.82826073959756]
Face anti-spoofing approach based on domain generalization(DG) has drawn growing attention due to its robustness forunseen scenarios.
We propose domain dy-namic adjustment meta-learning (D2AM) without using do-main labels.
To overcome the limitation, we propose domain dy-namic adjustment meta-learning (D2AM) without using do-main labels.
arXiv Detail & Related papers (2021-05-06T06:04:59Z) - Robust Domain-Free Domain Generalization with Class-aware Alignment [4.442096198968069]
Domain-Free Domain Generalization (DFDG) is a model-agnostic method to achieve better generalization performance on the unseen test domain.
DFDG uses novel strategies to learn domain-invariant class-discriminative features.
It obtains competitive performance on both time series sensor and image classification public datasets.
arXiv Detail & Related papers (2021-02-17T17:46:06Z) - Domain Adaptation in LiDAR Semantic Segmentation by Aligning Class
Distributions [9.581605678437032]
This work addresses the problem of unsupervised domain adaptation for LiDAR semantic segmentation models.
Our approach combines novel ideas on top of the current state-of-the-art approaches and yields new state-of-the-art results.
arXiv Detail & Related papers (2020-10-23T08:52:15Z) - Learning Invariant Representations and Risks for Semi-supervised Domain
Adaptation [109.73983088432364]
We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA)
We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
arXiv Detail & Related papers (2020-10-09T15:42:35Z) - Alleviating Semantic-level Shift: A Semi-supervised Domain Adaptation
Method for Semantic Segmentation [97.8552697905657]
A key challenge of this task is how to alleviate the data distribution discrepancy between the source and target domains.
We propose Alleviating Semantic-level Shift (ASS), which can successfully promote the distribution consistency from both global and local views.
We apply our ASS to two domain adaptation tasks, from GTA5 to Cityscapes and from Synthia to Cityscapes.
arXiv Detail & Related papers (2020-04-02T03:25:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.