SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of
Invariances in Domain Generalization
- URL: http://arxiv.org/abs/2106.02266v1
- Date: Fri, 4 Jun 2021 05:20:54 GMT
- Title: SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of
Invariances in Domain Generalization
- Authors: Soroosh Shahtalebi, Jean-Christophe Gagnon-Audet, Touraj Laleh,
Mojtaba Faramarzi, Kartik Ahuja, Irina Rish
- Abstract summary: We propose a masking strategy, which determines a continuous weight based on the agreement of gradients that flow in each edge of network.
SAND-mask is validated over the Domainbed benchmark for domain generalization.
- Score: 7.253255826783766
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: A major bottleneck in the real-world applications of machine learning models
is their failure in generalizing to unseen domains whose data distribution is
not i.i.d to the training domains. This failure often stems from learning
non-generalizable features in the training domains that are spuriously
correlated with the label of data. To address this shortcoming, there has been
a growing surge of interest in learning good explanations that are hard to
vary, which is studied under the notion of Out-of-Distribution (OOD)
Generalization. The search for good explanations that are \textit{invariant}
across different domains can be seen as finding local (global) minimas in the
loss landscape that hold true across all of the training domains. In this
paper, we propose a masking strategy, which determines a continuous weight
based on the agreement of gradients that flow in each edge of network, in order
to control the amount of update received by the edge in each step of
optimization. Particularly, our proposed technique referred to as "Smoothed-AND
(SAND)-masking", not only validates the agreement in the direction of gradients
but also promotes the agreement among their magnitudes to further ensure the
discovery of invariances across training domains. SAND-mask is validated over
the Domainbed benchmark for domain generalization and significantly improves
the state-of-the-art accuracy on the Colored MNIST dataset while providing
competitive results on other domain generalization datasets.
Related papers
- Generalize or Detect? Towards Robust Semantic Segmentation Under Multiple Distribution Shifts [56.57141696245328]
In open-world scenarios, where both novel classes and domains may exist, an ideal segmentation model should detect anomaly classes for safety.
Existing methods often struggle to distinguish between domain-level and semantic-level distribution shifts.
arXiv Detail & Related papers (2024-11-06T11:03:02Z) - Disentangling Masked Autoencoders for Unsupervised Domain Generalization [57.56744870106124]
Unsupervised domain generalization is fast gaining attention but is still far from well-studied.
Disentangled Masked Auto (DisMAE) aims to discover the disentangled representations that faithfully reveal intrinsic features.
DisMAE co-trains the asymmetric dual-branch architecture with semantic and lightweight variation encoders.
arXiv Detail & Related papers (2024-07-10T11:11:36Z) - META: Mimicking Embedding via oThers' Aggregation for Generalizable
Person Re-identification [68.39849081353704]
Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time.
This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
arXiv Detail & Related papers (2021-12-16T08:06:50Z) - Fishr: Invariant Gradient Variances for Out-of-distribution
Generalization [98.40583494166314]
Fishr is a learning scheme to enforce domain invariance in the space of the gradients of the loss function.
Fishr exhibits close relations with the Fisher Information and the Hessian of the loss.
In particular, Fishr improves the state of the art on the DomainBed benchmark and performs significantly better than Empirical Risk Minimization.
arXiv Detail & Related papers (2021-09-07T08:36:09Z) - Discriminative Domain-Invariant Adversarial Network for Deep Domain
Generalization [33.84004077585957]
We propose a discriminative domain-invariant adversarial network (DDIAN) for domain generalization.
DDIAN achieves better prediction on unseen target data during training compared to state-of-the-art domain generalization approaches.
arXiv Detail & Related papers (2021-08-20T04:24:12Z) - Generalizable Representation Learning for Mixture Domain Face
Anti-Spoofing [53.82826073959756]
Face anti-spoofing approach based on domain generalization(DG) has drawn growing attention due to its robustness forunseen scenarios.
We propose domain dy-namic adjustment meta-learning (D2AM) without using do-main labels.
To overcome the limitation, we propose domain dy-namic adjustment meta-learning (D2AM) without using do-main labels.
arXiv Detail & Related papers (2021-05-06T06:04:59Z) - Robust Domain-Free Domain Generalization with Class-aware Alignment [4.442096198968069]
Domain-Free Domain Generalization (DFDG) is a model-agnostic method to achieve better generalization performance on the unseen test domain.
DFDG uses novel strategies to learn domain-invariant class-discriminative features.
It obtains competitive performance on both time series sensor and image classification public datasets.
arXiv Detail & Related papers (2021-02-17T17:46:06Z) - Domain Adaptation in LiDAR Semantic Segmentation by Aligning Class
Distributions [9.581605678437032]
This work addresses the problem of unsupervised domain adaptation for LiDAR semantic segmentation models.
Our approach combines novel ideas on top of the current state-of-the-art approaches and yields new state-of-the-art results.
arXiv Detail & Related papers (2020-10-23T08:52:15Z) - Learning Invariant Representations and Risks for Semi-supervised Domain
Adaptation [109.73983088432364]
We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA)
We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
arXiv Detail & Related papers (2020-10-09T15:42:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.