Improving Generalization with Domain Convex Game

• URL: http://arxiv.org/abs/2303.13297v1
• Date: Thu, 23 Mar 2023 14:27:49 GMT
• Title: Improving Generalization with Domain Convex Game
• Authors: Fangrui Lv, Jian Liang, Shuang Li, Jinming Zhang, Di Liu
• Abstract summary: Domain generalization tends to alleviate the poor generalization capability of deep neural networks by learning model with multiple source domains. A classical solution to DG is domain augmentation, the common belief of which is that diversifying source domains will be conducive to the out-of-distribution generalization. Our explorations reveal that the correlation between model generalization and the diversity of domains may be not strictly positive, which limits the effectiveness of domain augmentation.
• Score: 32.07275105040802
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Domain generalization (DG) tends to alleviate the poor generalization capability of deep neural networks by learning model with multiple source domains. A classical solution to DG is domain augmentation, the common belief of which is that diversifying source domains will be conducive to the out-of-distribution generalization. However, these claims are understood intuitively, rather than mathematically. Our explorations empirically reveal that the correlation between model generalization and the diversity of domains may be not strictly positive, which limits the effectiveness of domain augmentation. This work therefore aim to guarantee and further enhance the validity of this strand. To this end, we propose a new perspective on DG that recasts it as a convex game between domains. We first encourage each diversified domain to enhance model generalization by elaborately designing a regularization term based on supermodularity. Meanwhile, a sample filter is constructed to eliminate low-quality samples, thereby avoiding the impact of potentially harmful information. Our framework presents a new avenue for the formal analysis of DG, heuristic analysis and extensive experiments demonstrate the rationality and effectiveness.

Related papers

• Domain Generalization via Causal Adjustment for Cross-Domain Sentiment Analysis [59.73582306457387]
  We focus on the problem of domain generalization for cross-domain sentiment analysis. We propose a backdoor adjustment-based causal model to disentangle the domain-specific and domain-invariant representations. A series of experiments show the great performance and robustness of our model.
  arXiv  Detail & Related papers  (2024-02-22T13:26:56Z)
• When Neural Networks Fail to Generalize? A Model Sensitivity Perspective [82.36758565781153]
  Domain generalization (DG) aims to train a model to perform well in unseen domains under different distributions. This paper considers a more realistic yet more challenging scenario, namely Single Domain Generalization (Single-DG) We empirically ascertain a property of a model that correlates strongly with its generalization that we coin as "model sensitivity" We propose a novel strategy of Spectral Adversarial Data Augmentation (SADA) to generate augmented images targeted at the highly sensitive frequencies.
  arXiv  Detail & Related papers  (2022-12-01T20:15:15Z)
• Domain Generalization through the Lens of Angular Invariance [44.76809026901016]
  Domain generalization (DG) aims at generalizing a classifier trained on multiple source domains to an unseen target domain with domain shift. We propose a novel deep DG method called Angular Invariance Domain Generalization Network (AIDGN)
  arXiv  Detail & Related papers  (2022-10-28T02:05:38Z)
• Constrained Maximum Cross-Domain Likelihood for Domain Generalization [14.91361835243516]
  Domain generalization aims to learn a generalizable model on multiple source domains, which is expected to perform well on unseen test domains. In this paper, we propose a novel domain generalization method, which minimizes the KL-divergence between posterior distributions from different domains. Experiments on four standard benchmark datasets, i.e., Digits-DG, PACS, Office-Home and miniDomainNet, highlight the superior performance of our method.
  arXiv  Detail & Related papers  (2022-10-09T03:41:02Z)
• Adaptive Domain Generalization via Online Disagreement Minimization [17.215683606365445]
  Domain Generalization aims to safely transfer a model to unseen target domains. AdaODM adaptively modifies the source model at test time for different target domains. Results show AdaODM stably improves the generalization capacity on unseen domains.
  arXiv  Detail & Related papers  (2022-08-03T11:51:11Z)
• Generalizing to Unseen Domains with Wasserstein Distributional Robustness under Limited Source Knowledge [22.285156929279207]
  Domain generalization aims at learning a universal model that performs well on unseen target domains. We propose a novel domain generalization framework called Wasserstein Distributionally Robust Domain Generalization (WDRDG)
  arXiv  Detail & Related papers  (2022-07-11T14:46:50Z)
• Improving Diversity with Adversarially Learned Transformations for Domain Generalization [81.26960899663601]
  We present a novel framework that uses adversarially learned transformations (ALT) using a neural network to model plausible, yet hard image transformations. We show that ALT can naturally work with existing diversity modules to produce highly distinct, and large transformations of the source domain leading to state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-06-15T18:05:24Z)
• Compound Domain Generalization via Meta-Knowledge Encoding [55.22920476224671]
  We introduce Style-induced Domain-specific Normalization (SDNorm) to re-normalize the multi-modal underlying distributions. We harness the prototype representations, the centroids of classes, to perform relational modeling in the embedding space. Experiments on four standard Domain Generalization benchmarks reveal that COMEN exceeds the state-of-the-art performance without the need of domain supervision.
  arXiv  Detail & Related papers  (2022-03-24T11:54:59Z)
• Towards Principled Disentanglement for Domain Generalization [90.9891372499545]
  A fundamental challenge for machine learning models is generalizing to out-of-distribution (OOD) data. We first formalize the OOD generalization problem as constrained optimization, called Disentanglement-constrained Domain Generalization (DDG) Based on the transformation, we propose a primal-dual algorithm for joint representation disentanglement and domain generalization.
  arXiv  Detail & Related papers  (2021-11-27T07:36:32Z)
• Domain Generalization Needs Stochastic Weight Averaging for Robustness on Domain Shifts [19.55308715031151]
  Domain generalization aims to learn a generalizable model to unseen target domains from multiple source domains. Recent benchmarks show that most approaches do not provide significant improvements compared to the simple empirical risk minimization. In this paper, we analyze how ERM works in views of domain-invariant feature learning and domain-specific normalization.
  arXiv  Detail & Related papers  (2021-02-17T06:42:09Z)
• Learning to Generate Novel Domains for Domain Generalization [115.21519842245752]
  This paper focuses on the task of learning from multiple source domains a model that generalizes well to unseen domains. We employ a data generator to synthesize data from pseudo-novel domains to augment the source domains. Our method, L2A-OT, outperforms current state-of-the-art DG methods on four benchmark datasets.
  arXiv  Detail & Related papers  (2020-07-07T09:34:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.