Domain Generalization: A Tale of Two ERMs

• URL: http://arxiv.org/abs/2510.04441v1
• Date: Mon, 06 Oct 2025 02:17:12 GMT
• Title: Domain Generalization: A Tale of Two ERMs
• Authors: Yilun Zhu, Naihao Deng, Naichen Shi, Aditya Gangrade, Clayton Scott,
• Abstract summary: We show that domain-informed ERM'', wherein feature vectors are augmented with domain-specific information, outperforms pooling ERM.<n>These claims are supported by a theoretical framework and experiments on language and vision tasks.
• Score: 22.836459792619014
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Domain generalization (DG) is the problem of generalizing from several distributions (or domains), for which labeled training data are available, to a new test domain for which no labeled data is available. A common finding in the DG literature is that it is difficult to outperform empirical risk minimization (ERM) on the pooled training data. In this work, we argue that this finding has primarily been reported for datasets satisfying a \emph{covariate shift} assumption. When the dataset satisfies a \emph{posterior drift} assumption instead, we show that ``domain-informed ERM,'' wherein feature vectors are augmented with domain-specific information, outperforms pooling ERM. These claims are supported by a theoretical framework and experiments on language and vision tasks.

Related papers

• GenGMM: Generalized Gaussian-Mixture-based Domain Adaptation Model for Semantic Segmentation [0.9626666671366837]
  We introduce the Generalized Gaussian-mixture-based (GenGMM) domain adaptation model, which harnesses the underlying data distribution in both domains. Experiments demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-10-21T20:21:09Z)
• Cross Domain Generative Augmentation: Domain Generalization with Latent Diffusion Models [11.309433257851122]
  Cross Domain Generative Augmentation (CDGA) generates synthetic images to fill the gap between all domains. We show that CDGA outperforms SOTA DG methods under the Domainbed benchmark.
  arXiv  Detail & Related papers  (2023-12-08T21:52:00Z)
• SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation [62.889835139583965]
  We introduce an unsupervised auxiliary task of learning an implicit underlying surface representation simultaneously on source and target data. As both domains share the same latent representation, the model is forced to accommodate discrepancies between the two sources of data. Our experiments demonstrate that our method achieves a better performance than the current state of the art, both in real-to-real and synthetic-to-real scenarios.
  arXiv  Detail & Related papers  (2023-04-06T17:36:23Z)
• ERM++: An Improved Baseline for Domain Generalization [69.80606575323691]
  Empirical Risk Minimization (ERM) can outperform most more complex Domain Generalization (DG) methods when properly tuned.<n>ERM++ improves DG performance by over 5% compared to prior ERM baselines.
  arXiv  Detail & Related papers  (2023-04-04T17:31:15Z)
• Probable Domain Generalization via Quantile Risk Minimization [90.15831047587302]
  Domain generalization seeks predictors which perform well on unseen test distributions. We propose a new probabilistic framework for DG where the goal is to learn predictors that perform well with high probability.
  arXiv  Detail & Related papers  (2022-07-20T14:41:09Z)
• Improving Multi-Domain Generalization through Domain Re-labeling [31.636953426159224]
  We study the important link between pre-specified domain labels and the generalization performance. We introduce a general approach for multi-domain generalization, MulDEns, that uses an ERM-based deep ensembling backbone. We show that MulDEns does not require tailoring the augmentation strategy or the training process specific to a dataset.
  arXiv  Detail & Related papers  (2021-12-17T23:21:50Z)
• META: Mimicking Embedding via oThers' Aggregation for Generalizable Person Re-identification [68.39849081353704]
  Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time. This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
  arXiv  Detail & Related papers  (2021-12-16T08:06:50Z)
• Domain-Specific Bias Filtering for Single Labeled Domain Generalization [19.679447374738498]
  Domain generalization utilizes multiple labeled source datasets to train a generalizable model for unseen target domains. Due to expensive annotation costs, the requirements of labeling all the source data are hard to be met in real-world applications. We propose a novel method called Domain-Specific Bias Filtering (DSBF), which filters out its domain-specific bias with the unlabeled source data.
  arXiv  Detail & Related papers  (2021-10-02T05:08:01Z)
• Inferring Latent Domains for Unsupervised Deep Domain Adaptation [54.963823285456925]
  Unsupervised Domain Adaptation (UDA) refers to the problem of learning a model in a target domain where labeled data are not available. This paper introduces a novel deep architecture which addresses the problem of UDA by automatically discovering latent domains in visual datasets. We evaluate our approach on publicly available benchmarks, showing that it outperforms state-of-the-art domain adaptation methods.
  arXiv  Detail & Related papers  (2021-03-25T14:33:33Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.