Domain Generalization -- A Causal Perspective

• URL: http://arxiv.org/abs/2209.15177v1
• Date: Fri, 30 Sep 2022 01:56:49 GMT
• Title: Domain Generalization -- A Causal Perspective
• Authors: Paras Sheth, Raha Moraffah, K. Sel\c{c}uk Candan, Adrienne Raglin, Huan Liu
• Abstract summary: Machine learning models have gained widespread success, from healthcare to personalized recommendations. One of the preliminary assumptions of these models is the independent and identical distribution. Since the models rely heavily on this assumption, they exhibit poor generalization capabilities.
• Score: 20.630396283221838
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning models have gained widespread success, from healthcare to personalized recommendations. One of the preliminary assumptions of these models is the independent and identical distribution. Therefore, the train and test data are sampled from the same observation per this assumption. However, this assumption seldom holds in the real world due to distribution shifts. Since the models rely heavily on this assumption, they exhibit poor generalization capabilities. Over the recent years, dedicated efforts have been made to improve the generalization capabilities of these models. The primary idea behind these methods is to identify stable features or mechanisms that remain invariant across the different distributions. Many generalization approaches employ causal theories to describe invariance since causality and invariance are inextricably intertwined. However, current surveys deal with the causality-aware domain generalization methods on a very high-level. Furthermore, none of the existing surveys categorize the causal domain generalization methods based on the problem and causal theories these methods leverage. To this end, we present a comprehensive survey on causal domain generalization models from the aspects of the problem and causal theories. Furthermore, this survey includes in-depth insights into publicly accessible datasets and benchmarks for domain generalization in various domains. Finally, we conclude the survey with insights and discussions on future research directions. Finally, we conclude the survey with insights and discussions on future research directions.

Related papers

• Generalizing to any diverse distribution: uniformity, gentle finetuning and rebalancing [55.791818510796645]
  We aim to develop models that generalize well to any diverse test distribution, even if the latter deviates significantly from the training data. Various approaches like domain adaptation, domain generalization, and robust optimization attempt to address the out-of-distribution challenge. We adopt a more conservative perspective by accounting for the worst-case error across all sufficiently diverse test distributions within a known domain.
  arXiv  Detail & Related papers  (2024-10-08T12:26:48Z)
• On the Generalization Properties of Diffusion Models [33.93850788633184]
  This work embarks on a comprehensive theoretical exploration of the generalization attributes of diffusion models. We establish theoretical estimates of the generalization gap that evolves in tandem with the training dynamics of score-based diffusion models. We extend our quantitative analysis to a data-dependent scenario, wherein target distributions are portrayed as a succession of densities.
  arXiv  Detail & Related papers  (2023-11-03T09:20:20Z)
• Assaying Out-Of-Distribution Generalization in Transfer Learning [103.57862972967273]
  We take a unified view of previous work, highlighting message discrepancies that we address empirically. We fine-tune over 31k networks, from nine different architectures in the many- and few-shot setting.
  arXiv  Detail & Related papers  (2022-07-19T12:52:33Z)
• Causality Inspired Representation Learning for Domain Generalization [47.574964496891404]
  We introduce a general structural causal model to formalize the Domain generalization problem. Our goal is to extract the causal factors from inputs and then reconstruct the invariant causal mechanisms. We highlight that ideal causal factors should meet three basic properties: separated from the non-causal ones, jointly independent, and causally sufficient for the classification.
  arXiv  Detail & Related papers  (2022-03-27T08:08:33Z)
• Causal Forecasting:Generalization Bounds for Autoregressive Models [19.407531303870087]
  We introduce the framework of *causal learning theory* for forecasting. We obtain a characterization of the difference between statistical and causal risks. This is the first work that provides theoretical guarantees for causal generalization in the time-series setting.
  arXiv  Detail & Related papers  (2021-11-18T17:56:20Z)
• A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation [0.0]
  Evidential Deep Learning: For unfamiliar data, they admit "what they don't know" and fall back onto a prior belief. This survey aims to familiarize the reader with an alternative class of models based on the concept of Evidential Deep Learning: For unfamiliar data, they admit "what they don't know" and fall back onto a prior belief.
  arXiv  Detail & Related papers  (2021-10-06T20:13:57Z)
• Towards Out-Of-Distribution Generalization: A Survey [46.329995334444156]
  Out-of-Distribution generalization is an emerging topic of machine learning research. This paper represents the first comprehensive, systematic review of OOD generalization.
  arXiv  Detail & Related papers  (2021-08-31T05:28:42Z)
• Self-balanced Learning For Domain Generalization [64.99791119112503]
  Domain generalization aims to learn a prediction model on multi-domain source data such that the model can generalize to a target domain with unknown statistics. Most existing approaches have been developed under the assumption that the source data is well-balanced in terms of both domain and class. We propose a self-balanced domain generalization framework that adaptively learns the weights of losses to alleviate the bias caused by different distributions of the multi-domain source data.
  arXiv  Detail & Related papers  (2021-08-31T03:17:54Z)
• In Search of Robust Measures of Generalization [79.75709926309703]
  We develop bounds on generalization error, optimization error, and excess risk. When evaluated empirically, most of these bounds are numerically vacuous. We argue that generalization measures should instead be evaluated within the framework of distributional robustness.
  arXiv  Detail & Related papers  (2020-10-22T17:54:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.