Environment-Adaptive Covariate Selection: Learning When to Use Spurious Correlations for Out-of-Distribution Prediction

• URL: http://arxiv.org/abs/2601.02322v1
• Date: Mon, 05 Jan 2026 18:13:02 GMT
• Title: Environment-Adaptive Covariate Selection: Learning When to Use Spurious Correlations for Out-of-Distribution Prediction
• Authors: Shuozhi Zuo, Yixin Wang,
• Abstract summary: We show that out-of-distribution prediction fails when only a subset of the true causes of the outcome is observed.<n>We propose an environment-adaptive covariate selection algorithm.<n>EACS consistently outperforms static causal, invariant, and ERM-based predictors under diverse distribution shifts.
• Score: 33.69413571438309
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Out-of-distribution (OOD) prediction is often approached by restricting models to causal or invariant covariates, avoiding non-causal spurious associations that may be unstable across environments. Despite its theoretical appeal, this strategy frequently underperforms empirical risk minimization (ERM) in practice. We investigate the source of this gap and show that such failures naturally arise when only a subset of the true causes of the outcome is observed. In these settings, non-causal spurious covariates can serve as informative proxies for unobserved causes and substantially improve prediction, except under distribution shifts that break these proxy relationships. Consequently, the optimal set of predictive covariates is neither universal nor necessarily exhibits invariant relationships with the outcome across all environments, but instead depends on the specific type of shift encountered. Crucially, we observe that different covariate shifts induce distinct, observable signatures in the covariate distribution itself. Moreover, these signatures can be extracted from unlabeled data in the target OOD environment and used to assess when proxy covariates remain reliable and when they fail. Building on this observation, we propose an environment-adaptive covariate selection (EACS) algorithm that maps environment-level covariate summaries to environment-specific covariate sets, while allowing the incorporation of prior causal knowledge as constraints. Across simulations and applied datasets, EACS consistently outperforms static causal, invariant, and ERM-based predictors under diverse distribution shifts.

Related papers

• Anti-causal domain generalization: Leveraging unlabeled data [9.670511796477852]
  Existing methods typically require labeled data from multiple training environments.<n>We study domain generalization in an anti-causal setting.<n>We propose two methods that penalize the model's sensitivity to variations in the mean and co variance.
  arXiv  Detail & Related papers  (2026-02-19T09:13:42Z)
• Variance-Gated Ensembles: An Epistemic-Aware Framework for Uncertainty Estimation [0.6340400318304492]
  Variance-Gated Ensembles (VGE) is an intuitive framework that injects epistemic sensitivity via a signal-to-noise gate computed from ensemble statistics.<n>We derive closed-form vector-Jacobian products enabling end-to-end training through ensemble sample mean and variance.
  arXiv  Detail & Related papers  (2026-02-08T22:05:23Z)
• CURVE: Learning Causality-Inspired Invariant Representations for Robust Scene Understanding via Uncertainty-Guided Regularization [30.613712415224473]
  CURVE is a framework that integrates variational uncertainty modeling with uncertainty-guided structural regularization to suppress high-variance relations.<n>Specifically, we apply prototype-conditioned debiasing to disentangle invariant interaction dynamics from environment-dependent variations, promoting a sparse and domain-stable topology.<n> Empirically, we evaluate CURVE in zero-shot transfer and low-data sim-to-real adaptation, verifying its ability to learn domain-stable sparse topologies and provide reliable uncertainty estimates to support risk prediction under distribution shifts.
  arXiv  Detail & Related papers  (2026-01-28T08:15:56Z)
• Diagnosing and Rectifying Fake OOD Invariance: A Restructured Causal Approach [51.012396632595554]
  Invariant representation learning (IRL) encourages the prediction from invariant causal features to labels de-confounded from the environments. Recent theoretical results verified that some causal features recovered by IRLs merely pretend domain-invariantly in the training environments but fail in unseen domains. We develop an approach based on conditional mutual information with respect to RS-SCM, then rigorously rectify the spurious and fake invariant effects.
  arXiv  Detail & Related papers  (2023-12-15T12:58:05Z)
• Model-based causal feature selection for general response types [8.228587135343071]
  Invariant causal prediction (ICP) is a method for causal feature selection which requires data from heterogeneous settings. We develop transformation-model (TRAM) based ICP, allowing for continuous, categorical, count-type, and uninformatively censored responses. We provide an open-source R package 'tramicp' and evaluate our approach on simulated data and in a case study investigating causal features of survival in critically ill patients.
  arXiv  Detail & Related papers  (2023-09-22T12:42:48Z)
• Adapting to Latent Subgroup Shifts via Concepts and Proxies [82.01141290360562]
  We show that the optimal target predictor can be non-parametrically identified with the help of concept and proxy variables available only in the source domain. For continuous observations, we propose a latent variable model specific to the data generation process at hand.
  arXiv  Detail & Related papers  (2022-12-21T18:30:22Z)
• Breaking the Spurious Causality of Conditional Generation via Fairness Intervention with Corrective Sampling [77.15766509677348]
  Conditional generative models often inherit spurious correlations from the training dataset. This can result in label-conditional distributions that are imbalanced with respect to another latent attribute. We propose a general two-step strategy to mitigate this issue.
  arXiv  Detail & Related papers  (2022-12-05T08:09:33Z)
• Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shift [50.98086766507025]
  We propose a simple-yet-effective data augmentation strategy, Adversarial Invariant Augmentation (AIA) AIA aims to extrapolate and generate new environments, while concurrently preserving the original stable features during the augmentation process.
  arXiv  Detail & Related papers  (2022-11-05T07:55:55Z)
• Balancing Fairness and Robustness via Partial Invariance [17.291131923335918]
  Invariant Risk Minimization (IRM) aims to learn invariant features from a set of environments for solving the out-of-distribution (OOD) problem. We argue for a partial invariance framework to mitigate the failure case. Our results show the capability of the partial invariant risk minimization to alleviate the trade-off between fairness and risk in certain settings.
  arXiv  Detail & Related papers  (2021-12-17T06:41:47Z)
• Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap [140.98628848491146]
  We introduce deconfounding scores, which induce better overlap without biasing the target of estimation. We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data. In particular, we show that this technique could be an attractive alternative to standard regularizations.
  arXiv  Detail & Related papers  (2021-04-12T18:50:11Z)
• The Risks of Invariant Risk Minimization [52.7137956951533]
  Invariant Risk Minimization is an objective based on the idea for learning deep, invariant features of data. We present the first analysis of classification under the IRM objective--as well as these recently proposed alternatives--under a fairly natural and general model. We show that IRM can fail catastrophically unless the test data are sufficiently similar to the training distribution--this is precisely the issue that it was intended to solve.
  arXiv  Detail & Related papers  (2020-10-12T14:54:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.