Environment-Robust Representation Learning with Empirical Bayes

• URL: http://arxiv.org/abs/2606.05365v1
• Date: Wed, 03 Jun 2026 19:12:46 GMT
• Title: Environment-Robust Representation Learning with Empirical Bayes
• Abstract summary: We consider multi-environment prediction problems.<n>Given a dataset from multiple environments, we formulate a Bayesian model for such problems.<n>We show that this objective decomposes into per-environment terms and an additional cross-environment balancing term induced by the model's structure.<n>We develop an amortized variational algorithm for posterior approximation, and use the resulting learned latent variables to form predictions in new environments.
• Score: 18.76526979302572
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider multi-environment prediction problems. We assume the environments change the distribution of a latent variable, while the mechanisms generating observed covariates and targets remain stable conditional on that variable. For example, hospitals or clinical cohorts may differ in the prevalence of latent patient states, even though the relationships between those states, physiological measurements, and outcomes remain unchanged. Given a dataset from multiple environments, we formulate a Bayesian model for such problems and derive the corresponding variational objective. We show that this objective decomposes into per-environment terms and an additional cross-environment balancing term induced by the model's structure. We use an empirical Bayes method to set the prior and incorporate it into the objective. Based on this objective, we develop an amortized variational algorithm for posterior approximation, and use the resulting learned latent variables to form predictions in new environments.We study our approach through simulations and real-world studies of astronomical source identification, microbiome-based disease detection, and ICU sepsis prediction. Across these settings, our method outperforms previous approaches for prediction in new environments.

Related papers

• Investigating the Impact of Histopathological Foundation Models on Regressive Prediction of Homologous Recombination Deficiency [52.50039435394964]
  We systematically evaluate foundation models for regression-based tasks.<n>We extract patch-level features from whole slide images (WSI) using five state-of-the-art foundation models.<n>Models are trained to predict continuous HRD scores based on these extracted features across breast, endometrial, and lung cancer cohorts.
  arXiv  Detail & Related papers  (2026-01-29T14:06:50Z)
• Identifying biological perturbation targets through causal differential networks [23.568795598997376]
  We propose a causality-inspired approach to identify variables responsible for changes to a biological system.<n>First, we infer noisy causal graphs from the observational and interventional data.<n>We then learn to map the differences between these graphs, along with additional statistical features, to sets of variables that were intervened upon.
  arXiv  Detail & Related papers  (2024-10-04T12:48:21Z)
• Learning When the Concept Shifts: Confounding, Invariance, and Dimension Reduction [5.38274042816001]
  In observational data, the distribution shift is often driven by unobserved confounding factors. This motivates us to study the domain adaptation problem with observational data. We show a model that uses the learned lower-dimensional subspace can incur nearly ideal gap between target and source risk.
  arXiv  Detail & Related papers  (2024-06-22T17:43:08Z)
• Meta-Learners for Partially-Identified Treatment Effects Across Multiple Environments [67.80453452949303]
  Estimating the conditional average treatment effect (CATE) from observational data is relevant for many applications such as personalized medicine. Here, we focus on the widespread setting where the observational data come from multiple environments. We propose different model-agnostic learners (so-called meta-learners) to estimate the bounds that can be used in combination with arbitrary machine learning models.
  arXiv  Detail & Related papers  (2024-06-04T16:31:43Z)
• Sparsity regularization via tree-structured environments for disentangled representations [4.824771782127179]
  Causal representation learning could advance scientific understanding by enabling inference of latent variables such as pathway activation. We develop methods for inferring latent variables from multiple related datasets (environments) and tasks. We find that Tree-Based Regularization (TBR) minimizes both prediction error and regularizes closely related environments to learn similar predictors.
  arXiv  Detail & Related papers  (2024-05-30T21:08:14Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• Prediction under Latent Subgroup Shifts with High-Dimensional Observations [30.433078066683848]
  We introduce a new approach to prediction in graphical models with latent-shift adaptation. Our novel form of RPM identifies causal latent structure in the source environment, and adapts properly to predict in the target.
  arXiv  Detail & Related papers  (2023-06-23T12:26:24Z)
• Environment Invariant Linear Least Squares [16.831875263656027]
  This paper considers a multi-environment linear regression model in which data from multiple experimental settings are collected.<n>We construct a novel environment invariant linear least squares (EILLS) objective function, a multi-environment version of linear least-squares regression.
  arXiv  Detail & Related papers  (2023-03-06T13:10:54Z)
• Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations [84.42837346400151]
  Estimating counterfactual outcomes over time has the potential to unlock personalized healthcare. Existing causal inference approaches consider regular, discrete-time intervals between observations and treatment decisions. We propose a controllable simulation environment based on a model of tumor growth for a range of scenarios.
  arXiv  Detail & Related papers  (2022-06-16T17:15:15Z)
• Leveraging Global Parameters for Flow-based Neural Posterior Estimation [90.21090932619695]
  Inferring the parameters of a model based on experimental observations is central to the scientific method. A particularly challenging setting is when the model is strongly indeterminate, i.e., when distinct sets of parameters yield identical observations. We present a method for cracking such indeterminacy by exploiting additional information conveyed by an auxiliary set of observations sharing global parameters.
  arXiv  Detail & Related papers  (2021-02-12T12:23:13Z)
• Decision-Making with Auto-Encoding Variational Bayes [71.44735417472043]
  We show that a posterior approximation distinct from the variational distribution should be used for making decisions. Motivated by these theoretical results, we propose learning several approximate proposals for the best model. In addition to toy examples, we present a full-fledged case study of single-cell RNA sequencing.
  arXiv  Detail & Related papers  (2020-02-17T19:23:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.