Bayesian Hierarchical Invariant Prediction

• URL: http://arxiv.org/abs/2505.11211v1
• Date: Fri, 16 May 2025 13:06:25 GMT
• Title: Bayesian Hierarchical Invariant Prediction
• Authors: Francisco Madaleno, Pernille Julie Viuff Sand, Francisco C. Pereira, Sergio Hernan Garrido Mejia,
• Abstract summary: We propose Bayesian Hierarchical Invariant Prediction (BHIP) reframing Invariant Causal Prediction (ICP) through the lens of Hierarchical Bayes.<n>We leverage the hierarchical structure to explicitly test invariance of causal mechanisms under heterogeneous data, resulting in improved computational scalability for a larger number of predictors compared to ICP.<n>In this paper, we test two sparsity inducing priors: horseshoe and spike-and-slab, both of which allow us a more reliable identification of causal features.
• Score: 4.182645056052712
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose Bayesian Hierarchical Invariant Prediction (BHIP) reframing Invariant Causal Prediction (ICP) through the lens of Hierarchical Bayes. We leverage the hierarchical structure to explicitly test invariance of causal mechanisms under heterogeneous data, resulting in improved computational scalability for a larger number of predictors compared to ICP. Moreover, given its Bayesian nature BHIP enables the use of prior information. In this paper, we test two sparsity inducing priors: horseshoe and spike-and-slab, both of which allow us a more reliable identification of causal features. We test BHIP in synthetic and real-world data showing its potential as an alternative inference method to ICP.

Related papers

• Bayesian Invariance Modeling of Multi-Environment Data [48.7892531847207]
  We develop a probabilistic model for invariant prediction.<n>In simulations and real data, we find that BIP and VI-BIP are more accurate and scalable than existing methods for invariant prediction.
  arXiv  Detail & Related papers  (2025-06-27T22:58:37Z)
• Exponential Convergence of CAVI for Bayesian PCA [0.7929564340244416]
  Probabilistic principal component analysis (PCA) and its Bayesian variant (BPCA) are widely used for dimension reduction in machine learning and statistics.<n>In our paper, we prove a precise exponential convergence result in the case where the model uses a single principal component (PC)<n>We also leverage recent tools to prove exponential convergence of CAVI for the model with any number of PCs.
  arXiv  Detail & Related papers  (2025-05-22T02:44:00Z)
• Divide-and-Conquer Predictive Coding: a structured Bayesian inference algorithm [11.722226132995978]
  We introduce a novel predictive coding algorithm for structured generative models, that we call divide-and-conquer predictive coding (D CPC) D CPC performs maximum-likelihood updates of model parameters without sacrificing biological plausibility. Empirically, DCPC achieves better numerical performance than competing algorithms and provides accurate inference in a number of problems not previously addressed with predictive coding.
  arXiv  Detail & Related papers  (2024-08-11T17:29:03Z)
• Adapting Conformal Prediction to Distribution Shifts Without Labels [16.478151550456804]
  Conformal prediction (CP) enables machine learning models to output prediction sets with guaranteed coverage rate. Our goal is to improve the quality of CP-generated prediction sets using only unlabeled data from the test domain. This is achieved by two new methods called ECP and EACP, that adjust the score function in CP according to the base model's uncertainty on the unlabeled test data.
  arXiv  Detail & Related papers  (2024-06-03T15:16:02Z)
• Bayes in Wonderland! Predictive supervised classification inference hits unpredictability [1.8814209805277506]
  We show the convergence of the sBpc and mBpc under de Finetti type of exchangeability. We also provide a parameter estimation of the generative model giving rise to the partition exchangeable sequence.
  arXiv  Detail & Related papers  (2021-12-03T12:34:52Z)
• Variance Minimization in the Wasserstein Space for Invariant Causal Prediction [72.13445677280792]
  In this work, we show that the approach taken in ICP may be reformulated as a series of nonparametric tests that scales linearly in the number of predictors. Each of these tests relies on the minimization of a novel loss function that is derived from tools in optimal transport theory. We prove under mild assumptions that our method is able to recover the set of identifiable direct causes, and we demonstrate in our experiments that it is competitive with other benchmark causal discovery algorithms.
  arXiv  Detail & Related papers  (2021-10-13T22:30:47Z)
• Which Invariance Should We Transfer? A Causal Minimax Learning Approach [18.71316951734806]
  We present a comprehensive minimax analysis from a causal perspective. We propose an efficient algorithm to search for the subset with minimal worst-case risk. The effectiveness and efficiency of our methods are demonstrated on synthetic data and the diagnosis of Alzheimer's disease.
  arXiv  Detail & Related papers  (2021-07-05T09:07:29Z)
• Variational Causal Networks: Approximate Bayesian Inference over Causal Structures [132.74509389517203]
  We introduce a parametric variational family modelled by an autoregressive distribution over the space of discrete DAGs. In experiments, we demonstrate that the proposed variational posterior is able to provide a good approximation of the true posterior.
  arXiv  Detail & Related papers  (2021-06-14T17:52:49Z)
• Robust Bayesian Inference for Discrete Outcomes with the Total Variation Distance [5.139874302398955]
  Models of discrete-valued outcomes are easily misspecified if the data exhibit zero-inflation, overdispersion or contamination. Here, we introduce a robust discrepancy-based Bayesian approach using the Total Variation Distance (TVD) We empirically demonstrate that our approach is robust and significantly improves predictive performance on a range of simulated and real world data.
  arXiv  Detail & Related papers  (2020-10-26T09:53:06Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Stable Prediction via Leveraging Seed Variable [73.9770220107874]
  Previous machine learning methods might exploit subtly spurious correlations in training data induced by non-causal variables for prediction. We propose a conditional independence test based algorithm to separate causal variables with a seed variable as priori, and adopt them for stable prediction. Our algorithm outperforms state-of-the-art methods for stable prediction.
  arXiv  Detail & Related papers  (2020-06-09T06:56:31Z)
• Balance-Subsampled Stable Prediction [55.13512328954456]
  We propose a novel balance-subsampled stable prediction (BSSP) algorithm based on the theory of fractional factorial design. A design-theoretic analysis shows that the proposed method can reduce the confounding effects among predictors induced by the distribution shift. Numerical experiments on both synthetic and real-world data sets demonstrate that our BSSP algorithm significantly outperforms the baseline methods for stable prediction across unknown test data.
  arXiv  Detail & Related papers  (2020-06-08T07:01:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.