Frequentist Consistency of Prior-Data Fitted Networks for Causal Inference

• URL: http://arxiv.org/abs/2603.12037v1
• Date: Thu, 12 Mar 2026 15:14:35 GMT
• Title: Frequentist Consistency of Prior-Data Fitted Networks for Causal Inference
• Abstract summary: Foundation models based on prior-data fitted networks (PFNs) have shown strong empirical performance in causal inference.<n>It is unclear whether PFN-based causal estimators provide uncertainty quantification consistent with classical frequentist estimators.
• Score: 49.70323515651261
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Foundation models based on prior-data fitted networks (PFNs) have shown strong empirical performance in causal inference by framing the task as an in-context learning problem.However, it is unclear whether PFN-based causal estimators provide uncertainty quantification that is consistent with classical frequentist estimators. In this work, we address this gap by analyzing the frequentist consistency of PFN-based estimators for the average treatment effect (ATE). (1) We show that existing PFNs, when interpreted as Bayesian ATE estimators, can exhibit prior-induced confounding bias: the prior is not asymptotically overwritten by data, which, in turn, prevents frequentist consistency. (2) As a remedy, we suggest employing a calibration procedure based on a one-step posterior correction (OSPC). We show that the OSPC helps to restore frequentist consistency and can yield a semi-parametric Bernstein-von Mises theorem for calibrated PFNs (i.e., both the calibrated PFN-based estimators and the classical semi-parametric efficient estimators converge in distribution with growing data size). (3) Finally, we implement OSPC through tailoring martingale posteriors on top of the PFNs. In this way, we are able to recover functional nuisance posteriors from PFNs, required by the OSPC. In multiple (semi-)synthetic experiments, PFNs calibrated with our martingale posterior OSPC produce ATE uncertainty that (i) asymptotically matches frequentist uncertainty and (ii) is well calibrated in finite samples in comparison to other Bayesian ATE estimators.

Related papers

• Generalized Bayes for Causal Inference [47.66564705927885]
  Uncertainty quantification is central to many applications of causal machine learning.<n>Standard Bayesian approaches typically require specifying a probabilistic model for the data-generating process.<n>We propose a generalized Bayesian framework for causal inference.
  arXiv  Detail & Related papers  (2026-03-03T14:27:23Z)
• Stabilizing Diffusion Posterior Sampling by Noise--Frequency Continuation [52.736416985173776]
  At high noise, data-consistency gradients computed from inaccurate estimates can be geometrically incongruent with the posterior geometry.<n>We propose a noise--frequency Continuation framework that constructs a continuous family of intermediate posteriors whose likelihood enforces measurement consistency only within a noise-dependent frequency band.<n>Our method achieves state-of-the-art performance and improves motion deblurring PSNR by up to 5 dB over strong baselines.
  arXiv  Detail & Related papers  (2026-01-30T03:14:01Z)
• Bayesian Semiparametric Causal Inference: Targeted Doubly Robust Estimation of Treatment Effects [1.2833734915643464]
  We propose a semiparametric Bayesian methodology for estimating the average treatment effect (ATE) within the potential outcomes framework.<n>Our method introduces a Bayesian debiasing procedure that corrects for bias arising from nuisance estimation.<n>Extensive simulations confirm the theoretical results, demonstrating accurate point estimation and credible intervals with nominal coverage.
  arXiv  Detail & Related papers  (2025-11-19T22:15:04Z)
• Uncertainty Quantification for Prior-Data Fitted Networks using Martingale Posteriors [20.086105345966455]
  We propose a principled and efficient sampling procedure to construct Bayesian posteriors for estimates based on Martingale posteriors.<n>Several simulated and real-world data examples showcase the uncertainty quantification of our method in inference applications.
  arXiv  Detail & Related papers  (2025-05-16T14:47:43Z)
• In-Context Parametric Inference: Point or Distribution Estimators? [66.22308335324239]
  We show that amortized point estimators generally outperform posterior inference, though the latter remain competitive in some low-dimensional problems.<n>Our experiments indicate that amortized point estimators generally outperform posterior inference, though the latter remain competitive in some low-dimensional problems.
  arXiv  Detail & Related papers  (2025-02-17T10:00:24Z)
• Calibrating Neural Simulation-Based Inference with Differentiable Coverage Probability [50.44439018155837]
  We propose to include a calibration term directly into the training objective of the neural model. By introducing a relaxation of the classical formulation of calibration error we enable end-to-end backpropagation. It is directly applicable to existing computational pipelines allowing reliable black-box posterior inference.
  arXiv  Detail & Related papers  (2023-10-20T10:20:45Z)
• Double Robust Bayesian Inference on Average Treatment Effects [2.458652618559425]
  We propose a double robust Bayesian inference procedure on the average treatment effect (ATE) under unconfoundedness.<n>For our new Bayesian approach, we first adjust the prior distributions of the conditional mean functions, and then correct the posterior distribution of the resulting ATE.
  arXiv  Detail & Related papers  (2022-11-29T15:32:25Z)
• Sample-Efficient Optimisation with Probabilistic Transformer Surrogates [66.98962321504085]
  This paper investigates the feasibility of employing state-of-the-art probabilistic transformers in Bayesian optimisation. We observe two drawbacks stemming from their training procedure and loss definition, hindering their direct deployment as proxies in black-box optimisation. We introduce two components: 1) a BO-tailored training prior supporting non-uniformly distributed points, and 2) a novel approximate posterior regulariser trading-off accuracy and input sensitivity to filter favourable stationary points for improved predictive performance.
  arXiv  Detail & Related papers  (2022-05-27T11:13:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.