Pre-trained Tabular Foundation Models as Versatile Summary Networks for Neural Posterior Estimation

• URL: http://arxiv.org/abs/2605.07765v1
• Date: Fri, 08 May 2026 14:07:10 GMT
• Title: Pre-trained Tabular Foundation Models as Versatile Summary Networks for Neural Posterior Estimation
• Abstract summary: We study TabPFN as a training-free, modular summary network for simulation-based Bayesian inference ( SBI)<n>We propose PFN-NPE: a general recipe that uses a pretrained TabPFN encoder as a fixed summary network for simulator outputs, then pairs the resulting summaries with a downstream inference head chosen for the problem.<n> diagnostic probes show that the TabPFN-derived summaries often preserve useful posterior location and marginal information.
• Score: 0.746464933382582
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we study TabPFN as a training-free, modular summary network for simulation-based Bayesian inference (SBI). Tabular foundation models such as TabPFN are pretrained on broad families of synthetic tabular data-generating processes and adapt at test time through in-context learning, making them natural candidates for SBI, where posterior estimation often depends on learning informative summaries of simulated observations. We propose PFN-NPE: a general recipe that uses a pretrained TabPFN encoder as a fixed summary network for simulator outputs, then pairs the resulting summaries with a downstream inference head chosen for the problem. With normalizing flows as the default inference head, PFN-NPE matches established posterior approximation methods and sometimes outperforms them. More importantly, diagnostic probes show that the TabPFN-derived summaries often preserve useful posterior location and marginal information. These analyses also reveal a limitation in that TabPFN-derived summaries may struggle to represent the joint posterior structure even when the marginals are well recovered. Still, our experiments show that TabPFN can serve as an effective summary network across a diverse set of SBI settings, with the inference network left modular and task-dependent.

Related papers

• Tokenised Flow Matching for Hierarchical Simulation Based Inference [0.4199844472131922]
  Cost of simulator evaluations is a key practical bottleneck for Simulation Based Inference.<n>In hierarchical settings with shared global parameters and exchangeable site-level parameters and observations, this structure can be exploited to improve simulation efficiency.<n>We propose Tokenised Flow Matching for Posterior Estimation (TFMPE), a tokenised flow matching approach that supports function-valued observations through likelihood factorisation.
  arXiv  Detail & Related papers  (2026-04-22T16:07:47Z)
• Minimum Distance Summaries for Robust Neural Posterior Estimation [7.4716500353679685]
  Simulation-based inference ( SBI) enables amortized Bayesian inference by first training a neural posterior estimator (NPE) on prior-simulator pairs.<n>We introduce minimum-distance summaries, a plug-in robust NPE method that adapts queried test-time summaries independently of the pretrained NPE.
  arXiv  Detail & Related papers  (2026-02-09T20:06:15Z)
• Causal Pre-training Under the Fairness Lens: An Empirical Study of TabPFN [3.059960033014892]
  We evaluate the Tabular Prior-data Fitted Network (TabPFN) and its fine-tuned variants.<n>Our results reveal that while TabPFN achieves stronger predictive accuracy compared to baselines, improvements in fairness are moderate and inconsistent.<n>These findings suggest that the causal pre-training in TabPFN is helpful but insufficient for algorithmic fairness.
  arXiv  Detail & Related papers  (2026-01-25T17:17:12Z)
• Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models [14.125143586947177]
  We show how TabPFN can be used as pre-trained autoregressive conditional density estimators for SBI.<n>NPE-PFN eliminates the need for inference network selection, training, and hyper parameter tuning.<n>It exhibits superior robustness to model misspecification and can be scaled to simulation budgets that exceed the context size limit of TabPFN.
  arXiv  Detail & Related papers  (2025-04-24T15:29:39Z)
• Prior-Fitted Networks Scale to Larger Datasets When Treated as Weak Learners [82.72552644267724]
  BoostPFN can outperform standard PFNs with the same size of training samples in large datasets.<n>High performance is maintained for up to 50x of the pre-training size of PFNs.
  arXiv  Detail & Related papers  (2025-03-03T07:31:40Z)
• TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks [90.00817095558094]
  Prior-data fitted networks (PFNs) make use of pretraining and in-context learning to achieve strong performance on new tasks in a single forward pass. We introduce TuneTables, a parameter-efficient fine-tuning strategy for PFNs that compresses large datasets into a smaller learned context. We show that TuneTables can be used as an interpretability tool and can even be used to mitigate biases by optimizing a fairness objective.
  arXiv  Detail & Related papers  (2024-02-17T00:02:23Z)
• Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks [31.82225213006849]
  Tabular classification has traditionally relied on supervised algorithms, which estimate the parameters of a prediction model using its training data. Recently, Prior-Data Fitted Networks (PFNs) such as TabPFN have successfully learned to classify tabular data in-context. While such models show great promise, their applicability to real-world data remains limited due to the computational scale needed.
  arXiv  Detail & Related papers  (2023-11-17T16:04:27Z)
• Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network [59.79008107609297]
  We propose in this paper to approximate the joint posterior over the structure of a Bayesian Network. We use a single GFlowNet whose sampling policy follows a two-phase process. Since the parameters are included in the posterior distribution, this leaves more flexibility for the local probability models.
  arXiv  Detail & Related papers  (2023-05-30T19:16:44Z)
• Bayesian Structure Learning with Generative Flow Networks [85.84396514570373]
  In Bayesian structure learning, we are interested in inferring a distribution over the directed acyclic graph (DAG) from data. Recently, a class of probabilistic models, called Generative Flow Networks (GFlowNets), have been introduced as a general framework for generative modeling. We show that our approach, called DAG-GFlowNet, provides an accurate approximation of the posterior over DAGs.
  arXiv  Detail & Related papers  (2022-02-28T15:53:10Z)
• Transformers Can Do Bayesian Inference [56.99390658880008]
  We present Prior-Data Fitted Networks (PFNs) PFNs leverage in-context learning in large-scale machine learning techniques to approximate a large set of posteriors. We demonstrate that PFNs can near-perfectly mimic Gaussian processes and also enable efficient Bayesian inference for intractable problems.
  arXiv  Detail & Related papers  (2021-12-20T13:07:39Z)
• Structural Learning of Probabilistic Sentential Decision Diagrams under Partial Closed-World Assumption [127.439030701253]
  Probabilistic sentential decision diagrams are a class of structured-decomposable circuits. We propose a new scheme based on a partial closed-world assumption: data implicitly provide the logical base of the circuit. Preliminary experiments show that the proposed approach might properly fit training data, and generalize well to test data, provided that these remain consistent with the underlying logical base.
  arXiv  Detail & Related papers  (2021-07-26T12:01:56Z)
• FF-NSL: Feed-Forward Neural-Symbolic Learner [70.978007919101]
  This paper introduces a neural-symbolic learning framework, called Feed-Forward Neural-Symbolic Learner (FF-NSL) FF-NSL integrates state-of-the-art ILP systems based on the Answer Set semantics, with neural networks, in order to learn interpretable hypotheses from labelled unstructured data.
  arXiv  Detail & Related papers  (2021-06-24T15:38:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.