Position: The Future of Bayesian Prediction Is Prior-Fitted

• URL: http://arxiv.org/abs/2505.23947v1
• Date: Thu, 29 May 2025 18:56:45 GMT
• Title: Position: The Future of Bayesian Prediction Is Prior-Fitted
• Authors: Samuel Müller, Arik Reuter, Noah Hollmann, David Rügamer, Frank Hutter,
• Abstract summary: Prior-data Fitted Networks (PFNs) are a class of methods designed to leverage this insight.<n>PFNs enable the efficient allocation of pre-training compute to low-data scenarios.<n>This position paper argues that PFNs and other amortized inference approaches represent the future of Bayesian inference.
• Score: 41.71468589060514
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Training neural networks on randomly generated artificial datasets yields Bayesian models that capture the prior defined by the dataset-generating distribution. Prior-data Fitted Networks (PFNs) are a class of methods designed to leverage this insight. In an era of rapidly increasing computational resources for pre-training and a near stagnation in the generation of new real-world data in many applications, PFNs are poised to play a more important role across a wide range of applications. They enable the efficient allocation of pre-training compute to low-data scenarios. Originally applied to small Bayesian modeling tasks, the field of PFNs has significantly expanded to address more complex domains and larger datasets. This position paper argues that PFNs and other amortized inference approaches represent the future of Bayesian inference, leveraging amortized learning to tackle data-scarce problems. We thus believe they are a fruitful area of research. In this position paper, we explore their potential and directions to address their current limitations.

Related papers

• Prior-Fitted Networks Scale to Larger Datasets When Treated as Weak Learners [82.72552644267724]
  BoostPFN can outperform standard PFNs with the same size of training samples in large datasets.<n>High performance is maintained for up to 50x of the pre-training size of PFNs.
  arXiv  Detail & Related papers  (2025-03-03T07:31:40Z)
• Scalable Spatiotemporal Prediction with Bayesian Neural Fields [3.3299088915999295]
  BayesNF integrates a deep neural network architecture for high-capacity estimation with hierarchical Bayesian inference for robust predictive uncertainty.<n>BayesNF delivers improvements on prediction problems from climate and public health data containing tens of thousands of measurements.
  arXiv  Detail & Related papers  (2024-03-12T13:47:50Z)
• Cumulative Distribution Function based General Temporal Point Processes [49.758080415846884]
  CuFun model represents a novel approach to TPPs that revolves around the Cumulative Distribution Function (CDF) Our approach addresses several critical issues inherent in traditional TPP modeling. Our contributions encompass the introduction of a pioneering CDF-based TPP model, the development of a methodology for incorporating past event information into future event prediction.
  arXiv  Detail & Related papers  (2024-02-01T07:21:30Z)
• Tackling Computational Heterogeneity in FL: A Few Theoretical Insights [68.8204255655161]
  We introduce and analyse a novel aggregation framework that allows for formalizing and tackling computational heterogeneous data. Proposed aggregation algorithms are extensively analyzed from a theoretical, and an experimental prospective.
  arXiv  Detail & Related papers  (2023-07-12T16:28:21Z)
• Learning Representations on the Unit Sphere: Investigating Angular Gaussian and von Mises-Fisher Distributions for Online Continual Learning [7.145581090959242]
  We propose a memory-based representation learning technique equipped with our new loss functions. We demonstrate that the proposed method outperforms the current state-of-the-art methods on both standard evaluation scenarios and realistic scenarios with blurry task boundaries.
  arXiv  Detail & Related papers  (2023-06-06T02:38:01Z)
• Transformers Can Do Bayesian Inference [56.99390658880008]
  We present Prior-Data Fitted Networks (PFNs) PFNs leverage in-context learning in large-scale machine learning techniques to approximate a large set of posteriors. We demonstrate that PFNs can near-perfectly mimic Gaussian processes and also enable efficient Bayesian inference for intractable problems.
  arXiv  Detail & Related papers  (2021-12-20T13:07:39Z)
• Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
  In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
  arXiv  Detail & Related papers  (2020-03-10T03:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.