Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learning

• URL: http://arxiv.org/abs/2605.21765v1
• Date: Wed, 20 May 2026 21:57:21 GMT
• Title: Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learning
• Abstract summary: Practical adoption of sampling-based inference (SAI) in Bayesian neural networks (BNNs) remains limited, partly due to persistent misconceptions about the feasibility and efficiency of sampling.<n>This paper argues that SAI has achieved computational parity with optimization-based methods and is at the verge of superseding such methods for effective and efficient inference in BNNs.
• Score: 15.645799592866569
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The practical adoption of sampling-based inference (SAI) in Bayesian neural networks (BNNs) remains limited, partly due to persistent misconceptions about the feasibility and efficiency of sampling. This position paper argues that SAI has achieved computational parity with optimization-based methods and is at the verge of superseding such methods for effective and efficient inference in BNNs. This development should be in the interest of the whole community, promoting BNNs as a principled paradigm with its long-standing yet unfulfilled promise of providing principled uncertainty quantification for neural networks. SAI can even do more -- yielding superior prediction performance through model averaging, serving as the foundation for a plethora of possible downstream tasks, and providing crucial insights into the landscape of BNNs. In order to make such a change happen and unfold the potential of sampling, overcoming current misconceptions is a necessary first step. The next step is to realign research efforts toward addressing remaining challenges in SAI. In particular, the community must focus on two core problems: sufficient exploration of the posterior landscape and high-fidelity distillation of posterior samples for efficient downstream inference. By addressing conceptual and practical obstacles, we can unlock the full potential of SAI and establish it as a central tool in Bayesian deep learning.

Related papers

• Possibilistic Predictive Uncertainty for Deep Learning [75.00607359322863]
  We introduce Dirichlet-approximated possibilistic posterior predictions (DAPPr)<n>DAPPr is a principled framework leveraging possibility theory.<n>This projection-and-approximation strategy yields a simple training objective with closed-form solutions.
  arXiv  Detail & Related papers  (2026-05-01T12:14:01Z)
• Interior-Point Vanishing Problem in Semidefinite Relaxations for Neural Network Verification [9.063592930122113]
  Semidefinite programming (SDP) relaxation has emerged as a promising approach for neural network verification.<n>We identify a critical limitation in the SDP relaxation when applied to deep networks: interior-point vanishing.<n>We investigate five solutions to enhance the feasibility conditions of the verification problem.
  arXiv  Detail & Related papers  (2025-06-12T01:25:21Z)
• Bayesian Neural Scaling Law Extrapolation with Prior-Data Fitted Networks [100.13335639780415]
  Scaling laws often follow the power-law and proposed several variants of power-law functions to predict the scaling behavior at larger scales.<n>Existing methods mostly rely on point estimation and do not quantify uncertainty, which is crucial for real-world applications.<n>In this work, we explore a Bayesian framework based on Prior-data Fitted Networks (PFNs) for neural scaling law extrapolation.
  arXiv  Detail & Related papers  (2025-05-29T03:19:17Z)
• Last Layer Empirical Bayes [24.94944362766761]
  Bayesian neural networks (BNNs) and deep ensembles are among the most prominent approaches to tackle this task.<n>Inspired by recent work showing that the distribution used by ensembles can be understood as a posterior corresponding to a learned data-dependent prior, we propose last layer empirical Bayes (LLEB)<n>LLEB performs on par with existing approaches, highlighting that empirical Bayes is a promising direction for future research in uncertainty quantification.
  arXiv  Detail & Related papers  (2025-05-21T18:00:00Z)
• A Primer on Bayesian Neural Networks: Review and Debates [10.074350562162921]
  Bayesian neural networks (BNNs) have emerged as a compelling extension of conventional neural networks. This primer presents a systematic introduction to the fundamental concepts of neural networks and Bayesian inference. The target audience comprises statisticians with a potential background in Bayesian methods but lacking deep learning expertise.
  arXiv  Detail & Related papers  (2023-09-28T10:09:15Z)
• Likelihood-ratio-based confidence intervals for neural networks [1.4610038284393165]
  This paper introduces a first implementation of a novel likelihood-ratio-based approach for constructing confidence intervals for neural networks. While the current implementation of the method is prohibitively expensive for many deep-learning applications, the high cost may already be justified in specific fields like medical predictions or astrophysics. This work highlights the significant potential of a likelihood-ratio-based uncertainty estimate and establishes a promising avenue for future research.
  arXiv  Detail & Related papers  (2023-08-04T09:34:48Z)
• Efficient Bayes Inference in Neural Networks through Adaptive Importance Sampling [19.518237361775533]
  In BNNs, a complete posterior distribution of the unknown weight and bias parameters of the network is produced during the training stage. This feature is useful in countless machine learning applications. It is particularly appealing in areas where decision-making has a crucial impact, such as medical healthcare or autonomous driving.
  arXiv  Detail & Related papers  (2022-10-03T14:59:23Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• Spatial-Temporal-Fusion BNN: Variational Bayesian Feature Layer [77.78479877473899]
  We design a spatial-temporal-fusion BNN for efficiently scaling BNNs to large models. Compared to vanilla BNNs, our approach can greatly reduce the training time and the number of parameters, which contributes to scale BNNs efficiently.
  arXiv  Detail & Related papers  (2021-12-12T17:13:14Z)
• On the Practicality of Deterministic Epistemic Uncertainty [106.06571981780591]
  deterministic uncertainty methods (DUMs) achieve strong performance on detecting out-of-distribution data. It remains unclear whether DUMs are well calibrated and can seamlessly scale to real-world applications.
  arXiv  Detail & Related papers  (2021-07-01T17:59:07Z)
• S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration [74.5509794733707]
  We present a novel guided learning paradigm from real-valued to distill binary networks on the final prediction distribution. Our proposed method can boost the simple contrastive learning baseline by an absolute gain of 5.515% on BNNs. Our method achieves substantial improvement over the simple contrastive learning baseline, and is even comparable to many mainstream supervised BNN methods.
  arXiv  Detail & Related papers  (2021-02-17T18:59:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.