Variational Approximations for Robust Bayesian Inference via Rho-Posteriors

• URL: http://arxiv.org/abs/2601.07325v1
• Date: Mon, 12 Jan 2026 08:47:13 GMT
• Title: Variational Approximations for Robust Bayesian Inference via Rho-Posteriors
• Abstract summary: PAC-Bayesian framework for $$-posterior inference developed.<n>First practical implementation of $$-posterior inference with rigorous finite-sample guarantees.
• Score: 4.042110592015443
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The $ρ$-posterior framework provides universal Bayesian estimation with explicit contamination rates and optimal convergence guarantees, but has remained computationally difficult due to an optimization over reference distributions that precludes intractable posterior computation. We develop a PAC-Bayesian framework that recovers these theoretical guarantees through temperature-dependent Gibbs posteriors, deriving finite-sample oracle inequalities with explicit rates and introducing tractable variational approximations that inherit the robustness properties of exact $ρ$-posteriors. Numerical experiments demonstrate that this approach achieves theoretical contamination rates while remaining computationally feasible, providing the first practical implementation of $ρ$-posterior inference with rigorous finite-sample guarantees.

Related papers

• Revisiting Zeroth-Order Hessian Approximation: A Single-Step Policy Optimization Lens [18.253856190297405]
  ZoVH is a suite of variance-reduced estimators for the full Hessian matrix, its regularized inverse, and the bias-corrected inverse Hessian-gradient product.<n>We show that ZoVH achieves superior estimation accuracy and convergence performance in real-world applications.
  arXiv  Detail & Related papers  (2026-05-29T07:55:57Z)
• Possibilistic Predictive Uncertainty for Deep Learning [75.00607359322863]
  We introduce Dirichlet-approximated possibilistic posterior predictions (DAPPr)<n>DAPPr is a principled framework leveraging possibility theory.<n>This projection-and-approximation strategy yields a simple training objective with closed-form solutions.
  arXiv  Detail & Related papers  (2026-05-01T12:14:01Z)
• Robust Predictive Uncertainty and Double Descent in Contaminated Bayesian Random Features [9.140494844209336]
  We propose a robust Bayesian formulation of random feature (RF) regression that accounts explicitly for prior and likelihood misspecification.<n>We derive explicit and tractable bounds for the resulting lower and upper posterior predictive envelopes.
  arXiv  Detail & Related papers  (2026-02-22T10:50:04Z)
• On the Probabilistic Learnability of Compact Neural Network Preimage Bounds [71.59148070050212]
  In this work, we adopt a novel probabilistic perspective, aiming to deliver solutions with high-confidence guarantees and bounded error.<n>We introduce $textbfR$andom $textbfF$orest $textbfPro$perty $textbfVe$rifier ($texttRF-ProVe$), a method that exploits an ensemble of randomized decision trees to generate candidate input regions satisfying a desired output property.<n>Our theoretical derivations offer formal statistical guarantees on region purity and global coverage, providing a practical, scalable solution for computing
  arXiv  Detail & Related papers  (2025-11-10T16:56:51Z)
• Dependent Randomized Rounding for Budget Constrained Experimental Design [18.677808169753032]
  We propose a framework that applies a dependent randomized rounding procedure to convert assignment probabilities into binary treatment decisions.<n>Our proposed solution preserves the marginal treatment probabilities while inducing negative correlations among assignments, leading to improved estimator precision through variance reduction.
  arXiv  Detail & Related papers  (2025-06-15T01:10:06Z)
• Doubly-Robust Estimation of Counterfactual Policy Mean Embeddings [23.3862001690226]
  Estimating the distribution of outcomes under counterfactual policies is critical for decision-making in domains such as recommendation, advertising, and healthcare.<n>We propose and analyze a novel framework-Counterfactual Policy Mean Embedding (CPME)-that represents the entire counterfactual outcome distribution in a reproducing kernel space (RKHS)<n>We introduce both a plug-in estimator and a doubly robust estimator; the latter enjoys improved convergence rates by correcting for bias in both the outcome embedding and propensity models.
  arXiv  Detail & Related papers  (2025-06-03T12:16:46Z)
• Epistemic Uncertainty in Conformal Scores: A Unified Approach [2.449909275410288]
  Conformal prediction methods create prediction bands with distribution-free guarantees but do not explicitly capture uncertainty.<n>We introduce $texttEPICSCORE$, a model-agnostic approach that enhances any conformal score by explicitly integrating uncertainty.<n>$texttEPICSCORE$ adaptively expands predictive intervals in regions with limited data while maintaining compact intervals where data is abundant.
  arXiv  Detail & Related papers  (2025-02-10T19:42:54Z)
• Achieving $\widetilde{\mathcal{O}}(\sqrt{T})$ Regret in Average-Reward POMDPs with Known Observation Models [69.1820058966619]
  We tackle average-reward infinite-horizon POMDPs with an unknown transition model.<n>We present a novel and simple estimator that overcomes this barrier.
  arXiv  Detail & Related papers  (2025-01-30T22:29:41Z)
• Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networks [11.218066045459778]
  This paper addresses the problems of conditional variance estimation and confidence interval construction in nonparametric regression using dense networks with the Rectified Linear Unit (ReLU) activation function.<n>We present a residual-based framework for conditional variance estimation, deriving nonasymptotic bounds for variance estimation under both heteroscedastic and homoscedastic settings.<n>We develop a ReLU network based robust bootstrap procedure for constructing confidence intervals for the true mean that comes with a theoretical guarantee on the coverage, providing a significant advancement in uncertainty quantification and the construction of reliable confidence intervals in deep learning settings.
  arXiv  Detail & Related papers  (2024-12-29T05:17:58Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Model-Based Epistemic Variance of Values for Risk-Aware Policy Optimization [59.758009422067]
  We consider the problem of quantifying uncertainty over expected cumulative rewards in model-based reinforcement learning. We propose a new uncertainty Bellman equation (UBE) whose solution converges to the true posterior variance over values. We introduce a general-purpose policy optimization algorithm, Q-Uncertainty Soft Actor-Critic (QU-SAC) that can be applied for either risk-seeking or risk-averse policy optimization.
  arXiv  Detail & Related papers  (2023-12-07T15:55:58Z)
• Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergence [77.06203118175335]
  Variational Inference (VI) is a popular alternative to exact sampling in Bayesian inference. Importance sampling (IS) is often used to fine-tune and de-bias the estimates of approximate Bayesian inference procedures. We propose a novel combination of optimization and sampling techniques for approximate Bayesian inference.
  arXiv  Detail & Related papers  (2021-06-30T11:00:24Z)
• Adversarial Robustness Guarantees for Gaussian Processes [22.403365399119107]
  Gaussian processes (GPs) enable principled computation of model uncertainty, making them attractive for safety-critical applications. We present a framework to analyse adversarial robustness of GPs, defined as invariance of the model's decision to bounded perturbations. We develop a branch-and-bound scheme to refine the bounds and show, for any $epsilon > 0$, that our algorithm is guaranteed to converge to values $epsilon$-close to the actual values in finitely many iterations.
  arXiv  Detail & Related papers  (2021-04-07T15:14:56Z)
• $\gamma$-ABC: Outlier-Robust Approximate Bayesian Computation Based on a Robust Divergence Estimator [95.71091446753414]
  We propose to use a nearest-neighbor-based $gamma$-divergence estimator as a data discrepancy measure. Our method achieves significantly higher robustness than existing discrepancy measures.
  arXiv  Detail & Related papers  (2020-06-13T06:09:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.