Nearly Optimal Bayesian Inference for Structural Missingness

• URL: http://arxiv.org/abs/2601.18500v2
• Date: Tue, 27 Jan 2026 12:05:37 GMT
• Title: Nearly Optimal Bayesian Inference for Structural Missingness
• Authors: Chen Liang, Donghua Yang, Yutong Zhao, Tianle Zhang, Shenghang Zhou, Zhiyu Liang, Hengtong Zhang, Hongzhi Wang, Ziqi Li, Xiyang Zhang, Zheng Liang, Yifei Li,
• Abstract summary: In the Bayesian view, prediction via the posterior predictive distribution integrates over the full model posterior uncertainty.<n>This framework decouples learning an in-model missing-value posterior from (ii) label prediction by optimizing the predictive posterior distribution.<n>It achieves SOTA on 43 classification and 15 imputation benchmarks, with finite-sample near Bayes-optimality guarantees.
• Score: 23.988531482641307
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Structural missingness breaks 'just impute and train': values can be undefined by causal or logical constraints, and the mask may depend on observed variables, unobserved variables (MNAR), and other missingness indicators. It simultaneously brings (i) a catch-22 situation with causal loop, prediction needs the missing features, yet inferring them depends on the missingness mechanism, (ii) under MNAR, the unseen are different, the missing part can come from a shifted distribution, and (iii) plug-in imputation, a single fill-in can lock in uncertainty and yield overconfident, biased decisions. In the Bayesian view, prediction via the posterior predictive distribution integrates over the full model posterior uncertainty, rather than relying on a single point estimate. This framework decouples (i) learning an in-model missing-value posterior from (ii) label prediction by optimizing the predictive posterior distribution, enabling posterior integration. This decoupling yields an in-model almost-free-lunch: once the posterior is learned, prediction is plug-and-play while preserving uncertainty propagation. It achieves SOTA on 43 classification and 15 imputation benchmarks, with finite-sample near Bayes-optimality guarantees under our SCM prior.

Related papers

• Robust Predictive Uncertainty and Double Descent in Contaminated Bayesian Random Features [9.140494844209336]
  We propose a robust Bayesian formulation of random feature (RF) regression that accounts explicitly for prior and likelihood misspecification.<n>We derive explicit and tractable bounds for the resulting lower and upper posterior predictive envelopes.
  arXiv  Detail & Related papers  (2026-02-22T10:50:04Z)
• Predictively Oriented Posteriors [4.135680181585462]
  We advocate a new statistical principle that combines the most desirable aspects of both parameter inference and density estimation.<n>PrO posteriors converge to the predictively optimal model average at rate $n-1/2$.<n>We show that PrO posteriors can be sampled from by evolving particles based on mean field Langevin dynamics.
  arXiv  Detail & Related papers  (2025-10-02T11:33:26Z)
• Revisiting Multivariate Time Series Forecasting with Missing Values [65.30332997607141]
  Missing values are common in real-world time series.<n>Current approaches have developed an imputation-then-prediction framework that uses imputation modules to fill in missing values, followed by forecasting on the imputed data.<n>This framework overlooks a critical issue: there is no ground truth for the missing values, making the imputation process susceptible to errors that can degrade prediction accuracy.<n>We introduce Consistency-Regularized Information Bottleneck (CRIB), a novel framework built on the Information Bottleneck principle.
  arXiv  Detail & Related papers  (2025-09-27T20:57:48Z)
• Principled Input-Output-Conditioned Post-Hoc Uncertainty Estimation for Regression Networks [1.4671424999873808]
  Uncertainty is critical in safety-sensitive applications but is often omitted from off-the-shelf neural networks due to adverse effects on predictive performance.<n>We propose a theoretically grounded framework for post-hoc uncertainty estimation in regression tasks by fitting an auxiliary model to both original inputs and frozen model outputs.
  arXiv  Detail & Related papers  (2025-06-01T09:13:27Z)
• Predictive variational inference: Learn the predictively optimal posterior distribution [1.7648680700685022]
  Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification.<n>We propose predictive variational inference (PVI): a general inference framework that seeks and samples from an optimal posterior density.
  arXiv  Detail & Related papers  (2024-10-18T19:44:57Z)
• Calibrating Neural Simulation-Based Inference with Differentiable Coverage Probability [50.44439018155837]
  We propose to include a calibration term directly into the training objective of the neural model. By introducing a relaxation of the classical formulation of calibration error we enable end-to-end backpropagation. It is directly applicable to existing computational pipelines allowing reliable black-box posterior inference.
  arXiv  Detail & Related papers  (2023-10-20T10:20:45Z)
• Looking at the posterior: accuracy and uncertainty of neural-network predictions [0.0]
  We show that prediction accuracy depends on both epistemic and aleatoric uncertainty. We introduce a novel acquisition function that outperforms common uncertainty-based methods.
  arXiv  Detail & Related papers  (2022-11-26T16:13:32Z)
• Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [68.34559610536614]
  We argue that the aleatoric uncertainty is an inherent attribute of the data and can only be correctly estimated with an unbiased oracle model. We propose a new sampling and selection strategy at train time to approximate the oracle model for aleatoric uncertainty estimation. Our results show that our solution achieves both accurate deterministic results and reliable uncertainty estimation.
  arXiv  Detail & Related papers  (2021-11-22T08:54:10Z)
• Dense Uncertainty Estimation [62.23555922631451]
  In this paper, we investigate neural networks and uncertainty estimation techniques to achieve both accurate deterministic prediction and reliable uncertainty estimation. We work on two types of uncertainty estimations solutions, namely ensemble based methods and generative model based methods, and explain their pros and cons while using them in fully/semi/weakly-supervised framework.
  arXiv  Detail & Related papers  (2021-10-13T01:23:48Z)
• Multivariate Probabilistic Regression with Natural Gradient Boosting [63.58097881421937]
  We propose a Natural Gradient Boosting (NGBoost) approach based on nonparametrically modeling the conditional parameters of the multivariate predictive distribution. Our method is robust, works out-of-the-box without extensive tuning, is modular with respect to the assumed target distribution, and performs competitively in comparison to existing approaches.
  arXiv  Detail & Related papers  (2021-06-07T17:44:49Z)
• Uncertainty quantification for nonconvex tensor completion: Confidence intervals, heteroscedasticity and optimality [92.35257908210316]
  We study the problem of estimating a low-rank tensor given incomplete and corrupted observations. We find that it attains unimprovable rates $ell-2$ accuracy.
  arXiv  Detail & Related papers  (2020-06-15T17:47:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.