Vector Copula Variational Inference and Dependent Block Posterior Approximations

• URL: http://arxiv.org/abs/2503.01072v2
• Date: Mon, 06 Oct 2025 00:50:29 GMT
• Title: Vector Copula Variational Inference and Dependent Block Posterior Approximations
• Abstract summary: Key to VI is the selection of a tractable density to approximate the Bayesian posterior.<n>This paper proposes using vector copulas to capture dependence between the blocks parsimoniously.
• Score: 5.442153262282512
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The key to VI is the selection of a tractable density to approximate the Bayesian posterior. For large and complex models a common choice is to assume independence between multivariate blocks in a partition of the parameter space. While this simplifies the problem it can reduce accuracy. This paper proposes using vector copulas to capture dependence between the blocks parsimoniously. Tailored multivariate marginals are constructed using learnable transport maps. We call the resulting joint distribution a ``dependent block posterior'' approximation. Vector copula models are suggested that make tractable and flexible variational approximations. They allow for differing marginals, numbers of blocks, block sizes and forms of between block dependence. They also allow for solution of the variational optimization using efficient stochastic gradient methods. The approach is demonstrated using four different statistical models and 16 datasets which have posteriors that are challenging to approximate. This includes models that use global-local shrinkage priors for regularization, and hierarchical models for smoothing and heteroscedastic time series. In all cases, our method produces more accurate posterior approximations than benchmark VI methods that either assume block independence or factor-based dependence, at limited additional computational cost. A python package implementing the method is available on GitHub at https://github.com/YuFuOliver/VCVI_Rep_PyPackage.

Related papers

• Flow-Transformed Implicit Processes for Function-Space Variational Inference [4.970364068620607]
  Implicit-process priors define distributions over functions through flexible generative mechanisms.<n>One practical strategy is to approximate the prior using a finite collection of sampled functions, and then represent posterior functions as learned combinations of these samples.<n>We propose Flow-Transformed Implicit Processes (FTIP), a variational inference method that makes this finite-dimensional function-space approximation more expressive.
  arXiv  Detail & Related papers  (2026-06-01T09:14:09Z)
• Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention [58.484607892210015]
  Diffusion Language Models (DLMs) enable globally coherent, bidirectional, and controllable text generation.<n>Many existing block-sparse attention methods select blocks by fixed sampling patterns over the high-resolution attention space.<n>We propose the Block Approximate Sparse Attention framework (BA-Att) with block-wise pre-downsampled operation.
  arXiv  Detail & Related papers  (2026-05-19T12:01:13Z)
• PFF-Net: Patch Feature Fitting for Point Cloud Normal Estimation [81.94096000733127]
  We present a new idea of feature extraction for robust normal estimation of point clouds.<n>We use the fusion of multi-scale features from different neighborhood sizes to address the issue of selecting reasonable patch sizes for various data or geometries.<n>Our approximation strategy based on aggregating the features of multiple scales enables the model to achieve scale adaptation of varying local patches.
  arXiv  Detail & Related papers  (2025-11-26T13:12:14Z)
• Kernel Trace Distance: Quantum Statistical Metric between Measures through RKHS Density Operators [11.899035547580201]
  We introduce a novel distance between measures that compares them through a Schatten norm of their kernel covariance operators.<n>We show that this new distance is an integral probability metric that can be framed between a Maximum Mean Discrepancy (MMD) and a Wasserstein distance.
  arXiv  Detail & Related papers  (2025-07-08T14:56:44Z)
• EigenVI: score-based variational inference with orthogonal function expansions [23.696028065251497]
  EigenVI is an eigenvalue-based approach for black-box variational inference (BBVI) We use EigenVI to approximate a variety of target distributions, including a benchmark suite of Bayesian models from posteriordb.
  arXiv  Detail & Related papers  (2024-10-31T15:48:34Z)
• Scalable Inference for Bayesian Multinomial Logistic-Normal Dynamic Linear Models [0.5735035463793009]
  This article develops an efficient and accurate approach to posterior state estimation, called $textitFenrir$.<n>Our experiments suggest that Fenrir can be three orders of magnitude more efficient than Stan.<n>Our methods are made available to the community as a user-friendly software library written in C++ with an R interface.
  arXiv  Detail & Related papers  (2024-10-07T23:20:14Z)
• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate. We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Fast Semisupervised Unmixing Using Nonconvex Optimization [80.11512905623417]
  We introduce a novel convex convex model for semi/library-based unmixing. We demonstrate the efficacy of Alternating Methods of sparse unsupervised unmixing.
  arXiv  Detail & Related papers  (2024-01-23T10:07:41Z)
• PAVI: Plate-Amortized Variational Inference [55.975832957404556]
  Inference is challenging for large population studies where millions of measurements are performed over a cohort of hundreds of subjects. This large cardinality renders off-the-shelf Variational Inference (VI) computationally impractical. In this work, we design structured VI families that efficiently tackle large population studies.
  arXiv  Detail & Related papers  (2023-08-30T13:22:20Z)
• BayOTIDE: Bayesian Online Multivariate Time series Imputation with functional decomposition [31.096125530322933]
  In real-world scenarios like traffic and energy, massive time-series data with missing values and noises are widely observed, even sampled irregularly. While many imputation methods have been proposed, most of them work with a local horizon. Almost all methods assume the observations are sampled at regular time stamps, and fail to handle complex irregular sampled time series.
  arXiv  Detail & Related papers  (2023-08-28T21:17:12Z)
• Bayesian Pseudo-Coresets via Contrastive Divergence [5.479797073162603]
  We introduce a novel approach for constructing pseudo-coresets by utilizing contrastive divergence. It eliminates the need for approximations in the pseudo-coreset construction process. We conduct extensive experiments on multiple datasets, demonstrating its superiority over existing BPC techniques.
  arXiv  Detail & Related papers  (2023-03-20T17:13:50Z)
• VI-DGP: A variational inference method with deep generative prior for solving high-dimensional inverse problems [0.7734726150561089]
  We propose a novel approximation method for estimating the high-dimensional posterior distribution. This approach leverages a deep generative model to learn a prior model capable of generating spatially-varying parameters. The proposed method can be fully implemented in an automatic differentiation manner.
  arXiv  Detail & Related papers  (2023-02-22T06:48:10Z)
• Manifold Gaussian Variational Bayes on the Precision Matrix [70.44024861252554]
  We propose an optimization algorithm for Variational Inference (VI) in complex models. We develop an efficient algorithm for Gaussian Variational Inference whose updates satisfy the positive definite constraint on the variational covariance matrix. Due to its black-box nature, MGVBP stands as a ready-to-use solution for VI in complex models.
  arXiv  Detail & Related papers  (2022-10-26T10:12:31Z)
• Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization [49.58290066287418]
  We propose a novel method named Multi-block-probe Variance Reduced (MSVR) to alleviate the complexity of compositional problems. Our results improve upon prior ones in several aspects, including the order of sample complexities and dependence on strongity.
  arXiv  Detail & Related papers  (2022-07-18T12:03:26Z)
• Variational Inference for Bayesian Bridge Regression [0.0]
  We study the implementation of Automatic Differentiation Variational inference (ADVI) for Bayesian inference on regression models with bridge penalization. The bridge approach uses $ell_alpha$ norm, with $alpha in (0, +infty)$ to define a penalization on large values of the regression coefficients. We illustrate the approach on non-parametric regression models with B-splines, although the method works seamlessly for other choices of basis functions.
  arXiv  Detail & Related papers  (2022-05-19T12:29:09Z)
• Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression [151.3826781154146]
  latent variables with priors and hyperpriors is an essential problem in variational image compression. We find inter-correlations and intra-correlations exist when observing latent variables in a vectorized perspective. Our model has better rate-distortion performance and an impressive $3.18times$ compression speed up.
  arXiv  Detail & Related papers  (2022-03-21T11:44:17Z)
• State space partitioning based on constrained spectral clustering for block particle filtering [7.32172860877574]
  The particle filter (PF) is a powerful inference tool widely used to estimate the filtering distribution in non-linear and/or non-Gaussian problems. To overcome the curse of dimensionality of PF, the block PF (BPF) inserts a blocking step to partition the state space into several subspaces or blocks of smaller dimension. Using blocks of small size reduces the variance of the filtering distribution estimate, but in turn the correlation between blocks is broken and a bias is introduced.
  arXiv  Detail & Related papers  (2022-03-07T15:45:47Z)
• Machine Learning for Multi-Output Regression: When should a holistic multivariate approach be preferred over separate univariate ones? [62.997667081978825]
  Tree-based ensembles such as the Random Forest are modern classics among statistical learning methods. We compare these methods in extensive simulations to help in answering the primary question when to use multivariate ensemble techniques.
  arXiv  Detail & Related papers  (2022-01-14T08:44:25Z)
• Scaling Structured Inference with Randomization [64.18063627155128]
  We propose a family of dynamic programming (RDP) randomized for scaling structured models to tens of thousands of latent states. Our method is widely applicable to classical DP-based inference. It is also compatible with automatic differentiation so can be integrated with neural networks seamlessly.
  arXiv  Detail & Related papers  (2021-12-07T11:26:41Z)
• Efficient semidefinite bounds for multi-label discrete graphical models [6.226454551201676]
  One of the main queries on such models is to identify the SDPWCSP Function on Cost of a Posteri (MAP) Networks. We consider a traditional dualized constraint approach and a dedicated dedicated SDP/Monteiro style method based on row-by-row updates.
  arXiv  Detail & Related papers  (2021-11-24T13:38:34Z)
• On the Use of Minimum Penalties in Statistical Learning [2.1320960069210475]
  We propose a framework to simultaneously estimate regression coefficients associated with a multivariate regression model and relationships between outcome variables. An iterative algorithm that generalizes current state art methods is proposed as a solution. We extend the proposed MinPen framework to other exponential family loss functions, with a specific focus on multiple binomial responses.
  arXiv  Detail & Related papers  (2021-06-09T16:15:46Z)
• Multivariate Probabilistic Regression with Natural Gradient Boosting [63.58097881421937]
  We propose a Natural Gradient Boosting (NGBoost) approach based on nonparametrically modeling the conditional parameters of the multivariate predictive distribution. Our method is robust, works out-of-the-box without extensive tuning, is modular with respect to the assumed target distribution, and performs competitively in comparison to existing approaches.
  arXiv  Detail & Related papers  (2021-06-07T17:44:49Z)
• Causality-based Counterfactual Explanation for Classification Models [11.108866104714627]
  We propose a prototype-based counterfactual explanation framework (ProCE) ProCE is capable of preserving the causal relationship underlying the features of the counterfactual data. In addition, we design a novel gradient-free optimization based on the multi-objective genetic algorithm that generates the counterfactual explanations.
  arXiv  Detail & Related papers  (2021-05-03T09:25:59Z)
• Probabilistic Circuits for Variational Inference in Discrete Graphical Models [101.28528515775842]
  Inference in discrete graphical models with variational methods is difficult. Many sampling-based methods have been proposed for estimating Evidence Lower Bound (ELBO) We propose a new approach that leverages the tractability of probabilistic circuit models, such as Sum Product Networks (SPN) We show that selective-SPNs are suitable as an expressive variational distribution, and prove that when the log-density of the target model is aweighted the corresponding ELBO can be computed analytically.
  arXiv  Detail & Related papers  (2020-10-22T05:04:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.