Fast sampling and model selection for Bayesian mixture models

• URL: http://arxiv.org/abs/2501.07668v2
• Date: Wed, 22 Oct 2025 15:43:12 GMT
• Title: Fast sampling and model selection for Bayesian mixture models
• Authors: M. E. J. Newman,
• Abstract summary: We argue in favor of fitting the marginal posterior distribution over component assignments directly, rather than Gibbs sampling from the joint posterior.<n>We describe a new Monte Carlo algorithm for sampling from the marginal posterior of a general integrable mixture.<n>We demonstrate the approach with a selection of applications to Gaussian, Poisson, and categorical models.
• Score: 3.198144010381572
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study Bayesian estimation of mixture models and argue in favor of fitting the marginal posterior distribution over component assignments directly, rather than Gibbs sampling from the joint posterior on components and parameters as is commonly done. Some previous authors have found the former approach to have slow mixing, but we show that, implemented correctly, it can achieve excellent performance. In particular, we describe a new Monte Carlo algorithm for sampling from the marginal posterior of a general integrable mixture that makes use of rejection-free sampling from the prior over component assignments to achieve excellent mixing times in typical applications, outperforming standard Gibbs sampling, in some cases by a wide margin. We demonstrate the approach with a selection of applications to Gaussian, Poisson, and categorical models.

Related papers

• Learnable Chernoff Baselines for Inference-Time Alignment [64.81256817158851]
  We introduce Learnable Chernoff Baselines as a method for efficiently and approximately sampling from exponentially tilted kernels.<n>We establish total-variation guarantees to the ideal aligned model, and demonstrate in both continuous and discrete diffusion settings that LCB sampling closely matches ideal rejection sampling.
  arXiv  Detail & Related papers  (2026-02-08T00:09:40Z)
• A fast non-reversible sampler for Bayesian finite mixture models [1.4466802614938334]
  We introduce a novel and simple non-reversible sampling scheme for finite mixture models.<n>We show that the performance of the proposed non-reversible scheme cannot be worse than the standard one.<n>We also discuss why the statistical features of mixture models make them an ideal case for the use of non-reversible discrete samplers.
  arXiv  Detail & Related papers  (2025-10-03T17:57:44Z)
• The Gaussian Latent Machine: Efficient Prior and Posterior Sampling for Inverse Problems [19.914084083626694]
  We show that a product-of-experts-type model can be easily lifted into a novel latent variable model.<n>This leads to a general sampling approach that unifies and generalizes many existing sampling algorithms.
  arXiv  Detail & Related papers  (2025-05-19T08:21:23Z)
• Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts [64.34482582690927]
  We provide an efficient and principled method for sampling from a sequence of annealed, geometric-averaged, or product distributions derived from pretrained score-based models. We propose Sequential Monte Carlo (SMC) resampling algorithms that leverage inference-time scaling to improve sampling quality.
  arXiv  Detail & Related papers  (2025-03-04T17:46:51Z)
• Mixture models for data with unknown distributions [0.6345523830122168]
  We describe and analyze a broad class of mixture models for real-valued multivariate data. We return both a division of the data and an estimate of the distributions, effectively performing clustering and density estimation within each cluster at the same time. We demonstrate our methods with a selection of illustrative applications and give code implementing both algorithms.
  arXiv  Detail & Related papers  (2025-02-26T22:42:40Z)
• Generative Modeling with Bayesian Sample Inference [50.07758840675341]
  We derive a novel generative model from iterative Gaussian posterior inference.<n>Our model uses a sequence of prediction and posterior update steps to iteratively narrow down the unknown sample.<n>In experiments, we demonstrate that our model improves sample quality on ImageNet32 over both BFNs and the closely related Variational Diffusion Models.
  arXiv  Detail & Related papers  (2025-02-11T14:27:10Z)
• Low-rank Bayesian matrix completion via geodesic Hamiltonian Monte Carlo on Stiefel manifolds [0.18416014644193066]
  We present a new sampling-based approach for enabling efficient computation of low-rank Bayesian matrix completion. We show that our approach resolves the sampling difficulties encountered by standard Gibbs samplers for the common two-matrix factorization used in matrix completion. Numerical examples demonstrate superior sampling performance, including better mixing and faster convergence to a stationary distribution.
  arXiv  Detail & Related papers  (2024-10-27T03:12:53Z)
• Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplers [49.1574468325115]
  We introduce a unified convergence analysis framework for deterministic samplers. Our framework achieves iteration complexity of $tilde O(d2/epsilon)$. We also provide a detailed analysis of Denoising Implicit Diffusion Models (DDIM)-type samplers.
  arXiv  Detail & Related papers  (2024-10-18T07:37:36Z)
• You Only Accept Samples Once: Fast, Self-Correcting Stochastic Variational Inference [0.0]
  YOASOVI is an algorithm for performing fast, self-correcting intuition optimization for Variational Inference (VI) on large Bayesian heirarchical models. To accomplish this, we take advantage of available information on the objective function used for VI at each iteration and replace regular Monte Carlo sampling with acceptance sampling.
  arXiv  Detail & Related papers  (2024-06-05T01:28:53Z)
• Fusion of Gaussian Processes Predictions with Monte Carlo Sampling [61.31380086717422]
  In science and engineering, we often work with models designed for accurate prediction of variables of interest. Recognizing that these models are approximations of reality, it becomes desirable to apply multiple models to the same data and integrate their outcomes.
  arXiv  Detail & Related papers  (2024-03-03T04:21:21Z)
• Fast Semisupervised Unmixing Using Nonconvex Optimization [80.11512905623417]
  We introduce a novel convex convex model for semi/library-based unmixing. We demonstrate the efficacy of Alternating Methods of sparse unsupervised unmixing.
  arXiv  Detail & Related papers  (2024-01-23T10:07:41Z)
• Dimension-free mixing times of Gibbs samplers for Bayesian hierarchical models [0.0]
  We analyse the behaviour of total variation mixing times of Gibbs samplers targeting hierarchical models. We obtain convergence results under random data-generating assumptions for a broad class of two-level models.
  arXiv  Detail & Related papers  (2023-04-14T08:30:40Z)
• Rethinking Collaborative Metric Learning: Toward an Efficient Alternative without Negative Sampling [156.7248383178991]
  Collaborative Metric Learning (CML) paradigm has aroused wide interest in the area of recommendation systems (RS) We find that negative sampling would lead to a biased estimation of the generalization error. Motivated by this, we propose an efficient alternative without negative sampling for CML named textitSampling-Free Collaborative Metric Learning (SFCML)
  arXiv  Detail & Related papers  (2022-06-23T08:50:22Z)
• Langevin Monte Carlo for Contextual Bandits [72.00524614312002]
  Langevin Monte Carlo Thompson Sampling (LMC-TS) is proposed to directly sample from the posterior distribution in contextual bandits. We prove that the proposed algorithm achieves the same sublinear regret bound as the best Thompson sampling algorithms for a special case of contextual bandits.
  arXiv  Detail & Related papers  (2022-06-22T17:58:23Z)
• Sampling from Arbitrary Functions via PSD Models [55.41644538483948]
  We take a two-step approach by first modeling the probability distribution and then sampling from that model. We show that these models can approximate a large class of densities concisely using few evaluations, and present a simple algorithm to effectively sample from these models.
  arXiv  Detail & Related papers  (2021-10-20T12:25:22Z)
• Direct sampling of projected entangled-pair states [0.0]
  Variational Monte Carlo studies employing projected entangled-pair states (PEPS) have recently shown that they can provide answers on long-standing questions. We propose a sampling algorithm that generates independent samples from a PEPS, bypassing all problems related to finite autocorrelation times.
  arXiv  Detail & Related papers  (2021-09-15T15:09:20Z)
• Finding Geometric Models by Clustering in the Consensus Space [61.65661010039768]
  We propose a new algorithm for finding an unknown number of geometric models, e.g., homographies. We present a number of applications where the use of multiple geometric models improves accuracy. These include pose estimation from multiple generalized homographies; trajectory estimation of fast-moving objects.
  arXiv  Detail & Related papers  (2021-03-25T14:35:07Z)
• Efficiently Sampling Functions from Gaussian Process Posteriors [76.94808614373609]
  We propose an easy-to-use and general-purpose approach for fast posterior sampling. We demonstrate how decoupled sample paths accurately represent Gaussian process posteriors at a fraction of the usual cost.
  arXiv  Detail & Related papers  (2020-02-21T14:03:16Z)
• Learning Gaussian Graphical Models via Multiplicative Weights [54.252053139374205]
  We adapt an algorithm of Klivans and Meka based on the method of multiplicative weight updates. The algorithm enjoys a sample complexity bound that is qualitatively similar to others in the literature. It has a low runtime $O(mp2)$ in the case of $m$ samples and $p$ nodes, and can trivially be implemented in an online manner.
  arXiv  Detail & Related papers  (2020-02-20T10:50:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.