How to Approximate Inference with Subtractive Mixture Models

• URL: http://arxiv.org/abs/2604.16714v1
• Date: Fri, 17 Apr 2026 21:28:24 GMT
• Title: How to Approximate Inference with Subtractive Mixture Models
• Abstract summary: We study how to effectively use subtractive mixture models (SMMs) for approximate inference settings such as variational inference (VI) and importance sampling (IS)<n>We discuss the additional challenges in estimation stability and learning efficiency that they carry and propose ways to overcome them.
• Score: 35.41848896315461
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Classical mixture models (MMs) are widely used tractable proposals for approximate inference settings such as variational inference (VI) and importance sampling (IS). Recently, mixture models with negative coefficients, called subtractive mixture models (SMMs), have been proposed as a potentially more expressive alternative. However, how to effectively use SMMs for VI and IS is still an open question as they do not provide latent variable semantics and therefore cannot use sampling schemes for classical MMs. In this work, we study how to circumvent this issue by designing several expectation estimators for IS and learning schemes for VI with SMMs, and we empirically evaluate them for distribution approximation. Finally, we discuss the additional challenges in estimation stability and learning efficiency that they carry and propose ways to overcome them. Code is available at: https://github.com/april-tools/delta-vi.

Related papers

• Variational Inference for Latent Variable Models in High Dimensions [4.3012765978447565]
  We introduce a general framework for quantifying the statistical accuracy of mean-field variational inference (MFVI)<n>We capture the exact regime where MFVI 'works' for the celebrated latent Dirichlet allocation model.<n>Our proof techniques, which extend the framework of nonlinear large deviations, open the door for the analysis of MFVI in other latent variable models.
  arXiv  Detail & Related papers  (2025-06-02T17:19:58Z)
• Scalable Expectation Estimation with Subtractive Mixture Models [11.150301369804664]
  negative mixture models (SMMs) complicate sampling, requiring costly auto-regressive techniques or accept-reject algorithms that do not scale in high dimensions.<n>In this work, we use the difference representation of SMMs to construct an unbiased IS estimator ($DeltatextEx$) that removes the need to sample from the SMM.<n>In our experiments, we show that $DeltatextEx$ can achieve comparable estimation quality to auto-regressive sampling while being considerably faster in MC estimation.
  arXiv  Detail & Related papers  (2025-03-27T10:25:03Z)
• Recursive Learning of Asymptotic Variational Objectives [49.69399307452126]
  General state-space models (SSMs) are widely used in statistical machine learning and are among the most classical generative models for sequential time-series data. Online sequential IWAE (OSIWAE) allows for online learning of both model parameters and a Markovian recognition model for inferring latent states. This approach is more theoretically well-founded than recently proposed online variational SMC methods.
  arXiv  Detail & Related papers  (2024-11-04T16:12:37Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• Adversarial Schrödinger Bridge Matching [66.39774923893103]
  Iterative Markovian Fitting (IMF) procedure alternates between Markovian and reciprocal projections of continuous-time processes. We propose a novel Discrete-time IMF (D-IMF) procedure in which learning of processes is replaced by learning just a few transition probabilities in discrete time. We show that our D-IMF procedure can provide the same quality of unpaired domain translation as the IMF, using only several generation steps instead of hundreds.
  arXiv  Detail & Related papers  (2024-05-23T11:29:33Z)
• Fast Semisupervised Unmixing Using Nonconvex Optimization [80.11512905623417]
  We introduce a novel convex convex model for semi/library-based unmixing. We demonstrate the efficacy of Alternating Methods of sparse unsupervised unmixing.
  arXiv  Detail & Related papers  (2024-01-23T10:07:41Z)
• Mixture of Coupled HMMs for Robust Modeling of Multivariate Healthcare Time Series [7.5986411724707095]
  We propose a novel class of models, a mixture of coupled hidden Markov models (M-CHMM) To make the model learning feasible, we derive two algorithms to sample the sequences of the latent variables in the CHMM. Compared to existing inference methods, our algorithms are computationally tractable, improve mixing, and allow for likelihood estimation.
  arXiv  Detail & Related papers  (2023-11-14T02:55:37Z)
• Diffusion models for probabilistic programming [56.47577824219207]
  Diffusion Model Variational Inference (DMVI) is a novel method for automated approximate inference in probabilistic programming languages (PPLs) DMVI is easy to implement, allows hassle-free inference in PPLs without the drawbacks of, e.g., variational inference using normalizing flows, and does not make any constraints on the underlying neural network model.
  arXiv  Detail & Related papers  (2023-11-01T12:17:05Z)
• Fitting Sparse Markov Models to Categorical Time Series Using Convex Clustering [0.0]
  A parsimonious modeling approach is given by Sparse Markov Models (SMMs)<n>We develop an elegant method of fitting SMMs based on convex clustering and regularization.
  arXiv  Detail & Related papers  (2022-02-11T07:27:16Z)
• Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions [24.389388509299543]
  Implicit Maximum Likelihood Estimation is a framework for end-to-end learning of models combining discrete exponential family distributions and differentiable neural components. We show that I-MLE is competitive with and often outperforms existing approaches which rely on problem-specific relaxations.
  arXiv  Detail & Related papers  (2021-06-03T12:42:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.