Scalable Importance Sampling in High Dimensions with Low-Rank Mixture Proposals

• URL: http://arxiv.org/abs/2505.13335v1
• Date: Mon, 19 May 2025 16:44:48 GMT
• Title: Scalable Importance Sampling in High Dimensions with Low-Rank Mixture Proposals
• Authors: Liam A. Kruse, Marc R. Schlichting, Mykel J. Kochenderfer,
• Abstract summary: Importance sampling is a Monte Carlo technique for efficiently estimating the likelihood of rare events.<n>We propose using mixtures of probabilistic principal component analyzers (MPPCA) as the parametric proposal density for importance sampling methods.<n>We validate our method on three simulated systems, demonstrating consistent gains in sample efficiency and quality of failure distribution characterization.
• Score: 37.634056981112444
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Importance sampling is a Monte Carlo technique for efficiently estimating the likelihood of rare events by biasing the sampling distribution towards the rare event of interest. By drawing weighted samples from a learned proposal distribution, importance sampling allows for more sample-efficient estimation of rare events or tails of distributions. A common choice of proposal density is a Gaussian mixture model (GMM). However, estimating full-rank GMM covariance matrices in high dimensions is a challenging task due to numerical instabilities. In this work, we propose using mixtures of probabilistic principal component analyzers (MPPCA) as the parametric proposal density for importance sampling methods. MPPCA models are a type of low-rank mixture model that can be fit quickly using expectation-maximization, even in high-dimensional spaces. We validate our method on three simulated systems, demonstrating consistent gains in sample efficiency and quality of failure distribution characterization.

Related papers

• Enhanced Importance Sampling through Latent Space Exploration in Normalizing Flows [69.8873421870522]
  importance sampling is a rare event simulation technique used in Monte Carlo simulations.<n>We propose a method for more efficient sampling by updating the proposal distribution in the latent space of a normalizing flow.
  arXiv  Detail & Related papers  (2025-01-06T21:18:02Z)
• Iterated Denoising Energy Matching for Sampling from Boltzmann Densities [109.23137009609519]
  Iterated Denoising Energy Matching (iDEM) iDEM alternates between (I) sampling regions of high model density from a diffusion-based sampler and (II) using these samples in our matching objective. We show that the proposed approach achieves state-of-the-art performance on all metrics and trains $2-5times$ faster.
  arXiv  Detail & Related papers  (2024-02-09T01:11:23Z)
• PQMass: Probabilistic Assessment of the Quality of Generative Models using Probability Mass Estimation [7.143427689586699]
  We propose a likelihood-free method for comparing two distributions given samples from each.<n>PQMass divides the sample space into non-overlapping regions and applies chi-squared tests to the number of data samples that fall within each region.<n>We show that PQMass scales well to moderately high-dimensional data and thus obviates the need for feature extraction in practical applications.
  arXiv  Detail & Related papers  (2024-02-06T19:39:26Z)
• Variational autoencoder with weighted samples for high-dimensional non-parametric adaptive importance sampling [0.0]
  We extend the existing framework to the case of weighted samples by introducing a new objective function. In order to add flexibility to the model and to be able to learn multimodal distributions, we consider a learnable prior distribution. We exploit the proposed procedure in existing adaptive importance sampling algorithms to draw points from a target distribution and to estimate a rare event probability in high dimension.
  arXiv  Detail & Related papers  (2023-10-13T15:40:55Z)
• A Flow-Based Generative Model for Rare-Event Simulation [0.483420384410068]
  We present a method in which a Normalizing Flow generative model is trained to simulate samples directly from a conditional distribution. We illustrate that by simulating directly from a rare-event distribution significant insight can be gained into the way rare events happen.
  arXiv  Detail & Related papers  (2023-05-13T08:25:57Z)
• Optimization of Annealed Importance Sampling Hyperparameters [77.34726150561087]
  Annealed Importance Sampling (AIS) is a popular algorithm used to estimates the intractable marginal likelihood of deep generative models. We present a parameteric AIS process with flexible intermediary distributions and optimize the bridging distributions to use fewer number of steps for sampling. We assess the performance of our optimized AIS for marginal likelihood estimation of deep generative models and compare it to other estimators.
  arXiv  Detail & Related papers  (2022-09-27T07:58:25Z)
• Sampling from Discrete Energy-Based Models with Quality/Efficiency Trade-offs [3.491202838583993]
  Energy-Based Models (EBMs) allow for extremely flexible specifications of probability distributions. They do not provide a mechanism for obtaining exact samples from these distributions. We propose a new approximate sampling technique, Quasi Rejection Sampling (QRS), that allows for a trade-off between sampling efficiency and sampling quality.
  arXiv  Detail & Related papers  (2021-12-10T17:51:37Z)
• Sampling from Arbitrary Functions via PSD Models [55.41644538483948]
  We take a two-step approach by first modeling the probability distribution and then sampling from that model. We show that these models can approximate a large class of densities concisely using few evaluations, and present a simple algorithm to effectively sample from these models.
  arXiv  Detail & Related papers  (2021-10-20T12:25:22Z)
• Unrolling Particles: Unsupervised Learning of Sampling Distributions [102.72972137287728]
  Particle filtering is used to compute good nonlinear estimates of complex systems. We show in simulations that the resulting particle filter yields good estimates in a wide range of scenarios.
  arXiv  Detail & Related papers  (2021-10-06T16:58:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.