Liouville Flow Importance Sampler

• URL: http://arxiv.org/abs/2405.06672v2
• Date: Mon, 10 Jun 2024 00:08:07 GMT
• Title: Liouville Flow Importance Sampler
• Authors: Yifeng Tian, Nishant Panda, Yen Ting Lin,
• Abstract summary: We present the Liouville Flow Importance Sampler (LFIS), an innovative flow-based model for generating samples from unnormalized density functions. LFIS learns a time-dependent velocity field that deterministically transports samples from a simple initial distribution to a complex target distribution. We demonstrate the effectiveness of LFIS through its application to a range of benchmark problems, on many of which LFIS achieved state-of-the-art performance.
• Score: 2.3603292593876324
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present the Liouville Flow Importance Sampler (LFIS), an innovative flow-based model for generating samples from unnormalized density functions. LFIS learns a time-dependent velocity field that deterministically transports samples from a simple initial distribution to a complex target distribution, guided by a prescribed path of annealed distributions. The training of LFIS utilizes a unique method that enforces the structure of a derived partial differential equation to neural networks modeling velocity fields. By considering the neural velocity field as an importance sampler, sample weights can be computed through accumulating errors along the sample trajectories driven by neural velocity fields, ensuring unbiased and consistent estimation of statistical quantities. We demonstrate the effectiveness of LFIS through its application to a range of benchmark problems, on many of which LFIS achieved state-of-the-art performance.

Related papers

• Neural Flow Samplers with Shortcut Models [19.81513273510523]
  Flow-based samplers generate samples by learning a velocity field that satisfies the continuity equation. While importance sampling provides an approximation, it suffers from high variance.
  arXiv  Detail & Related papers  (2025-02-11T07:55:41Z)
• Accelerated Diffusion Models via Speculative Sampling [89.43940130493233]
  Speculative sampling is a popular technique for accelerating inference in Large Language Models. We extend speculative sampling to diffusion models, which generate samples via continuous, vector-valued Markov chains. We propose various drafting strategies, including a simple and effective approach that does not require training a draft model.
  arXiv  Detail & Related papers  (2025-01-09T16:50:16Z)
• Enhanced Importance Sampling through Latent Space Exploration in Normalizing Flows [69.8873421870522]
  importance sampling is a rare event simulation technique used in Monte Carlo simulations. We propose a method for more efficient sampling by updating the proposal distribution in the latent space of a normalizing flow.
  arXiv  Detail & Related papers  (2025-01-06T21:18:02Z)
• NETS: A Non-Equilibrium Transport Sampler [15.58993313831079]
  We propose an algorithm, termed the Non-Equilibrium Transport Sampler (NETS) NETS can be viewed as a variant of importance sampling (AIS) based on Jarzynski's equality. We show that this drift is the minimizer of a variety of objective functions, which can all be estimated in an unbiased fashion.
  arXiv  Detail & Related papers  (2024-10-03T17:35:38Z)
• Importance Corrected Neural JKO Sampling [0.0]
  We combine continuous normalizing flows (CNFs) with rejection-resampling steps based on importance weights. The arising model can be trained iteratively, reduces the reverse Kulback-Leibler (KL) loss function in each step and allows to generate iid samples. Numerical examples show that our method yields accurate results on various test distributions including high-dimensional multimodal targets.
  arXiv  Detail & Related papers  (2024-07-29T22:49:59Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation [53.27596811146316]
  Diffusion models operate over a sequence of timesteps instead of instantaneous input-output relationships in previous contexts. We present Diffusion-TracIn that incorporates this temporal dynamics and observe that samples' loss gradient norms are highly dependent on timestep. We introduce Diffusion-ReTrac as a re-normalized adaptation that enables the retrieval of training samples more targeted to the test sample of interest.
  arXiv  Detail & Related papers  (2024-01-17T07:58:18Z)
• Learning Optimal Flows for Non-Equilibrium Importance Sampling [13.469239537683299]
  We develop a method to perform calculations based on generating samples from a simple base distribution, transporting them along the flow generated by a velocity field, and performing averages along these flowlines. On the theory side we discuss how to tailor the velocity field to the target and establish general conditions under which the proposed estimator is a perfect estimator. On the computational side we show how to use deep learning to represent the velocity field by a neural network and train it towards the zero variance optimum.
  arXiv  Detail & Related papers  (2022-06-20T17:25:26Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Unrolling Particles: Unsupervised Learning of Sampling Distributions [102.72972137287728]
  Particle filtering is used to compute good nonlinear estimates of complex systems. We show in simulations that the resulting particle filter yields good estimates in a wide range of scenarios.
  arXiv  Detail & Related papers  (2021-10-06T16:58:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.