Neural Flow Samplers with Shortcut Models

• URL: http://arxiv.org/abs/2502.07337v2
• Date: Sun, 20 Jul 2025 09:34:31 GMT
• Title: Neural Flow Samplers with Shortcut Models
• Authors: Wuhao Chen, Zijing Ou, Yingzhen Li,
• Abstract summary: Continuous flow-based neural samplers offer a promising approach to generate samples from unnormalized densities.<n>We introduce an improved estimator for these challenging quantities, employing a velocity-driven Sequential Monte Carlo method.<n>Our proposed Neural Flow Shortcut Sampler empirically outperforms existing flow-based neural samplers on both synthetic datasets and complex n-body system targets.
• Score: 19.81513273510523
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sampling from unnormalized densities presents a fundamental challenge with wide-ranging applications, from posterior inference to molecular dynamics simulations. Continuous flow-based neural samplers offer a promising approach, learning a velocity field that satisfies key principles of marginal density evolution (e.g., the continuity equation) to generate samples. However, this learning procedure requires accurate estimation of intractable terms linked to the computationally challenging partition function, for which existing estimators often suffer from high variance or low accuracy. To overcome this, we introduce an improved estimator for these challenging quantities, employing a velocity-driven Sequential Monte Carlo method enhanced with control variates. Furthermore, we introduce a shortcut consistency model to boost the runtime efficiency of the flow-based neural sampler by minimizing its required sampling steps. Our proposed Neural Flow Shortcut Sampler empirically outperforms existing flow-based neural samplers on both synthetic datasets and complex n-body system targets.

Related papers

• Inference-Time Scaling of Diffusion Language Models with Particle Gibbs Sampling [62.640128548633946]
  We introduce a novel inference-time scaling approach based on particle Gibbs sampling for discrete diffusion models.<n>Our method consistently outperforms prior inference-time strategies on reward-guided text generation tasks.
  arXiv  Detail & Related papers  (2025-07-11T08:00:47Z)
• Quantizing Diffusion Models from a Sampling-Aware Perspective [43.95032520555463]
  We propose a sampling-aware quantization strategy, wherein a Mixed-Order Trajectory Alignment technique is devised.<n>Experiments on sparse-step fast sampling across multiple datasets demonstrate that our approach preserves the rapid convergence characteristics of high-speed samplers.
  arXiv  Detail & Related papers  (2025-05-04T20:50:44Z)
• Dimension-free Score Matching and Time Bootstrapping for Diffusion Models [11.743167854433306]
  Diffusion models generate samples by estimating the score function of the target distribution at various noise levels. In this work, we establish the first (nearly) dimension-free sample bounds complexity for learning these score functions. A key aspect of our analysis is the use of a single function approximator to jointly estimate scores across noise levels.
  arXiv  Detail & Related papers  (2025-02-14T18:32:22Z)
• No Trick, No Treat: Pursuits and Challenges Towards Simulation-free Training of Neural Samplers [41.867855070932706]
  We consider the sampling problem, where the aim is to draw samples from a distribution whose density is known only up to a normalization constant.<n>Recent breakthroughs in generative modeling to approximate a high-dimensional data distribution have sparked significant interest in developing neural network-based methods for this challenging problem.<n>We propose an elegant modification to previous methods, which allows simulation-free training with the help of a time-dependent normalizing flow.
  arXiv  Detail & Related papers  (2025-02-10T17:13:11Z)
• Self-Refining Diffusion Samplers: Enabling Parallelization via Parareal Iterations [53.180374639531145]
  Self-Refining Diffusion Samplers (SRDS) retain sample quality and can improve latency at the cost of additional parallel compute. We take inspiration from the Parareal algorithm, a popular numerical method for parallel-in-time integration of differential equations.
  arXiv  Detail & Related papers  (2024-12-11T11:08:09Z)
• Parallel simulation for sampling under isoperimetry and score-based diffusion models [56.39904484784127]
  As data size grows, reducing the iteration cost becomes an important goal.<n>Inspired by the success of the parallel simulation of the initial value problem in scientific computation, we propose parallel Picard methods for sampling tasks.<n>Our work highlights the potential advantages of simulation methods in scientific computation for dynamics-based sampling and diffusion models.
  arXiv  Detail & Related papers  (2024-12-10T11:50:46Z)
• A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models [14.859580045688487]
  A practical bottleneck of diffusion models is their sampling speed. We propose a novel framework capable of adaptively allocating compute required for the score estimation. We show that our method could significantly improve the sampling throughput of the diffusion models without compromising image quality.
  arXiv  Detail & Related papers  (2024-08-12T05:33:45Z)
• Score-based Generative Models with Adaptive Momentum [40.84399531998246]
  We propose an adaptive momentum sampling method to accelerate the transforming process. We show that our method can produce more faithful images/graphs in small sampling steps with 2 to 5 times speed up.
  arXiv  Detail & Related papers  (2024-05-22T15:20:27Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• Liouville Flow Importance Sampler [2.3603292593876324]
  We present the Liouville Flow Importance Sampler (LFIS), an innovative flow-based model for generating samples from unnormalized density functions. LFIS learns a time-dependent velocity field that deterministically transports samples from a simple initial distribution to a complex target distribution. We demonstrate the effectiveness of LFIS through its application to a range of benchmark problems, on many of which LFIS achieved state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-05-03T16:44:31Z)
• Stable generative modeling using Schrödinger bridges [0.22499166814992438]
  We propose a generative model combining Schr"odinger bridges and Langevin dynamics. Our framework can be naturally extended to generate conditional samples and to Bayesian inference problems.
  arXiv  Detail & Related papers  (2024-01-09T06:15:45Z)
• Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time [49.598085130313514]
  We propose discrete non-Markov diffusion models (DNDM), which naturally induce the predetermined transition time set.<n>This enables a training-free sampling algorithm that significantly reduces the number of function evaluations.<n>We study the transition from finite to infinite step sampling, offering new insights into bridging the gap between discrete and continuous-time processes.
  arXiv  Detail & Related papers  (2023-12-14T18:14:11Z)
• Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization [87.21285093582446]
  Diffusion Generative Flow Samplers (DGFS) is a sampling-based framework where the learning process can be tractably broken down into short partial trajectory segments. Our method takes inspiration from the theory developed for generative flow networks (GFlowNets)
  arXiv  Detail & Related papers  (2023-10-04T09:39:05Z)
• Boosting Fast and High-Quality Speech Synthesis with Linear Diffusion [85.54515118077825]
  This paper proposes a linear diffusion model (LinDiff) based on an ordinary differential equation to simultaneously reach fast inference and high sample quality. To reduce computational complexity, LinDiff employs a patch-based processing approach that partitions the input signal into small patches. Our model can synthesize speech of a quality comparable to that of autoregressive models with faster synthesis speed.
  arXiv  Detail & Related papers  (2023-06-09T07:02:43Z)
• Entropy-based Training Methods for Scalable Neural Implicit Sampler [15.978655106034113]
  Efficiently sampling from un-normalized target distributions is a fundamental problem in scientific computing and machine learning. In this paper, we propose an efficient and scalable neural implicit sampler that overcomes these limitations. Our sampler can generate large batches of samples with low computational costs by leveraging a neural transformation that directly maps easily sampled latent vectors to target samples.
  arXiv  Detail & Related papers  (2023-06-08T05:56:05Z)
• Adaptive Conditional Quantile Neural Processes [9.066817971329899]
  Conditional Quantile Neural Processes (CQNPs) are a new member of the neural processes family. We introduce an extension of quantile regression where the model learns to focus on estimating informative quantiles. Experiments with real and synthetic datasets demonstrate substantial improvements in predictive performance.
  arXiv  Detail & Related papers  (2023-05-30T06:19:19Z)
• Minimizing Trajectory Curvature of ODE-based Generative Models [45.89620603363946]
  Recent generative models, such as diffusion models, rectified flows, and flow matching, define a generative process as a time reversal of a fixed forward process. We present an efficient method of training the forward process to minimize the curvature of generative trajectories without any ODE/SDE simulation.
  arXiv  Detail & Related papers  (2023-01-27T21:52:03Z)
• Fast Sampling of Diffusion Models via Operator Learning [74.37531458470086]
  We use neural operators, an efficient method to solve the probability flow differential equations, to accelerate the sampling process of diffusion models. Compared to other fast sampling methods that have a sequential nature, we are the first to propose a parallel decoding method. We show our method achieves state-of-the-art FID of 3.78 for CIFAR-10 and 7.83 for ImageNet-64 in the one-model-evaluation setting.
  arXiv  Detail & Related papers  (2022-11-24T07:30:27Z)
• Unrolling Particles: Unsupervised Learning of Sampling Distributions [102.72972137287728]
  Particle filtering is used to compute good nonlinear estimates of complex systems. We show in simulations that the resulting particle filter yields good estimates in a wide range of scenarios.
  arXiv  Detail & Related papers  (2021-10-06T16:58:34Z)
• Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
  Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
  arXiv  Detail & Related papers  (2021-06-05T01:31:51Z)
• Efficiently Sampling Functions from Gaussian Process Posteriors [76.94808614373609]
  We propose an easy-to-use and general-purpose approach for fast posterior sampling. We demonstrate how decoupled sample paths accurately represent Gaussian process posteriors at a fraction of the usual cost.
  arXiv  Detail & Related papers  (2020-02-21T14:03:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.