Discrete Diffusion Inference-Time Control with Nested Sequential Monte Carlo

• URL: http://arxiv.org/abs/2608.20123v1
• Date: Thu, 20 Aug 2026 14:52:17 GMT
• Title: Discrete Diffusion Inference-Time Control with Nested Sequential Monte Carlo
• Abstract summary: We formulate nested SMC (NSMC) and fully-adapted nested SMC (FA-NSMC) for Feynman--Kac steering.<n>We evaluate these methods on toxicity and fluency steering tasks, showing that NSMC and FA-NSMC consistently outperform best-of-$n$ and bootstrap SMC.
• Score: 4.73459038844245
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study inference-time control for text generation in discrete diffusion language models, where the goal is to steer sampling toward sequence-level rewards without retraining. Prior work in this domain has focused on particle-based methods such as best-of-$n$ sampling and bootstrap sequential Monte Carlo, which may suffer from overoptimism and weight degeneracy, respectively. We address these limitations using \emph{nested} sequential Monte Carlo methods. We formulate nested SMC (NSMC) and fully-adapted nested SMC (FA-NSMC) for Feynman--Kac steering, identifying and correcting errors in prior formulations that lead to biased final estimates. We evaluate these methods on toxicity and fluency steering tasks, showing that NSMC and FA-NSMC consistently outperform best-of-$n$ and bootstrap SMC.

Related papers

• Self-Rewarding Sequential Monte Carlo for Masked Diffusion Language Models [58.946955321428845]
  This work presents self-rewarding sequential Monte Carlo (SMC)<n>Our algorithm stems from the observation that most existing MDLMs rely on a confidence-based sampling strategy.<n>We introduce the trajectory-level confidence as a self-rewarding signal for assigning particle importance weights.
  arXiv  Detail & Related papers  (2026-02-02T09:21:45Z)
• Reinforced sequential Monte Carlo for amortised sampling [49.92678178064033]
  We state a connection between sequential Monte Carlo (SMC) and neural sequential samplers trained by maximum-entropy reinforcement learning (MaxEnt RL)<n>We describe techniques for stable joint training of proposals and twist functions and an adaptive weight tempering scheme to reduce training signal variance.
  arXiv  Detail & Related papers  (2025-10-13T17:59:11Z)
• Inference-Time Scaling of Diffusion Language Models with Particle Gibbs Sampling [70.8832906871441]
  We study how to steer generation toward desired rewards without retraining the models.<n>Prior methods typically resample or filter within a single denoising trajectory, optimizing rewards step-by-step without trajectory-level refinement.<n>We introduce particle Gibbs sampling for diffusion language models (PG-DLM), a novel inference-time algorithm enabling trajectory-level refinement while preserving generation perplexity.
  arXiv  Detail & Related papers  (2025-07-11T08:00:47Z)
• Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensembles [3.4661537979254655]
  This work introduces a new method designed for Bayesian deep learning called scalable Bayesian Monte Carlo (SBMC)<n>The method is comprised of a model and an algorithm. The model interpolates between a point estimator and the posterior.<n>A systematic numerical study reveals that for the same wall-clock time as state-of-the-art (SOTA) methods like deep ensembles (DE), SBMC achieves comparable or better accuracy.
  arXiv  Detail & Related papers  (2025-05-19T17:55:32Z)
• Humble your Overconfident Networks: Unlearning Overfitting via Sequential Monte Carlo Tempered Deep Ensembles [3.2254941904559917]
  We introduce a scalable variant by incorporating Gradient Hamiltonian Monte Carlo proposals into Sequential Monte Carlo (SMC) methods.<n>Our resulting SMCSGHMC algorithm outperforms gradient descent ensembles across image classification, out-of-distribution detection, and transfer learning tasks.
  arXiv  Detail & Related papers  (2025-05-16T20:10:04Z)
• Tuning Sequential Monte Carlo Samplers via Greedy Incremental Divergence Minimization [33.836913624304444]
  We propose a general adaptation framework for tuning the Markov kernels in SMC samplers.<n>We provide a gradient- and tuning-free algorithm that is generally applicable for kernels such as Langevin Monte Carlo (LMC)<n>Our implementations are able to obtain a full schedule of tuned parameters at the cost of a few vanilla SMC runs.
  arXiv  Detail & Related papers  (2025-03-19T21:35:02Z)
• Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts [64.34482582690927]
  We provide an efficient and principled method for sampling from a sequence of annealed, geometric-averaged, or product distributions derived from pretrained score-based models.<n>We propose Sequential Monte Carlo (SMC) resampling algorithms that leverage inference-time scaling to improve sampling quality.
  arXiv  Detail & Related papers  (2025-03-04T17:46:51Z)
• Sequential Kalman Tuning of the $t$-preconditioned Crank-Nicolson algorithm: efficient, adaptive and gradient-free inference for Bayesian inverse problems [1.3654846342364308]
  We propose an adaptive implementation of EKI and Flow Annealed Kalman Inversion. EKI is only exact in the regime of Gaussian target measures and linear forward models. We show significant improvements in the rate of convergence compared to adaptation within standard SMC.
  arXiv  Detail & Related papers  (2024-07-10T15:56:30Z)
• Reverse Diffusion Monte Carlo [19.35592726471155]
  We propose a novel Monte Carlo sampling algorithm called reverse diffusion Monte Carlo (rdMC) rdMC is distinct from the Markov chain Monte Carlo (MCMC) methods.
  arXiv  Detail & Related papers  (2023-07-05T05:42:03Z)
• What Are Bayesian Neural Network Posteriors Really Like? [63.950151520585024]
  We show that Hamiltonian Monte Carlo can achieve significant performance gains over standard and deep ensembles. We also show that deep distributions are similarly close to HMC as standard SGLD, and closer than standard variational inference.
  arXiv  Detail & Related papers  (2021-04-29T15:38:46Z)
• Annealed Flow Transport Monte Carlo [91.20263039913912]
  Annealed Flow Transport (AFT) builds upon Annealed Importance Sampling (AIS) and Sequential Monte Carlo (SMC) AFT relies on NF which is learned sequentially to push particles towards the successive targets. We show that a continuous-time scaling limit of the population version of AFT is given by a Feynman--Kac measure.
  arXiv  Detail & Related papers  (2021-02-15T12:05:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.