Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures

• URL: http://arxiv.org/abs/2605.17850v1
• Date: Mon, 18 May 2026 04:45:33 GMT
• Title: Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures
• Abstract summary: textttURGE is a derivative-free inference-time scaling algorithm that performs path-wise importance reweighting via a change of measure.<n>No score, no Hessian, and no PDE evaluation is required. We establish an equivalence between path-wise and particle-wise SMC.
• Score: 13.41132148962756
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: iffusion-based generative models increasingly rely on inference-time guidance, adding a drift term or reweighting mixture of experts, to improve sample quality on task-specific objectives. However, most existing techniques require repeated score or gradient evaluations, introducing bias, high computational overhead, or both. We introduce \texttt{URGE}, Unbiased Resampling via Girsanov Estimation, a derivative-free inference-time scaling algorithm that performs path-wise importance reweighting via a Girsanov change of measure. Instead of computing gradient-based particle weights in previous work, \texttt{URGE} attaches a simple multiplicative weight to each simulated trajectory and periodically resamples. No score, no Hessian, and no PDE evaluation is required. We establish an equivalence between path-wise and particle-wise SMC: the Girsanov path weight admits a backward conditional expectation that recovers the previous particle-level weights, guaranteeing that both schemes produce the same unbiased terminal law. Empirically, \texttt{URGE} outperforms existing inference-time guidance baselines on synthetic tests and diffusion-model benchmarks, achieving better generation quality, while being significantly simpler to implement and fully gradient-free.

Related papers

• Causal Discovery in Equal Variance Linear Gaussian DAGs via SURE-Tuned Ridge Regression [18.884190660925444]
  SURE-Ridge is a non-iterative, closed-form estimator for equal variance linear Gaussian SEM.<n>It achieves the lowest structural Hamming distance in the small-sample regime and the lowest run time across all sample sizes tested.
  arXiv  Detail & Related papers  (2026-08-17T21:07:06Z)
• Generalization Error Estimation for Primal--Dual Algorithms in Non-Smooth Regression [6.462588783417218]
  We study trajectory-wise estimation of generalization error for primal--dual algorithms in non-smooth regression.<n>We develop a general framework that includes the Chambolle--Pock algorithm and related primal--dual splitting methods.
  arXiv  Detail & Related papers  (2026-08-14T01:45:11Z)
• LC-GRPO: Bridging Train-Inference Gap for Flow-Based GRPO with Langevin Correction [59.32834240526981]
  Flow-based generative models are typically sampled by solving a deterministic ordinary differential equation (ODE)<n>Existing GRPO methods for flow models replace the Euler-time ODE with aotropic differential equation (SDE) during training.<n>We introduce LC-GRPO, a flow-based GRPO framework with Langevin correction targeting the marginal distribution at the resulting timestep.
  arXiv  Detail & Related papers  (2026-08-06T04:47:57Z)
• $Ω$: Operator-based Mixture Ensemble for Generative Assimilation [7.915816961228985]
  $$ is a scalable framework that integrates conditional Gaussian surrogate modeling, unsupervised score learning, and generative sampling.<n>$$ exploits closed-form conditional posterior distributions to analytically recover the high-dimensional unobserved component.<n>$$ is validated on several turbulent models with intermittency and extreme events, consistently improving posterior accuracy.
  arXiv  Detail & Related papers  (2026-06-18T20:19:44Z)
• From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models [58.02657021435213]
  We propose a novel Gibbs-based corrector for discrete diffusion models, termed Gibbs-Accelerated Discrete Diffusion (GADD)<n>We show GADD achieves an overall sampling complexity of $mathcalO(mathrmpolylog (varepsilon-1)$, yielding the first such rate for diffusion-based samplers for uniform-rate discrete diffusion models.
  arXiv  Detail & Related papers  (2026-05-26T17:52:28Z)
• SURGE: Approximation-free Training Free Particle Filter for Diffusion Surrogate [10.190682562038047]
  Diffusion-based generative models increasingly rely on inference-time guidance, adding a drift term or reweighting mixture of experts.<n>We introduce textttURGE, a derivative-free inference-time scaling algorithm that performs path-wise importance reweighting via a change of measure.
  arXiv  Detail & Related papers  (2026-05-18T17:59:00Z)
• Temporal Pair Consistency for Variance-Reduced Flow Matching [13.328987133593154]
  Temporal Pair Consistency (TPC) is a lightweight variance-reduction principle that couples velocity predictions at paired timesteps along the same probability path.<n>Instantiated within flow matching, TPC improves sample quality and efficiency across CIFAR-10 and ImageNet at multiple resolutions.
  arXiv  Detail & Related papers  (2026-02-04T00:05:21Z)
• Stein-Rule Shrinkage for Stochastic Gradient Estimation in High Dimensions [0.0]
  gradient methods are central to large-scale learning, but they treat mini-batch gradients as unbiased estimators, which classical decision theory shows are inadmissible in high dimensions.<n>We introduce a framework based on Stein-rule shrinkage and construct a gradient estimator that adaptively contracts mini-batch gradients toward a stable estimator derived from historical momentum.<n> Empirical evaluations on CIFAR10 and CIFAR100 show consistent improvements over Adam in the large-batch regime.
  arXiv  Detail & Related papers  (2026-02-02T08:01:13Z)
• From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models [90.45197506653341]
  Large reasoning models generate intermediate reasoning traces before producing final answers.<n> aligning LRMs with human preferences, a crucial prerequisite for model deployment, remains underexplored.<n>A common workaround optimized a single sampled trajectory, which introduces substantial gradient variance from trace sampling.
  arXiv  Detail & Related papers  (2025-10-06T17:58:01Z)
• Inference-Time Scaling of Diffusion Language Models with Particle Gibbs Sampling [70.8832906871441]
  We study how to steer generation toward desired rewards without retraining the models.<n>Prior methods typically resample or filter within a single denoising trajectory, optimizing rewards step-by-step without trajectory-level refinement.<n>We introduce particle Gibbs sampling for diffusion language models (PG-DLM), a novel inference-time algorithm enabling trajectory-level refinement while preserving generation perplexity.
  arXiv  Detail & Related papers  (2025-07-11T08:00:47Z)
• Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts [64.34482582690927]
  We provide an efficient and principled method for sampling from a sequence of annealed, geometric-averaged, or product distributions derived from pretrained score-based models.<n>We propose Sequential Monte Carlo (SMC) resampling algorithms that leverage inference-time scaling to improve sampling quality.
  arXiv  Detail & Related papers  (2025-03-04T17:46:51Z)
• Low-Rank Tensor Completion via Novel Sparsity-Inducing Regularizers [30.920908325825668]
  To alleviate l1-norm in the low-rank tensor completion problem, non-rank surrogates/regularizers have been suggested. These regularizers are applied to nuclear-rank restoration, and efficient algorithms based on the method of multipliers are proposed.
  arXiv  Detail & Related papers  (2023-10-10T01:00:13Z)
• Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent [43.097493761380186]
  gradient algorithms are an efficient method of approximately solving linear systems. We show that gradient descent produces accurate predictions, even in cases where it does not converge quickly to the optimum. Experimentally, gradient descent achieves state-of-the-art performance on sufficiently large-scale or ill-conditioned regression tasks.
  arXiv  Detail & Related papers  (2023-06-20T15:07:37Z)
• Path Sample-Analytic Gradient Estimators for Stochastic Binary Networks [78.76880041670904]
  In neural networks with binary activations and or binary weights the training by gradient descent is complicated. We propose a new method for this estimation problem combining sampling and analytic approximation steps. We experimentally show higher accuracy in gradient estimation and demonstrate a more stable and better performing training in deep convolutional models.
  arXiv  Detail & Related papers  (2020-06-04T21:51:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.