On the Error-Correcting Effects of Stochasticity in Discrete Diffusion

• URL: http://arxiv.org/abs/2605.26582v1
• Date: Tue, 26 May 2026 05:58:46 GMT
• Title: On the Error-Correcting Effects of Stochasticity in Discrete Diffusion
• Abstract summary: Discrete diffusion models achieve strong performance in text and image generation, but their inference remains slow and must inherently balance sampling and sample quality.<n>We show that highly deterministic transitions converge rapidly but suffer from error accumulation, while more transitions converge more slowly yet can achieve higher final sample quality.<n>Motivated by this analysis, we propose emphDiscrete Churn and Restart Sampling (DCRS), a novel inference algorithm that injects controlledity by alternating between forward and reverse diffusion processes.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Discrete diffusion models achieve strong performance in text and image generation, but their inference remains slow and must inherently balance sampling efficiency and sample quality. In this work, we present a systematic study of how the \emph{degree of stochasticity} in Markov transitions governs the sampling tradeoff. We show that highly deterministic transitions converge rapidly but suffer from error accumulation, while more stochastic transitions converge more slowly yet can achieve higher final sample quality. Using an information-theoretic analysis, we identify the underlying mechanism as an error-correcting effect induced by \emph{redundant transitions} that symmetrically exchange mass between states, and show that these transitions can provably contract sampling errors. Motivated by this analysis, we propose \emph{Discrete Churn and Restart Sampling} (DCRS), a novel inference algorithm that injects controlled stochasticity by alternating between forward and reverse diffusion processes. Experiments on synthetic datasets and large-scale benchmarks show that DCRS improves the speed-quality tradeoff in the low number of function evaluations regime. On image datasets, DCRS achieves up to a $10\times$ reduction in sampling steps compared to standard samplers while maintaining competitive sample quality, whereas on language benchmarks, we observe more nuanced behavior depending on the corruption process and sampling procedure.

Related papers

• A Time-Reparameterized Cumulative Intensity Extrapolation Sampler for Discrete Flow Matching [25.54630601391996]
  TR-CIE aims to improve sampling quality when function evaluations are restricted.<n>We provide a theoretical analysis that bounds the local approximation error of cumulative intensities.<n>Experiments on synthetic tasks, text generation, and text-to-image benchmarks demonstrate that our method improves sampling quality under limited NFE.
  arXiv  Detail & Related papers  (2026-06-23T04:48:45Z)
• Precise: SDE-Consistent Stochastic Sampling for RL Post-Training of Flow-Matching Models [56.67321805551389]
  Reinforcement learning (RL) has become an effective way to improve prompt alignment and perceptual quality in diffusion and flow-matching generators.<n>The sampler, controlling the exploration behavior and denoising dynamics, is thus part of the policy.<n>We propose a new sampler that balances effective exploration with stability.
  arXiv  Detail & Related papers  (2026-05-22T11:37:22Z)
• Probability-Conserving Flow Guidance [49.03107678763765]
  Adaptive Manifold Guidance (AdaMaG) improves realism, reduces hallucinations, and controlled desaturation in high-guidance.<n>Across image generation benchmarks, AdaMaG improves realism, reduces hallucinations, and controlled desaturation in high-guidance.
  arXiv  Detail & Related papers  (2026-05-19T16:34:01Z)
• Training-Free Refinement of Flow Matching with Divergence-based Sampling [17.850801452423074]
  Flow Divergence Sampler (FDS) is a training-free framework that refines intermediate states before each solver step.<n>FDS exploits this signal to steer states toward less ambiguous regions.<n>As a plug-and-play framework compatible with standard solvers and off-the-shelf flow backbones, FDS consistently improves fidelity across various generation tasks.
  arXiv  Detail & Related papers  (2026-04-06T12:54:58Z)
• Coarse-Guided Visual Generation via Weighted h-Transform Sampling [12.670888779100972]
  Coarse-guided visual generation is essential for various real-world applications.<n>Recent training-free works propose to leverage pretrained diffusion models and incorporate guidance during the sampling process.<n>We propose a novel guided method by using the htransform, a tool that can constrain processes (e.g., sampling process) under desired conditions.
  arXiv  Detail & Related papers  (2026-03-12T15:26:19Z)
• Neural Flow Samplers with Shortcut Models [19.81513273510523]
  Continuous flow-based neural samplers offer a promising approach to generate samples from unnormalized densities.<n>We introduce an improved estimator for these challenging quantities, employing a velocity-driven Sequential Monte Carlo method.<n>Our proposed Neural Flow Shortcut Sampler empirically outperforms existing flow-based neural samplers on both synthetic datasets and complex n-body system targets.
  arXiv  Detail & Related papers  (2025-02-11T07:55:41Z)
• CCS: Controllable and Constrained Sampling with Diffusion Models via Initial Noise Perturbation [9.12693573953231]
  We first observe an interesting phenomenon: the relationship between the change of generation outputs and the scale of initial noise perturbation is highly linear through the diffusion ODE sampling.<n>We propose a novel Controllable and Constrained Sampling method (CCS) together with a new controller algorithm for diffusion models to sample with desired statistical properties.<n>Results show that our CCS method achieves more precisely controlled sampling while maintaining superior sample quality and diversity.
  arXiv  Detail & Related papers  (2025-02-07T05:30:48Z)
• Distributional Diffusion Models with Scoring Rules [83.38210785728994]
  Diffusion models generate high-quality synthetic data.<n> generating high-quality outputs requires many discretization steps.<n>We propose to accomplish sample generation by learning the posterior em distribution of clean data samples.
  arXiv  Detail & Related papers  (2025-02-04T16:59:03Z)
• Score-based Generative Models with Adaptive Momentum [40.84399531998246]
  We propose an adaptive momentum sampling method to accelerate the transforming process. We show that our method can produce more faithful images/graphs in small sampling steps with 2 to 5 times speed up.
  arXiv  Detail & Related papers  (2024-05-22T15:20:27Z)
• Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time [49.598085130313514]
  We propose discrete non-Markov diffusion models (DNDM), which naturally induce the predetermined transition time set.<n>This enables a training-free sampling algorithm that significantly reduces the number of function evaluations.<n>We study the transition from finite to infinite step sampling, offering new insights into bridging the gap between discrete and continuous-time processes.
  arXiv  Detail & Related papers  (2023-12-14T18:14:11Z)
• DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models [58.450152413700586]
  We introduce a soft absorbing state that facilitates the diffusion model in learning to reconstruct discrete mutations based on the underlying Gaussian space. We employ state-of-the-art ODE solvers within the continuous space to expedite the sampling process. Our proposed method effectively accelerates the training convergence by 4x and generates samples of similar quality 800x faster.
  arXiv  Detail & Related papers  (2023-10-09T15:29:10Z)
• Denoising Diffusion Implicit Models [117.03720513930335]
  We present denoising diffusion implicit models (DDIMs) for iterative implicit probabilistic models with the same training procedure as DDPMs. DDIMs can produce high quality samples $10 times$ to $50 times$ faster in terms of wall-clock time compared to DDPMs.
  arXiv  Detail & Related papers  (2020-10-06T06:15:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.