Interpolating Discrete Diffusion Models with Controllable Resampling

• URL: http://arxiv.org/abs/2604.17310v1
• Date: Sun, 19 Apr 2026 07:55:57 GMT
• Title: Interpolating Discrete Diffusion Models with Controllable Resampling
• Abstract summary: Masked diffusion models suffer from irreversible errors due to early unmasking.<n>Uniform diffusion models, despite enabling self-correction, often yield low-quality samples.<n>We introduce Interpolating Discrete Diffusion Model, that improves diffusion by reducing dependence on intermediate latent states.
• Score: 49.771004594828405
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Discrete diffusion models form a powerful class of generative models across diverse domains, including text and graphs. However, existing approaches face fundamental limitations. Masked diffusion models suffer from irreversible errors due to early unmasking, while uniform diffusion models, despite enabling self-correction, often yield low-quality samples due to their strong reliance on intermediate latent states. We introduce IDDM, an Interpolating Discrete Diffusion Model, that improves diffusion by reducing dependence on intermediate latent states. Central to IDDM is a controllable resampling mechanism that partially resets probability mass to the marginal distribution, mitigating error accumulation and enabling more effective token corrections. IDDM specifies a generative process whose transitions interpolate between staying at the current state, resampling from a prior, and flipping toward the target state, while enforcing marginal consistency and fully decoupling training from inference. We benchmark our model against state-of-the-art discrete diffusion models across molecular graph generation as well as text generation tasks, demonstrating competitive performance.

Related papers

• Bridging the Discrete-Continuous Gap: Unified Multimodal Generation via Coupled Manifold Discrete Absorbing Diffusion [60.186310080523135]
  Bifurcation of generative modeling into autoregressive approaches for discrete data (text) and diffusion approaches for continuous data (images) hinders development of truly unified multimodal systems.<n>We propose textbfCoM-DAD, a novel probabilistic framework that reformulates multimodal generation as a hierarchical dual-process.<n>Our method demonstrates superior stability over standard masked modeling, establishing a new paradigm for scalable, unified text-image generation.
  arXiv  Detail & Related papers  (2026-01-07T16:21:19Z)
• Authentic Discrete Diffusion Model [72.31371542619121]
  Authentic Discrete Diffusion (ADD) framework redefines prior pseudo-discrete approaches.<n>ADD reformulates the diffusion input by directly using float-encoded one-hot class data.<n> experiments demonstrate that ADD achieves superior performance on classification tasks compared to the baseline.
  arXiv  Detail & Related papers  (2025-10-01T15:51:10Z)
• Simple and Critical Iterative Denoising: A Recasting of Discrete Diffusion in Graph Generation [0.0]
  dependencies between intermediate noisy states lead to error accumulation and propagation during the reverse denoising process.<n>We propose a novel framework called Simple Iterative Denoising, which simplifies discrete diffusion and circumvents the issue.<n>Our empirical evaluations demonstrate that the proposed method significantly outperforms existing discrete diffusion baselines in graph generation tasks.
  arXiv  Detail & Related papers  (2025-03-27T15:08:58Z)
• Constrained Discrete Diffusion [61.81569616239755]
  This paper introduces Constrained Discrete Diffusion (CDD), a novel integration of differentiable constraint optimization within the diffusion process.<n>CDD directly imposes constraints into the discrete diffusion sampling process, resulting in a training-free and effective approach.
  arXiv  Detail & Related papers  (2025-03-12T19:48:12Z)
• Generalized Interpolating Discrete Diffusion [65.74168524007484]
  Masked diffusion is a popular choice due to its simplicity and effectiveness.<n>We generalize a new family of general interpolating discrete diffusion (GIDD) which offers greater flexibility in the design of the noising processes.<n>Exploiting GIDD's flexibility, we explore a hybrid approach combining masking and uniform noise, leading to improved sample quality.
  arXiv  Detail & Related papers  (2025-03-06T14:30:55Z)
• Remasking Discrete Diffusion Models with Inference-Time Scaling [21.362017006523086]
  We introduce the remasking diffusion model (ReMDM) sampler, a method that can be applied to pretrained masked diffusion models in a principled way.<n>Most interestingly, ReMDM endows discrete diffusion with a form of inference-time compute scaling.
  arXiv  Detail & Related papers  (2025-03-01T02:37:51Z)
• Continuous Diffusion Model for Language Modeling [64.7425225935854]
  Existing continuous diffusion models for discrete data underperform compared to discrete methods.<n>We propose a continuous diffusion model for language modeling that incorporates the geometry of the underlying categorical distribution.<n>Our method outperforms existing discrete diffusion models and approaches the performance of autoregressive models.
  arXiv  Detail & Related papers  (2025-02-17T08:54:29Z)
• Simple Guidance Mechanisms for Discrete Diffusion Models [44.377206440698586]
  We develop a new class of diffusion models that leverage uniform noise and that are more guidable because they can continuously edit their outputs.<n>We improve the quality of these models with a novel continuous-time variational lower bound that yields state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-12-13T15:08:30Z)
• Constrained Diffusion Models via Dual Training [80.03953599062365]
  Diffusion processes are prone to generating samples that reflect biases in a training dataset. We develop constrained diffusion models by imposing diffusion constraints based on desired distributions. We show that our constrained diffusion models generate new data from a mixture data distribution that achieves the optimal trade-off among objective and constraints.
  arXiv  Detail & Related papers  (2024-08-27T14:25:42Z)
• Neural Diffusion Models [2.1779479916071067]
  We present a generalization of conventional diffusion models that enables defining and learning time-dependent non-linear transformations of data. NDMs outperform conventional diffusion models in terms of likelihood and produce high-quality samples.
  arXiv  Detail & Related papers  (2023-10-12T13:54:55Z)
• On Error Propagation of Diffusion Models [77.91480554418048]
  We develop a theoretical framework to mathematically formulate error propagation in the architecture of DMs. We apply the cumulative error as a regularization term to reduce error propagation. Our proposed regularization reduces error propagation, significantly improves vanilla DMs, and outperforms previous baselines.
  arXiv  Detail & Related papers  (2023-08-09T15:31:17Z)
• Information-Theoretic Diffusion [18.356162596599436]
  Denoising diffusion models have spurred significant gains in density modeling and image generation. We introduce a new mathematical foundation for diffusion models inspired by classic results in information theory.
  arXiv  Detail & Related papers  (2023-02-07T23:03:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.