Constrained Code Generation with Discrete Diffusion

• URL: http://arxiv.org/abs/2605.16829v1
• Date: Sat, 16 May 2026 06:15:47 GMT
• Title: Constrained Code Generation with Discrete Diffusion
• Abstract summary: Constrained Diffusion for Code (CDC) is a training-free neurosymbolic inference framework for code generation.<n>CDC integrates constraint satisfaction directly into the reverse denoising process.<n>CDC consistently improves constraint satisfaction in functional correctness, security, and even syntax.
• Score: 40.46661913038948
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Discrete diffusion models are a powerful, emerging paradigm for code generation. They construct programs through iterative refinement of partially corrupted token sequences and enable parallel token refinement. Importantly, this paradigm exposes a global program state at each denoising step, which provides a natural intervention point for enforcing program-level functionality and security constraints, guiding the generation before the final code is committed. Building on this observation, the paper introduces Constrained Diffusion for Code (CDC), a training-free neurosymbolic inference framework that integrates constraint satisfaction directly into the reverse denoising process. CDC augments the base discrete diffusion sampler with constraint-aware denoising operators that combine mathematical optimization with program analysis to identify constraint-relevant regions of the intermediate program state and locally adjust the denoising trajectory, steering generation toward feasible programs while remaining close to the base model. Across code generation benchmarks, CDC consistently improves constraint satisfaction in functional correctness, security, and even syntax, outperforming discrete diffusion and autoregressive baselines with less corrective computation and more localized edits.

Related papers

• DiRecT: Safe Diffusion-Based Planning via Receding-Horizon Denoising [4.249024052507976]
  We introduce DiRecT, a training-free algorithm for constrained sampling from diffusion models via optimal control (SOC)<n>Inspired by model predictive control, we derive a receding-horizon surrogate for the otherwise intractable constrained SOC formulation.<n>Experiments demonstrate that DiRecT substantially improves deployment safety and task performance over existing diffusion-based planning baselines.
  arXiv  Detail & Related papers  (2026-06-13T15:41:26Z)
• Primal-Dual Guided Decoding for Constrained Discrete Diffusion [5.219025684249083]
  We propose primal-dual guided decoding, an inference-time method that formulates constrained generation as a KL-regularised optimisation problem.<n>At each denoising step, the method modifies token logits through an additive, constraint-dependent bias, with multipliers updated by mirror descent based on constraint violation.<n>We evaluate our approach on topical text generation, molecular design, and music playlist generation, showing that a single algorithm instantiated via domain-specific scoring functions improves constraint satisfaction while preserving relevant domain-specific quality metrics.
  arXiv  Detail & Related papers  (2026-05-10T20:55:19Z)
• Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models [54.16797570104461]
  Diffusion-based language models (dLLMs) have emerged as a promising alternative to autoregressive language models.<n>We investigate non-autoregressive decoding in dLLMs by systematically analyzing its inference dynamics along the temporal axis.
  arXiv  Detail & Related papers  (2026-04-12T10:26:41Z)
• Reinforcement Learning for Diffusion LLMs with Entropy-Guided Step Selection and Stepwise Advantages [7.728607011076579]
  Reinforcement learning has been effective for post-training autoregressive (AR) language models, but extending these methods to diffusion language models (DLMs) is challenging due to intractable sequence-level likelihoods.<n>We formulate a diffusion-based sequence generation as a finite-horizon Markov decision process over the denoising trajectory and derive an exact, unbiased policy gradient.<n>Experiments on coding and logical reasoning benchmarks demonstrate state-of-the-art results, with strong competitive performance on mathematical reasoning.
  arXiv  Detail & Related papers  (2026-03-13T01:38:44Z)
• Progressive Refinement Regulation for Accelerating Diffusion Language Model Decoding [18.99795632160924]
  We propose emphProgressive Refinement Regulation (PRR), a trajectory-grounded refinement control framework.<n>PRR learns a lightweight token-wise controller to regulate refinement via temperature-based distribution shaping.<n> Experiments show that PRR substantially accelerates diffusion language model decoding while preserving generation quality.
  arXiv  Detail & Related papers  (2026-03-04T19:04:22Z)
• Causal Autoregressive Diffusion Language Model [70.7353007255797]
  CARD reformulates the diffusion process within a strictly causal attention mask, enabling dense, per-token supervision in a single forward pass.<n>Our results demonstrate that CARD achieves ARM-level data efficiency while unlocking the latency benefits of parallel generation.
  arXiv  Detail & Related papers  (2026-01-29T17:38:29Z)
• READER: Retrieval-Assisted Drafter for Efficient LLM Inference [0.0386965802948046]
  Autoregressive Language Models instantiate a factorized likelihood over token sequences, yet their strictly sequential decoding process imposes an intrinsic lower bound on latency inference.<n>This bottleneck has emerged as a central obstacle to the scalable deployment of large-scale generative models.<n>We present READER, a speculative decoding framework that bypasses the training of the auxiliary draft model.
  arXiv  Detail & Related papers  (2025-08-12T16:47:48Z)
• Inference-Time Scaling of Diffusion Language Models with Particle Gibbs Sampling [70.8832906871441]
  We study how to steer generation toward desired rewards without retraining the models.<n>Prior methods typically resample or filter within a single denoising trajectory, optimizing rewards step-by-step without trajectory-level refinement.<n>We introduce particle Gibbs sampling for diffusion language models (PG-DLM), a novel inference-time algorithm enabling trajectory-level refinement while preserving generation perplexity.
  arXiv  Detail & Related papers  (2025-07-11T08:00:47Z)
• Constrained Discrete Diffusion [61.81569616239755]
  This paper introduces Constrained Discrete Diffusion (CDD), a novel integration of differentiable constraint optimization within the diffusion process.<n>CDD directly imposes constraints into the discrete diffusion sampling process, resulting in a training-free and effective approach.
  arXiv  Detail & Related papers  (2025-03-12T19:48:12Z)
• Denoising-Contrastive Alignment for Continuous Sign Language Recognition [22.800767994061175]
  Continuous sign language recognition aims to recognize signs in untrimmed sign language videos to textual glosses.<n>Current cross-modality alignment paradigms often neglect the role of textual grammar to guide the video representation.<n>We propose a Denoising-Contrastive Alignment paradigm to enhance video representations.
  arXiv  Detail & Related papers  (2023-05-05T15:20:27Z)
• Conditional Denoising Diffusion for Sequential Recommendation [62.127862728308045]
  Two prominent generative models, Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs) GANs suffer from unstable optimization, while VAEs are prone to posterior collapse and over-smoothed generations. We present a conditional denoising diffusion model, which includes a sequence encoder, a cross-attentive denoising decoder, and a step-wise diffuser.
  arXiv  Detail & Related papers  (2023-04-22T15:32:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.