Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correction

• URL: http://arxiv.org/abs/2606.06303v1
• Date: Thu, 04 Jun 2026 15:41:53 GMT
• Title: Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correction
• Abstract summary: Controllable generation with discrete diffusion models is often hindered by high computational overhead or the need for retraining.<n>We present underlinetextbfGradient-underlinetextbfInformed underlinetextbfCorrection (textbfGILC), a plug-and-play framework that efficiently estimates guidance signals.
• Score: 7.637257579386074
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Controllable generation with discrete diffusion models is often hindered by high computational overhead or the need for retraining. In this paper, we present \underline{\textbf{G}}radient-\underline{\textbf{I}}nformed \underline{\textbf{L}}ogit \underline{\textbf{C}}orrection (\textbf{GILC}), a plug-and-play framework that efficiently estimates guidance signals by repurposing the pretrained denoising network as a variational proxy. To circumvent the gradient instability inherent in high-dimensional discrete spaces, we introduce a Jacobian-free mechanism that directly corrects the clean prediction logits, facilitating stable and effective guidance. Our method accommodates both differentiable and non-differentiable reward functions. Extensive experiments across DNA, protein sequence, and molecular generation tasks demonstrate that GILC achieves state-of-the-art performance without additional training, frequently outperforming fine-tuning approaches.

Related papers

• Need We Teach Foundation Models What is a Generative Image? Gradient-Free Generative Artifact Detection via Analytic Spectral Adaptation [3.881211374324378]
  Adapting foundation models to detect generative artifacts via gradient-based updates compromises their intrinsic representations.<n>We propose a gradient-free methodology reframing detection from binary classification to an out-of-distribution (OOD) anomaly measurement problem.
  arXiv  Detail & Related papers  (2026-06-03T13:04:23Z)
• From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models [58.02657021435213]
  We propose a novel Gibbs-based corrector for discrete diffusion models, termed Gibbs-Accelerated Discrete Diffusion (GADD)<n>We show GADD achieves an overall sampling complexity of $mathcalO(mathrmpolylog (varepsilon-1)$, yielding the first such rate for diffusion-based samplers for uniform-rate discrete diffusion models.
  arXiv  Detail & Related papers  (2026-05-26T17:52:28Z)
• Probability-Conserving Flow Guidance [49.03107678763765]
  Adaptive Manifold Guidance (AdaMaG) improves realism, reduces hallucinations, and controlled desaturation in high-guidance.<n>Across image generation benchmarks, AdaMaG improves realism, reduces hallucinations, and controlled desaturation in high-guidance.
  arXiv  Detail & Related papers  (2026-05-19T16:34:01Z)
• Self-Distilled Trajectory-Aware Boltzmann Modeling: Bridging the Training-Inference Discrepancy in Diffusion Language Models [65.89572755202245]
  Diffusion Language Models (DLMs) offer stronger global awareness and highly parallel generation.<n>Post-training DLMs with standard Negative Evidence Lower Bound (NELBO)-based supervised fine-tuning remains inefficient.<n>We propose a self-distilled trajectory-based post-training framework that aligns training with the easy-to-hard structure of inference.
  arXiv  Detail & Related papers  (2026-05-12T09:39:06Z)
• Generalized Discrete Diffusion with Self-Correction [20.435906948993022]
  Self-correction is an effective technique for maintaining parallel sampling in discrete diffusion models with minimal performance degradation.<n>We propose a Self-Correcting Diffusion model to reformulate pretrained self-correction with explicit state transitions and learn directly in discrete time.<n>Our framework also simplifies the training noise schedule, eliminates a redundant remasking step, and relies exclusively on uniform transitions to learn self-correction.
  arXiv  Detail & Related papers  (2026-02-13T04:12:35Z)
• AERMANI-Diffusion: Regime-Conditioned Diffusion for Dynamics Learning in Aerial Manipulators [2.376629619475859]
  Aerial manipulators undergo rapid, configuration-dependent changes in inertial coupling forces and aerodynamic forces.<n>We propose a regime-conditioned diffusion framework that models the full distribution of residual forces using a conditional diffusion process and a lightweight temporal encoder.
  arXiv  Detail & Related papers  (2025-12-11T16:10:32Z)
• Enhancing Diffusion Model Guidance through Calibration and Regularization [9.22066257345387]
  This paper introduces two complementary contributions to address this issue.<n>First, we propose a differentiable calibration objective based on the Smooth Expected Error (Smooth ECE)<n>Second, we develop enhanced sampling guidance methods that operate on off-the-shelf classifiers without requiring retraining.
  arXiv  Detail & Related papers  (2025-11-08T04:23:42Z)
• TAG:Tangential Amplifying Guidance for Hallucination-Resistant Diffusion Sampling [53.61290359948953]
  Tangential Amplifying Guidance (TAG) operates solely on trajectory signals without modifying the underlying diffusion model.<n>We formalize this guidance process by leveraging a first-order Taylor expansion.<n> TAG is a plug-and-play, architecture-agnostic module that improves diffusion sampling fidelity with minimal computational addition.
  arXiv  Detail & Related papers  (2025-10-06T06:53:29Z)
• Training-Free Stein Diffusion Guidance: Posterior Correction for Sampling Beyond High-Density Regions [46.59494117137471]
  Training free diffusion guidance provides a flexible way to leverage off-the-shelf classifiers without additional training.<n>We introduce Stein Diffusion Guidance (SDG), a novel training-free framework grounded in a surrogate SOC objective.<n>Experiments on molecular low-density sampling tasks suggest that SDG consistently surpasses standard training-free guidance methods.
  arXiv  Detail & Related papers  (2025-07-07T21:14:27Z)
• Improved Immiscible Diffusion: Accelerate Diffusion Training by Reducing Its Miscibility [62.272571285823595]
  We show how immiscibility eases denoising and improves efficiency.<n>We propose a family of implementations including K-nearest neighbor (KNN) noise selection and image scaling to reduce miscibility.<n>This work establishes a potentially new direction for future research into high-efficiency diffusion training.
  arXiv  Detail & Related papers  (2025-05-24T05:38:35Z)
• Simple Guidance Mechanisms for Discrete Diffusion Models [44.377206440698586]
  We develop a new class of diffusion models that leverage uniform noise and that are more guidable because they can continuously edit their outputs.<n>We improve the quality of these models with a novel continuous-time variational lower bound that yields state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-12-13T15:08:30Z)
• Adaptive Federated Learning Over the Air [108.62635460744109]
  We propose a federated version of adaptive gradient methods, particularly AdaGrad and Adam, within the framework of over-the-air model training. Our analysis shows that the AdaGrad-based training algorithm converges to a stationary point at the rate of $mathcalO( ln(T) / T 1 - frac1alpha ).
  arXiv  Detail & Related papers  (2024-03-11T09:10:37Z)
• Characteristic Guidance: Non-linear Correction for Diffusion Model at Large Guidance Scale [0.0]
  We propose characteristic guidance, a guidance method that provides first-principle non-linear correction for classifier-free guidance. Experiments show that characteristic guidance enhances semantic characteristics of prompts and irregularities in image generation.
  arXiv  Detail & Related papers  (2023-12-11T02:40:40Z)
• Unsupervised Discovery of Interpretable Directions in h-space of Pre-trained Diffusion Models [63.1637853118899]
  We propose the first unsupervised and learning-based method to identify interpretable directions in h-space of pre-trained diffusion models. We employ a shift control module that works on h-space of pre-trained diffusion models to manipulate a sample into a shifted version of itself. By jointly optimizing them, the model will spontaneously discover disentangled and interpretable directions.
  arXiv  Detail & Related papers  (2023-10-15T18:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.