Enhancing Diffusion Model Guidance through Calibration and Regularization

• URL: http://arxiv.org/abs/2511.05844v2
• Date: Wed, 12 Nov 2025 01:20:35 GMT
• Title: Enhancing Diffusion Model Guidance through Calibration and Regularization
• Authors: Seyed Alireza Javid, Amirhossein Bagheri, Nuria González-Prelcic,
• Abstract summary: This paper introduces two complementary contributions to address this issue.<n>First, we propose a differentiable calibration objective based on the Smooth Expected Error (Smooth ECE)<n>Second, we develop enhanced sampling guidance methods that operate on off-the-shelf classifiers without requiring retraining.
• Score: 9.22066257345387
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Classifier-guided diffusion models have emerged as a powerful approach for conditional image generation, but they suffer from overconfident predictions during early denoising steps, causing the guidance gradient to vanish. This paper introduces two complementary contributions to address this issue. First, we propose a differentiable calibration objective based on the Smooth Expected Calibration Error (Smooth ECE), which improves classifier calibration with minimal fine-tuning and yields measurable improvements in Frechet Inception Distance (FID). Second, we develop enhanced sampling guidance methods that operate on off-the-shelf classifiers without requiring retraining. These include tilted sampling with batch-level reweighting, adaptive entropy-regularized sampling to preserve diversity, and a novel f-divergence-based sampling strategy that strengthens class-consistent guidance while maintaining mode coverage. Experiments on ImageNet 128x128 demonstrate that our divergence-regularized guidance achieves an FID of 2.13 using a ResNet-101 classifier, improving upon existing classifier-guided diffusion methods while requiring no diffusion model retraining. The results show that principled calibration and divergence-aware sampling provide practical and effective improvements for classifier-guided diffusion.

Related papers

• Learnable Chernoff Baselines for Inference-Time Alignment [64.81256817158851]
  We introduce Learnable Chernoff Baselines as a method for efficiently and approximately sampling from exponentially tilted kernels.<n>We establish total-variation guarantees to the ideal aligned model, and demonstrate in both continuous and discrete diffusion settings that LCB sampling closely matches ideal rejection sampling.
  arXiv  Detail & Related papers  (2026-02-08T00:09:40Z)
• Improving Classifier-Free Guidance of Flow Matching via Manifold Projection [3.6087998976768128]
  We provide a principled interpretation of CFG through the lens of optimization.<n>We reformulate the CFG sampling as a homotopy optimization with manifold constraint.<n>Our proposed methods are training-free and consistently refine generation fidelity, prompt alignment, and robustness to the guidance scale.
  arXiv  Detail & Related papers  (2026-01-29T15:49:31Z)
• Rethinking Refinement: Correcting Generative Bias without Noise Injection [7.28668585578288]
  Generative models, including diffusion and flow-based models, often exhibit systematic biases that degrade sample quality.<n>We show that effective bias correction can be achieved as a post-hoc procedure, without noise injection or multi-step resampling.
  arXiv  Detail & Related papers  (2026-01-29T02:34:08Z)
• Test-Time Anchoring for Discrete Diffusion Posterior Sampling [38.507644561076894]
  Posterior sampling is a challenging problem for pretrained discrete diffusion foundation models.<n>We introduce Anchored Posterior Sampling (APS) for masked diffusion foundation models.<n>Our approach achieves state-of-the-art performance among discrete diffusion samplers across linear and nonlinear inverse problems.
  arXiv  Detail & Related papers  (2025-10-02T17:58:37Z)
• Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations [67.35596444651037]
  Vision-language models (VLMs) exhibit remarkable zero-shot capabilities but struggle with distribution shifts in downstream tasks when labeled data is unavailable.<n>We propose a Reliable Test-time Adaptation (ReTA) method that enhances reliability from two perspectives.
  arXiv  Detail & Related papers  (2025-07-13T05:37:33Z)
• Inference-Time Scaling of Diffusion Language Models with Particle Gibbs Sampling [70.8832906871441]
  We study how to steer generation toward desired rewards without retraining the models.<n>Prior methods typically resample or filter within a single denoising trajectory, optimizing rewards step-by-step without trajectory-level refinement.<n>We introduce particle Gibbs sampling for diffusion language models (PG-DLM), a novel inference-time algorithm enabling trajectory-level refinement while preserving generation perplexity.
  arXiv  Detail & Related papers  (2025-07-11T08:00:47Z)
• Training-Free Stein Diffusion Guidance: Posterior Correction for Sampling Beyond High-Density Regions [46.59494117137471]
  Training free diffusion guidance provides a flexible way to leverage off-the-shelf classifiers without additional training.<n>We introduce Stein Diffusion Guidance (SDG), a novel training-free framework grounded in a surrogate SOC objective.<n>Experiments on molecular low-density sampling tasks suggest that SDG consistently surpasses standard training-free guidance methods.
  arXiv  Detail & Related papers  (2025-07-07T21:14:27Z)
• Solving Inverse Problems with FLAIR [68.87167940623318]
  We present FLAIR, a training-free variational framework that leverages flow-based generative models as prior for inverse problems.<n>Results on standard imaging benchmarks demonstrate that FLAIR consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.
  arXiv  Detail & Related papers  (2025-06-03T09:29:47Z)
• DIVE: Inverting Conditional Diffusion Models for Discriminative Tasks [79.50756148780928]
  This paper studies the problem of leveraging pretrained diffusion models for performing discriminative tasks.<n>We extend the discriminative capability of pretrained frozen generative diffusion models from the classification task to the more complex object detection task, by "inverting" a pretrained layout-to-image diffusion model.
  arXiv  Detail & Related papers  (2025-04-24T05:13:27Z)
• One-Step Diffusion Model for Image Motion-Deblurring [85.76149042561507]
  We propose a one-step diffusion model for deblurring (OSDD), a novel framework that reduces the denoising process to a single step.<n>To tackle fidelity loss in diffusion models, we introduce an enhanced variational autoencoder (eVAE), which improves structural restoration.<n>Our method achieves strong performance on both full and no-reference metrics.
  arXiv  Detail & Related papers  (2025-03-09T09:39:57Z)
• Informed Correctors for Discrete Diffusion Models [27.295990499157814]
  We propose a predictor-corrector sampling scheme for discrete diffusion models.<n>We show that our informed corrector consistently produces superior samples with fewer errors or improved FID scores.<n>Our results underscore the potential of informed correctors for fast and high-fidelity generation using discrete diffusion.
  arXiv  Detail & Related papers  (2024-07-30T23:29:29Z)
• Characteristic Guidance: Non-linear Correction for Diffusion Model at Large Guidance Scale [0.0]
  We propose characteristic guidance, a guidance method that provides first-principle non-linear correction for classifier-free guidance. Experiments show that characteristic guidance enhances semantic characteristics of prompts and irregularities in image generation.
  arXiv  Detail & Related papers  (2023-12-11T02:40:40Z)
• Bridging the Gap: Addressing Discrepancies in Diffusion Model Training for Classifier-Free Guidance [1.6804613362826175]
  Diffusion models have emerged as a pivotal advancement in generative models. In this paper we aim to underscore a discrepancy between conventional training methods and the desired conditional sampling behavior. We introduce an updated loss function that better aligns training objectives with sampling behaviors.
  arXiv  Detail & Related papers  (2023-11-02T02:03:12Z)
• (Certified!!) Adversarial Robustness for Free! [116.6052628829344]
  We certify 71% accuracy on ImageNet under adversarial perturbations constrained to be within a 2-norm of 0.5. We obtain these results using only pretrained diffusion models and image classifiers, without requiring any fine tuning or retraining of model parameters.
  arXiv  Detail & Related papers  (2022-06-21T17:27:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.