Noise Conditional Variational Score Distillation

• URL: http://arxiv.org/abs/2506.09416v1
• Date: Wed, 11 Jun 2025 06:01:39 GMT
• Title: Noise Conditional Variational Score Distillation
• Authors: Xinyu Peng, Ziyang Zheng, Yaoming Wang, Han Li, Nuowen Kan, Wenrui Dai, Chenglin Li, Junni Zou, Hongkai Xiong,
• Abstract summary: Noise Conditional Variational Score Distillation (NCVSD) is a novel method for distilling pretrained diffusion models into generative denoisers.<n>By integrating this insight into the Variational Score Distillation framework, we enable scalable learning of generative denoisers.
• Score: 60.38982038894823
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose Noise Conditional Variational Score Distillation (NCVSD), a novel method for distilling pretrained diffusion models into generative denoisers. We achieve this by revealing that the unconditional score function implicitly characterizes the score function of denoising posterior distributions. By integrating this insight into the Variational Score Distillation (VSD) framework, we enable scalable learning of generative denoisers capable of approximating samples from the denoising posterior distribution across a wide range of noise levels. The proposed generative denoisers exhibit desirable properties that allow fast generation while preserve the benefit of iterative refinement: (1) fast one-step generation through sampling from pure Gaussian noise at high noise levels; (2) improved sample quality by scaling the test-time compute with multi-step sampling; and (3) zero-shot probabilistic inference for flexible and controllable sampling. We evaluate NCVSD through extensive experiments, including class-conditional image generation and inverse problem solving. By scaling the test-time compute, our method outperforms teacher diffusion models and is on par with consistency models of larger sizes. Additionally, with significantly fewer NFEs than diffusion-based methods, we achieve record-breaking LPIPS on inverse problems.

Related papers

• CCS: Controllable and Constrained Sampling with Diffusion Models via Initial Noise Perturbation [9.12693573953231]
  We first observe an interesting phenomenon: the relationship between the change of generation outputs and the scale of initial noise perturbation is highly linear through the diffusion ODE sampling.<n>We propose a novel Controllable and Constrained Sampling method (CCS) together with a new controller algorithm for diffusion models to sample with desired statistical properties.<n>Results show that our CCS method achieves more precisely controlled sampling while maintaining superior sample quality and diversity.
  arXiv  Detail & Related papers  (2025-02-07T05:30:48Z)
• Arbitrary-steps Image Super-resolution via Diffusion Inversion [68.78628844966019]
  This study presents a new image super-resolution (SR) technique based on diffusion inversion, aiming at harnessing the rich image priors encapsulated in large pre-trained diffusion models to improve SR performance.<n>We design a Partial noise Prediction strategy to construct an intermediate state of the diffusion model, which serves as the starting sampling point.<n>Once trained, this noise predictor can be used to initialize the sampling process partially along the diffusion trajectory, generating the desirable high-resolution result.
  arXiv  Detail & Related papers  (2024-12-12T07:24:13Z)
• Enhancing Sample Generation of Diffusion Models using Noise Level Correction [9.014666170540304]
  We propose a novel method to enhance sample generation by aligning the estimated noise level with the true distance of noisy samples to the manifold.<n> Specifically, we introduce a noise level correction network, leveraging a pre-trained denoising network, to refine noise level estimates during the denoising process.<n> Experimental results demonstrate that our method significantly improves sample quality in both unconstrained and constrained generation scenarios.
  arXiv  Detail & Related papers  (2024-12-07T01:19:14Z)
• Your Diffusion Model is Secretly a Noise Classifier and Benefits from Contrastive Training [20.492630610281658]
  Diffusion models learn to denoise data and the trained denoiser is then used to generate new samples from the data distribution. We introduce a new self-supervised training objective that differentiates the levels of noise added to a sample. We show by diverse experiments that the proposed contrastive diffusion training is effective for both sequential and parallel settings.
  arXiv  Detail & Related papers  (2024-07-12T03:03:50Z)
• Score-based Generative Models with Adaptive Momentum [40.84399531998246]
  We propose an adaptive momentum sampling method to accelerate the transforming process. We show that our method can produce more faithful images/graphs in small sampling steps with 2 to 5 times speed up.
  arXiv  Detail & Related papers  (2024-05-22T15:20:27Z)
• Blue noise for diffusion models [50.99852321110366]
  We introduce a novel and general class of diffusion models taking correlated noise within and across images into account. Our framework allows introducing correlation across images within a single mini-batch to improve gradient flow. We perform both qualitative and quantitative evaluations on a variety of datasets using our method.
  arXiv  Detail & Related papers  (2024-02-07T14:59:25Z)
• Parallel Sampling of Diffusion Models [76.3124029406809]
  Diffusion models are powerful generative models but suffer from slow sampling. We present ParaDiGMS, a novel method to accelerate the sampling of pretrained diffusion models by denoising multiple steps in parallel.
  arXiv  Detail & Related papers  (2023-05-25T17:59:42Z)
• Accelerating Diffusion Models via Early Stop of the Diffusion Process [114.48426684994179]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved impressive performance on various generation tasks. In practice DDPMs often need hundreds even thousands of denoising steps to obtain a high-quality sample. We propose a principled acceleration strategy, referred to as Early-Stopped DDPM (ES-DDPM), for DDPMs.
  arXiv  Detail & Related papers  (2022-05-25T06:40:09Z)
• Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed [0.0]
  Iterative generative models, such as noise conditional score networks, produce high quality samples by gradually denoising an initial noise vector. We establish a novel connection between knowledge distillation and image generation with a technique that distills a multi-step denoising process into a single step. Our Denoising Student generates high quality samples comparable to GANs on the CIFAR-10 and CelebA datasets, without adversarial training.
  arXiv  Detail & Related papers  (2021-01-07T06:12:28Z)
• Denoising Diffusion Implicit Models [117.03720513930335]
  We present denoising diffusion implicit models (DDIMs) for iterative implicit probabilistic models with the same training procedure as DDPMs. DDIMs can produce high quality samples $10 times$ to $50 times$ faster in terms of wall-clock time compared to DDPMs.
  arXiv  Detail & Related papers  (2020-10-06T06:15:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.