SGMD: Score Gradient Matching Distillation for Few-Step Video Diffusion Distillation
- URL: http://arxiv.org/abs/2605.30116v1
- Date: Thu, 28 May 2026 15:50:55 GMT
- Title: SGMD: Score Gradient Matching Distillation for Few-Step Video Diffusion Distillation
- Abstract summary: Distribution Matching Distillation (DMD) is a widely used paradigm for accelerating inference in few-step video diffusion models.<n>We propose textbfScore Gradient Matching Distillation (SGMD).<n>It adopts a fake-score perspective by directly optimizing the fake score toward the teacher, while using teacher stop-gradient Fisher as a stable distribution-matching objective.<n>It substantially improves motion dynamics for 4-step distilled models while preserving temporal consistency.
- Score: 57.297118390628384
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Distribution Matching Distillation (DMD) is a widely used paradigm for accelerating inference in few-step video diffusion models. However, DMD-style video distillation faces two coupled challenges: the fake score must track a continuously evolving generator, making training costly when frequent updates are required, while reverse-KL-style matching can be mode-seeking and conservative for preserving strong motion dynamics. To address these issues, we propose \textbf{Score Gradient Matching Distillation (SGMD)}. SGMD adopts a fake-score perspective by directly optimizing the fake score toward the teacher, while using teacher stop-gradient Fisher as a stable distribution-matching objective. We provide a gradient analysis that motivates this objective choice under ideal tracking. Building on this, SGMD introduces a pair of dual potentials: negative-residual (NR) for outer-loop correction and residual-contraction (RC) for inner-loop tracking. Empirically, compared to DMD2, SGMD achieves an approximately $\sim 3\times$ training speedup and substantially improves motion dynamics for 4-step distilled models while preserving temporal consistency. A human study confirms that SGMD is preferred in motion quality and overall preference, while visual quality and text alignment remain comparable. Code is available at https://github.com/ModelTC/LightX2V.
Related papers
- Channel-wise Dynamic Knowledge Distillation via Adaptive Sample Generation for Action Recognition [53.278983063932706]
We develop an Adaptive Sample-aware Channel-wise Dynamic (ASCD) KD approach, which operates in two stages.<n>First, we use an adaptive sample generation module to create updated samples by incorporating semantics from sample gradients.<n>Second, we employ a channel-wise dynamic distillation module to train student on these generated samples, guided by sample gradients and feature frequencies.
arXiv Detail & Related papers (2026-08-04T04:18:13Z) - Diffusion Fine-tuning with Rewarded Moment Matching Distillation [38.86104469934903]
Rewarded Moment Matching Distillation (RMMD) is a novel framework that simultaneously distills diffusion models and maximizes a reward function.<n>We apply RMMD to GenCast, a state-of-the-art weather forecasting model, to distill it while optimizing the Continuous Ranked Probability Score (CRPS) metric.<n>The resulting distilled model achieves a 7.5x speedup while outperforming the teacher model on 93% of target weather variables, and being better calibrated.
arXiv Detail & Related papers (2026-06-29T15:00:56Z) - Data-Forcing Distillation: Restoring Diversity and Fidelity in Few-Step Video Generation [25.352409052792122]
We propose Data-Forcing Distillation (DFD), a simple post-training framework that restores diversity and fidelity in DMD with only a single-line of code change.<n>We provide an in-depth theoretical analysis of our framework and validate our approach on text-to-video, image-to-video, and autoregressive video generation.
arXiv Detail & Related papers (2026-06-16T20:38:30Z) - Continuous-Time Distribution Matching for Few-Step Diffusion Distillation [57.28746398500951]
We introduce Continuous-Time Distribution Matching (CDM), migrating the DMD framework from discrete anchoring to continuous optimization.<n>CDM achieves this through two continuous-time designs. First, we replace the fixed discrete schedule with a dynamic continuous schedule of random length.<n>Second, we propose a continuous-time alignment objective that performs active off-trajectory matching on latents extrapolated via the student's velocity field.
arXiv Detail & Related papers (2026-05-07T14:56:39Z) - Transition Matching Distillation for Fast Video Generation [63.1049790376783]
We present Transition Matching Distillation (TMD), a novel framework for distilling video diffusion models into efficient few-step generators.<n>TMD matches the multi-step denoising trajectory of a diffusion model with a few-step probability transition process.<n>TMD provides a flexible and strong trade-off between generation speed and visual quality.
arXiv Detail & Related papers (2026-01-14T21:30:03Z) - Phased DMD: Few-step Distribution Matching Distillation via Score Matching within Subintervals [48.14879329270912]
Phased DMD is a multi-step distillation framework that bridges the idea of phase-wise distillation with Mixture-of-Experts.<n>Phased DMD is built upon two key ideas: progressive distribution matching and score matching within subintervals.<n> Experimental results demonstrate that Phased DMD preserves output diversity better than DMD while retaining key generative capabilities.
arXiv Detail & Related papers (2025-10-31T17:55:10Z) - Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency [60.74505433956616]
continuous-time consistency model (sCM) is theoretically principled and empirically powerful for accelerating academic-scale diffusion.<n>We first develop a parallelism-compatible FlashAttention-2 JVP kernel, enabling sCM training on models with over 10 billion parameters and high-dimensional video tasks.<n>We propose the score-regularized continuous-time consistency model (rCM), which incorporates score distillation as a long-skip regularizer.
arXiv Detail & Related papers (2025-10-09T16:45:30Z) - Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis [65.77083310980896]
We propose Adrial Distribution Matching (ADM) to align latent predictions between real and fake score estimators for score distillation.<n>Our proposed method achieves superior one-step performance on SDXL compared to DMD2 while consuming less GPU time.<n>Additional experiments that apply multi-step ADM distillation on SD3-Medium, SD3.5-Large, and CogVideoX set a new benchmark towards efficient image and video synthesis.
arXiv Detail & Related papers (2025-07-24T16:45:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.