Guiding Distribution Matching Distillation with Gradient-Based Reinforcement Learning

• URL: http://arxiv.org/abs/2604.19009v1
• Date: Tue, 21 Apr 2026 02:57:13 GMT
• Title: Guiding Distribution Matching Distillation with Gradient-Based Reinforcement Learning
• Abstract summary: Diffusion distillation has shown great promise in few-step generation but often sacrifices quality for sampling speed.<n>We propose GDMD, a novel framework that redefines the reward mechanism by prioritizing distillation gradients over raw pixel outputs.<n>Our models outperform the quality of their multi-step teacher and substantially exceed previous DMDR results in GenEval and human-preference metrics.
• Score: 41.982957134224904
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion distillation, exemplified by Distribution Matching Distillation (DMD), has shown great promise in few-step generation but often sacrifices quality for sampling speed. While integrating Reinforcement Learning (RL) into distillation offers potential, a naive fusion of these two objectives relies on suboptimal raw sample evaluation. This sample-based scoring creates inherent conflicts with the distillation trajectory and produces unreliable rewards due to the noisy nature of early-stage generation. To overcome these limitations, we propose GDMD, a novel framework that redefines the reward mechanism by prioritizing distillation gradients over raw pixel outputs as the primary signal for optimization. By reinterpreting the DMD gradients as implicit target tensors, our framework enables existing reward models to directly evaluate the quality of distillation updates. This gradient-level guidance functions as an adaptive weighting that synchronizes the RL policy with the distillation objective, effectively neutralizing optimization divergence. Empirical results show that GDMD sets a new SOTA for few-step generation. Specifically, our 4-step models outperform the quality of their multi-step teacher and substantially exceed previous DMDR results in GenEval and human-preference metrics, exhibiting strong scalability potential.

Related papers

• Joint Alignment and Distillation for Video Generation via Sample-Guided Distribution Matching [36.00855152229835]
  We propose a unified, single-stage optimization framework grounded in Distribution Matching.<n>In the standard DM framework, distillation updates the model via a gradient direction that minimizes the gap between the real and fake models.<n>We introduce DM-Align, which derives a complementary gradient direction to guide the model toward human-preferred samples.
  arXiv  Detail & Related papers  (2026-09-03T04:46:46Z)
• Reinforcing Few-step Generators via Reward-Tilted Distribution Matching [34.875805803270524]
  We propose Reward-Tilted Distribution Matching Distillation (RTDMD)<n>We show that minimizing KL divergence to a reward-tilted teacher distribution naturally decomposes into a distribution matching term and a reward term.<n>Experiments on SD3, SD3.5, and FLUX.2 demonstrate that RTDMD establishes new state-of-the-art results across preference, aesthetic, and compositional metrics.
  arXiv  Detail & Related papers  (2026-05-25T17:59:21Z)
• $R_\text{dm}$: Re-conceptualizing Distribution Matching as a Reward for Diffusion Distillation [9.105357939499683]
  Diffusion models achieve state-of-the-art generative performance but are bottlenecked by their slow, iterative sampling process.<n>Recent approaches have attempted to break this ceiling by integrating Reinforcement Learning (RL)<n>We propose a novel paradigm by re-conceptualizing distribution matching as a reward, denoted as $R_textdm$.
  arXiv  Detail & Related papers  (2026-03-30T14:01:31Z)
• Phased DMD: Few-step Distribution Matching Distillation via Score Matching within Subintervals [48.14879329270912]
  Phased DMD is a multi-step distillation framework that bridges the idea of phase-wise distillation with Mixture-of-Experts.<n>Phased DMD is built upon two key ideas: progressive distribution matching and score matching within subintervals.<n> Experimental results demonstrate that Phased DMD preserves output diversity better than DMD while retaining key generative capabilities.
  arXiv  Detail & Related papers  (2025-10-31T17:55:10Z)
• Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis [65.77083310980896]
  We propose Adrial Distribution Matching (ADM) to align latent predictions between real and fake score estimators for score distillation.<n>Our proposed method achieves superior one-step performance on SDXL compared to DMD2 while consuming less GPU time.<n>Additional experiments that apply multi-step ADM distillation on SD3-Medium, SD3.5-Large, and CogVideoX set a new benchmark towards efficient image and video synthesis.
  arXiv  Detail & Related papers  (2025-07-24T16:45:05Z)
• Restoration Score Distillation: From Corrupted Diffusion Pretraining to One-Step High-Quality Generation [82.39763984380625]
  We propose textitRestoration Score Distillation (RSD), a principled generalization of Denoising Score Distillation (DSD)<n>RSD accommodates a broader range of corruption types, such as blurred, incomplete, or low-resolution images.<n>It consistently surpasses its teacher model across diverse restoration tasks on both natural and scientific datasets.
  arXiv  Detail & Related papers  (2025-05-19T17:21:03Z)
• Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generation [82.39763984380625]
  We introduce denoising score distillation (DSD), a surprisingly effective and novel approach for training high-quality generative models from low-quality data.<n>DSD pretrains a diffusion model exclusively on noisy, corrupted samples and then distills it into a one-step generator capable of producing refined, clean outputs.
  arXiv  Detail & Related papers  (2025-03-10T17:44:46Z)
• EM Distillation for One-step Diffusion Models [65.57766773137068]
  We propose a maximum likelihood-based approach that distills a diffusion model to a one-step generator model with minimal loss of quality.<n>We develop a reparametrized sampling scheme and a noise cancellation technique that together stabilizes the distillation process.
  arXiv  Detail & Related papers  (2024-05-27T05:55:22Z)
• Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood [64.95663299945171]
  Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming. There exists a noticeable gap in sample quality between EBMs and other generative frameworks like GANs and diffusion models. We propose cooperative diffusion recovery likelihood (CDRL), an effective approach to tractably learn and sample from a series of EBMs.
  arXiv  Detail & Related papers  (2023-09-10T22:05:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.