AdvDMD: Adversarial Reward Meets DMD For High-Quality Few-Step Generation

• URL: http://arxiv.org/abs/2604.28126v1
• Date: Wed, 29 Apr 2026 16:56:05 GMT
• Title: AdvDMD: Adversarial Reward Meets DMD For High-Quality Few-Step Generation
• Abstract summary: Diffusion models offer superior generation quality at the expense of extensive sampling steps.<n>We propose AdvDMD, a method that seamlessly unifies DMD distillation and RL.<n>We adopt a unified SDE backward simulation and a different training schedule for DMD and RL to enable a more stable and efficient training.
• Score: 29.31853528513521
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models offer superior generation quality at the expense of extensive sampling steps. Distillation methods, with Distribution Matching Distillation (DMD) as a popular example, can mitigate this issue, but performance degradation remains pronounced when sampling steps are limited. Reinforcement learning (RL) has been leveraged to improve the few-step generation quality during distillation, with the potential to even surpass the performance of the teacher model. However, existing approaches are combinatorial in nature, merely integrating an RL process with the distillation process, which introduces unnecessary complexities. To address this gap, we propose AdvDMD, a method that seamlessly unifies DMD distillation and RL. Specifically, AdvDMD employs the adversarially trained discriminator from DMD2 as the reward model, which assigns low scores to generated images and high scores to real ones. It is trained on both intermediate and final states of the denoising process and updated online with the distilled model, enabling a holistic supervision of the sampling trajectories and mitigating reward hacking. We adopt a unified SDE backward simulation and a different training schedule for DMD and RL to enable a more stable and efficient training. Experimental results demonstrate that the 4-step AdvDMD outperforms the original 40-step model for SD3.5 on DPG-Bench, while achieving significant performance gains for SD3 on the GenEval. On Qwen-Image, our 2-step AdvDMD achieves superior performance over TwinFlow.

Related papers

• Guiding Distribution Matching Distillation with Gradient-Based Reinforcement Learning [41.982957134224904]
  Diffusion distillation has shown great promise in few-step generation but often sacrifices quality for sampling speed.<n>We propose GDMD, a novel framework that redefines the reward mechanism by prioritizing distillation gradients over raw pixel outputs.<n>Our models outperform the quality of their multi-step teacher and substantially exceed previous DMDR results in GenEval and human-preference metrics.
  arXiv  Detail & Related papers  (2026-04-21T02:57:13Z)
• Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learning [32.32567390728913]
  Diffusion Models have emerged as a leading class of generative models.<n>Timestep distillation is a promising technique to accelerate generation, but it often requires extensive training and leads to image quality degradation.<n>We introduce Flash-DMD, a novel framework that enables fast convergence with distillation and joint RL-based refinement.
  arXiv  Detail & Related papers  (2025-11-25T17:47:11Z)
• Distribution Matching Distillation Meets Reinforcement Learning [30.960105413888943]
  We propose DMDR, a novel framework that combines Reinforcement Learning (RL) techniques into the distillation process.<n>We show that for the RL of the few-step generator, the DMD loss itself is a more effective regularization compared to the traditional ones.<n>Experiments demonstrate that DMDR can achieve leading visual quality, prompt coherence among few-step methods, and even exhibit performance that exceeds the multi-step teacher.
  arXiv  Detail & Related papers  (2025-11-17T17:59:54Z)
• Phased DMD: Few-step Distribution Matching Distillation via Score Matching within Subintervals [48.14879329270912]
  Phased DMD is a multi-step distillation framework that bridges the idea of phase-wise distillation with Mixture-of-Experts.<n>Phased DMD is built upon two key ideas: progressive distribution matching and score matching within subintervals.<n> Experimental results demonstrate that Phased DMD preserves output diversity better than DMD while retaining key generative capabilities.
  arXiv  Detail & Related papers  (2025-10-31T17:55:10Z)
• Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis [65.77083310980896]
  We propose Adrial Distribution Matching (ADM) to align latent predictions between real and fake score estimators for score distillation.<n>Our proposed method achieves superior one-step performance on SDXL compared to DMD2 while consuming less GPU time.<n>Additional experiments that apply multi-step ADM distillation on SD3-Medium, SD3.5-Large, and CogVideoX set a new benchmark towards efficient image and video synthesis.
  arXiv  Detail & Related papers  (2025-07-24T16:45:05Z)
• Improved Diffusion-based Generative Model with Better Adversarial Robustness [65.38540020916432]
  Diffusion Probabilistic Models (DPMs) have achieved significant success in generative tasks.<n>During the denoising process, the input data distributions differ between the training and inference stages.
  arXiv  Detail & Related papers  (2025-02-24T12:29:16Z)
• Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminator [81.81748032199813]
  Diffusion models have demonstrated excellent performance for real-world image super-resolution (Real-ISR)<n>We propose a new One-Step textbfDiffusion model with a larger-scale textbfDiscriminator for SR.<n>Our discriminator is able to distill noisy features from any time step of diffusion models in the latent space.
  arXiv  Detail & Related papers  (2024-10-05T16:41:36Z)
• EM Distillation for One-step Diffusion Models [65.57766773137068]
  We propose a maximum likelihood-based approach that distills a diffusion model to a one-step generator model with minimal loss of quality.<n>We develop a reparametrized sampling scheme and a noise cancellation technique that together stabilizes the distillation process.
  arXiv  Detail & Related papers  (2024-05-27T05:55:22Z)
• Improved Distribution Matching Distillation for Fast Image Synthesis [54.72356560597428]
  We introduce DMD2, a set of techniques that lift this limitation and improve DMD training. First, we eliminate the regression loss and the need for expensive dataset construction. Second, we integrate a GAN loss into the distillation procedure, discriminating between generated samples and real images.
  arXiv  Detail & Related papers  (2024-05-23T17:59:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.