1.x-Distill: Breaking the Diversity, Quality, and Efficiency Barrier in Distribution Matching Distillation

• URL: http://arxiv.org/abs/2604.04018v1
• Date: Sun, 05 Apr 2026 08:30:35 GMT
• Title: 1.x-Distill: Breaking the Diversity, Quality, and Efficiency Barrier in Distribution Matching Distillation
• Abstract summary: 1.x-Distill is the first fractional-step distillation framework that breaks the integer-step constraint of prior few-step methods.<n>We show that 1.x-Distill surpasses prior few-step methods, achieving better quality and diversity at 1.67 and 1.74 effective NFEs, respectively, with up to 33x speedup over original 28x2 NFE sampling.
• Score: 22.287956787775872
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models produce high-quality text-to-image results, but their iterative denoising is computationally expensive.Distribution Matching Distillation (DMD) emerges as a promising path to few-step distillation, but suffers from diversity collapse and fidelity degradation when reduced to two steps or fewer. We present 1.x-Distill, the first fractional-step distillation framework that breaks the integer-step constraint of prior few-step methods and establishes 1.x-step generation as a practical regime for distilled diffusion models.Specifically, we first analyze the overlooked role of teacher CFG in DMD and introduce a simple yet effective modification to suppress mode collapse. Then, to improve performance under extreme steps, we introduce Stagewise Focused Distillation, a two-stage strategy that learns coarse structure through diversity-preserving distribution matching and refines details with inference-consistent adversarial distillation. Furthermore, we design a lightweight compensation module for Distill--Cache co-Training, which naturally incorporates block-level caching into our distillation pipeline.Experiments on SD3-Medium and SD3.5-Large show that 1.x-Distill surpasses prior few-step methods, achieving better quality and diversity at 1.67 and 1.74 effective NFEs, respectively, with up to 33x speedup over original 28x2 NFE sampling.

Related papers

• Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis [65.77083310980896]
  We propose Adrial Distribution Matching (ADM) to align latent predictions between real and fake score estimators for score distillation.<n>Our proposed method achieves superior one-step performance on SDXL compared to DMD2 while consuming less GPU time.<n>Additional experiments that apply multi-step ADM distillation on SD3-Medium, SD3.5-Large, and CogVideoX set a new benchmark towards efficient image and video synthesis.
  arXiv  Detail & Related papers  (2025-07-24T16:45:05Z)
• MGD$^3$: Mode-Guided Dataset Distillation using Diffusion Models [50.2406741245418]
  We propose a mode-guided diffusion model leveraging a pre-trained diffusion model.<n>Our approach addresses dataset diversity in three stages: Mode Discovery to identify distinct data modes, Mode Guidance to enhance intra-class diversity, and Stop Guidance to mitigate artifacts in synthetic samples.<n>Our method eliminates the need for fine-tuning diffusion models with distillation losses, significantly reducing computational costs.
  arXiv  Detail & Related papers  (2025-05-25T03:40:23Z)
• Few-Step Diffusion via Score identity Distillation [67.07985339442703]
  Diffusion distillation has emerged as a promising strategy for accelerating text-to-image (T2I) diffusion models.<n>Existing methods rely on real or teacher-synthesized images to perform well when distilling high-resolution T2I diffusion models.<n>We propose two new guidance strategies: Zero-CFG, which disables CFG in the teacher and removes text conditioning in the fake score network, and Anti-CFG, which applies negative CFG in the fake score network.
  arXiv  Detail & Related papers  (2025-05-19T03:45:16Z)
• DDIL: Diversity Enhancing Diffusion Distillation With Imitation Learning [57.3467234269487]
  Diffusion models excel at generative modeling (e.g., text-to-image) but sampling requires multiple denoising network passes.<n>Progressive distillation or consistency distillation have shown promise by reducing the number of passes.<n>We show that DDIL consistency improves on baseline algorithms of progressive distillation (PD), Latent consistency models (LCM) and Distribution Matching Distillation (DMD2)
  arXiv  Detail & Related papers  (2024-10-15T18:21:47Z)
• Presto! Distilling Steps and Layers for Accelerating Music Generation [49.34961693154768]
  Presto! is an approach to inference acceleration for score-based diffusion transformers.<n>We develop a new score-based distribution matching distillation (DMD) method for the EDM-family of diffusion models.<n>To reduce the cost per step, we develop a simple, but powerful improvement to a recent layer distillation method.
  arXiv  Detail & Related papers  (2024-10-07T16:24:18Z)
• Distilling Diffusion Models into Conditional GANs [90.76040478677609]
  We distill a complex multistep diffusion model into a single-step conditional GAN student model. For efficient regression loss, we propose E-LatentLPIPS, a perceptual loss operating directly in diffusion model's latent space. We demonstrate that our one-step generator outperforms cutting-edge one-step diffusion distillation models.
  arXiv  Detail & Related papers  (2024-05-09T17:59:40Z)
• Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesis [20.2271205957037]
  Hyper-SD is a novel framework that amalgamates the advantages of ODE Trajectory Preservation and Reformulation. We introduce Trajectory Segmented Consistency Distillation to progressively perform consistent distillation within pre-defined time-step segments. We incorporate human feedback learning to boost the performance of the model in a low-step regime and mitigate the performance loss incurred by the distillation process.
  arXiv  Detail & Related papers  (2024-04-21T15:16:05Z)
• Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation [24.236841051249243]
  Distillation methods aim to shift the model from many-shot to single-step inference. We introduce Latent Adversarial Diffusion Distillation (LADD), a novel distillation approach overcoming the limitations of ADD. In contrast to pixel-based ADD, LADD utilizes generative features from pretrained latent diffusion models.
  arXiv  Detail & Related papers  (2024-03-18T17:51:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.