Representation Distribution Matching for One-Step Visual Generation

• URL: http://arxiv.org/abs/2607.02375v1
• Date: Thu, 02 Jul 2026 16:15:38 GMT
• Title: Representation Distribution Matching for One-Step Visual Generation
• Abstract summary: We elucidate the design space of Representation Distribution Matching (RDM)<n>We identify two design axes, how the distributions are compared and the representations they are compared in, and controlled studies along them yield three findings.<n>The same recipe post-trains the four-step FLUX.2 [klein] into a one-step generator, surpassing the four-step version on GenEval, 0.826 to 0.794, and on PickScore, 22.76 to 22.58, in 90 H200 GPU-hours.
• Score: 107.84208192304455
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We elucidate the design space of Representation Distribution Matching (RDM), our name for the paradigm that trains a one-step image generator by matching generated and reference feature distributions under frozen pretrained encoders. We identify two design axes, how the distributions are compared and the representations they are compared in, and controlled studies along them yield three findings. First, the classical MMD, which could not train convincing generators a decade ago, becomes a strong and scalable objective once estimated right. Second, the generated batch is then the operative variable, with an optimum above 2048, far beyond customary batch sizes. Third, any single representation can be gamed, driven below the real score while images stay visibly fake, so we match against a balanced battery of encoders and evaluate with SW_r14, a Sliced-Wasserstein distance over 14 encoders that is independent of the training loss and resists gaming. Combining the preferred choices yields improved RDM (iRDM): it sets the one-step state of the art on ImageNet at SW_r14 1.30, corroborated by PickScore, a human-preference proxy our objective never optimizes, which prefers it over the prior best one-step generator on 71.2% of matched samples. The same recipe post-trains the four-step FLUX.2 [klein] into a one-step generator, surpassing the four-step version on GenEval, 0.826 to 0.794, and on PickScore, 22.76 to 22.58, in 90 H200 GPU-hours. Project page: https://alan-lanfeng.github.io/rdm/.

Related papers

• GenSyn10: A Multi-Generative AI Dataset For Benchmarking Image Classification [17.505968172455784]
  GenSyn10 is a CIFAR-10-aligned synthetic image dataset of 60,000 images (10 classes, 32$times$32, 50k/10k split) generated using three state-of-the-art models.<n>We evaluate 17 image classification models under a four-stage protocol: real-data baseline, zero-shot transfer, fine-tuning, and retention.<n>Despite a measurable domain gap, CIFAR-10-trained models achieve up to 96.86% zero-shot accuracy on GenSyn10, increasing to 99.88% after fine-tuning.
  arXiv  Detail & Related papers  (2026-07-10T09:47:16Z)
• VA-$π$: Variational Policy Alignment for Pixel-Aware Autoregressive Generation [65.22452273252428]
  VA-$$ is a post-training framework to optimize autoregressive visual generation.<n>It unifies pixel reconstruction and autoregressive modeling.<n>It reduces FID from 14.36 to 7.65 and improves IS from 86.55 to 116.70 on LlamaGen-XXL.
  arXiv  Detail & Related papers  (2025-12-22T18:54:30Z)
• iGaussian: Real-Time Camera Pose Estimation via Feed-Forward 3D Gaussian Splatting Inversion [62.09575122593993]
  iGaussian is a two-stage feed-forward framework that achieves real-time camera pose estimation through direct 3D Gaussian inversion.<n> Experimental results on the NeRF Synthetic, Mip-NeRF 360, and T&T+DB datasets demonstrate a significant performance improvement over previous methods.
  arXiv  Detail & Related papers  (2025-11-18T05:22:22Z)
• David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-training [8.352666876052616]
  We propose Diff-Instruct* (DI*), a data-efficient post-training approach for one-step text-to-image generative models.<n>Our method frames alignment as online reinforcement learning from human feedback.<n>Our 2.6B emphDI*-SDXL-1step model outperforms the 50-step 12B FLUX-dev model.
  arXiv  Detail & Related papers  (2024-10-28T10:26:19Z)
• One-Step Diffusion Distillation through Score Implicit Matching [74.91234358410281]
  We present Score Implicit Matching (SIM) a new approach to distilling pre-trained diffusion models into single-step generator models. SIM shows strong empirical performances for one-step generators. By applying SIM to a leading transformer-based diffusion model, we distill a single-step generator for text-to-image generation.
  arXiv  Detail & Related papers  (2024-10-22T08:17:20Z)
• Improved Distribution Matching Distillation for Fast Image Synthesis [54.72356560597428]
  We introduce DMD2, a set of techniques that lift this limitation and improve DMD training. First, we eliminate the regression loss and the need for expensive dataset construction. Second, we integrate a GAN loss into the distillation procedure, discriminating between generated samples and real images.
  arXiv  Detail & Related papers  (2024-05-23T17:59:49Z)
• One-step Diffusion with Distribution Matching Distillation [54.723565605974294]
  We introduce Distribution Matching Distillation (DMD), a procedure to transform a diffusion model into a one-step image generator. We enforce the one-step image generator match the diffusion model at distribution level, by minimizing an approximate KL divergence. Our method outperforms all published few-step diffusion approaches, reaching 2.62 FID on ImageNet 64x64 and 11.49 FID on zero-shot COCO-30k.
  arXiv  Detail & Related papers  (2023-11-30T18:59:20Z)
• Improved DDIM Sampling with Moment Matching Gaussian Mixtures [1.450405446885067]
  We propose using a Gaussian Mixture Model (GMM) as reverse transition operator ( kernel) within the Denoising Diffusion Implicit Models (DDIM) framework.<n>We match the first and second order central moments of the DDPM forward marginals by constraining the parameters of the GMM.<n>Our results suggest that using the GMM kernel leads to significant improvements in the quality of the generated samples when the number of sampling steps is small.
  arXiv  Detail & Related papers  (2023-11-08T00:24:50Z)
• Binary and Ternary Natural Language Generation [24.295815261826153]
  Ternary and binary neural networks enable multiplication-free computation. They promise multiple orders of magnitude efficiency gains over full-precision networks. However, such networks have proven very difficult to optimize. We show first ternary and binary transformer models on the downstream tasks of summarization and machine translation.
  arXiv  Detail & Related papers  (2023-06-02T18:01:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.