One-Step Distillation of Discrete Diffusion Image Generators via Fixed-Point Iteration

• URL: http://arxiv.org/abs/2605.21484v1
• Date: Wed, 20 May 2026 17:59:10 GMT
• Title: One-Step Distillation of Discrete Diffusion Image Generators via Fixed-Point Iteration
• Abstract summary: Fixed-Point Distillation (FPD) is an end-to-end framework that constructs local correction targets by partially corrupting the student's one-step draft and refining it with a single teacher step.<n>FPD achieves competitive visual fidelity and structural alignment within a single inference step, narrowing the gap to multi-step teachers.
• Score: 20.457786704615454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Discrete diffusion models excel at visual synthesis but rely on slow, iterative decoding. Existing single-step distillation methods attempt to bypass this bottleneck, either by training auxiliary score networks that effectively double compute, or by introducing specialized parameterizations and multi-stage pipelines that fragment optimization. In this paper, we introduce Fixed-Point Distillation (FPD), an end-to-end framework that constructs local correction targets by partially corrupting the student's one-step draft and refining it with a single teacher step. To compute the training objective in a semantically meaningful space, we lift discrete tokens into continuous features and apply a multi-bandwidth drift loss that iteratively accumulates these corrections. To backpropagate through the discrete bottleneck, we employ a straight-through estimator that feeds exact hard-sampled tokens to the teacher and decoder during the forward pass, ensuring that training and inference operate on the same codebook manifold, while routing continuous gradients back to the student logits. This fully differentiable pathway additionally accommodates an optional unconditional adversarial objective to enhance perceptual realism. Evaluations on both class- and text-conditional generation validate the effectiveness of our framework. FPD achieves competitive visual fidelity and structural alignment within a single inference step, narrowing the gap to multi-step teachers while outperforming existing discrete distillation baselines.

Related papers

• Beyond Trajectory Matching: Reflow with Marginal Distribution Alignment [7.13416637929041]
  Diffusion and continuous-flow generative models achieve high-quality generation.<n>Reflow-based distillation simplifies teacher ODE trajectories so that a student model can approximate the teacher transport with fewer steps.<n>We introduce a marginal-alignment regularizer that penalizes the discrepancy between the student-induced marginal and the corresponding teacher marginal at the endpoint of each distillation interval.
  arXiv  Detail & Related papers  (2026-06-28T09:19:44Z)
• FlowBender: Feedback-Aware Training for Self-Correcting Conditional Flows [18.746098727212544]
  FlowBender is a new diffusion and flow model for 3D reconstruction.<n>We show that FlowBender consistently outperforms standard supervised alignment training.<n>We propose several variants of FlowBender for differentiable operators.
  arXiv  Detail & Related papers  (2026-06-18T15:56:07Z)
• Learn to Rank: Visual Attribution by Learning Importance Ranking [58.69028273772474]
  We propose a new approach to generate visual attribution maps for computer vision models.<n>Our method produces dense, pixel-level attributions in a single forward pass with optional, few-step gradient refinement.<n>Our experiments demonstrate consistent quantitative improvements and sharper, boundary-aligned explanations.
  arXiv  Detail & Related papers  (2026-04-07T12:53:22Z)
• Rethinking Vector Field Learning for Generative Segmentation [50.08025820235397]
  Taming diffusion models for generative segmentation has attracted increasing attention.<n>We revisit diffusion segmentation from the perspective of vector field learning.<n>We propose a principled vector field reshaping strategy that augments the learned vanishing velocity field with a detached distance-aware correction term.<n>This correction introduces both attractive and repulsive interactions, enhancing gradient magnitudes near centroids while preserving the original diffusion training framework.
  arXiv  Detail & Related papers  (2026-03-19T17:58:19Z)
• ManifoldGD: Training-Free Hierarchical Manifold Guidance for Diffusion-Based Dataset Distillation [9.230247128710865]
  We propose a training-free diffusion-based framework that integrates manifold consistent guidance at every denoising timestep.<n>ManifoldGD improves representativeness, diversity, and image fidelity without requiring any model retraining.
  arXiv  Detail & Related papers  (2026-02-26T18:07:10Z)
• Generalized Discrete Diffusion with Self-Correction [20.435906948993022]
  Self-correction is an effective technique for maintaining parallel sampling in discrete diffusion models with minimal performance degradation.<n>We propose a Self-Correcting Diffusion model to reformulate pretrained self-correction with explicit state transitions and learn directly in discrete time.<n>Our framework also simplifies the training noise schedule, eliminates a redundant remasking step, and relies exclusively on uniform transitions to learn self-correction.
  arXiv  Detail & Related papers  (2026-02-13T04:12:35Z)
• Stochastic Interpolants via Conditional Dependent Coupling [36.84747986070112]
  Existing image generation models face critical challenges regarding the trade-off between computation and fidelity.<n>We introduce a unified multistage generative framework based on our proposed Conditional Dependent Coupling strategy.<n>It decomposes the generative process into interpolant trajectories at multiple stages, ensuring accurate distribution learning while enabling end-to-end optimization.
  arXiv  Detail & Related papers  (2025-09-27T05:03:08Z)
• Efficient Diffusion as Low Light Enhancer [63.789138528062225]
  Reflectance-Aware Trajectory Refinement (RATR) is a simple yet effective module to refine the teacher trajectory using the reflectance component of images. textbfReflectance-aware textbfDiffusion with textbfDistilled textbfTrajectory (textbfReDDiT) is an efficient and flexible distillation framework tailored for Low-Light Image Enhancement (LLIE)
  arXiv  Detail & Related papers  (2024-10-16T08:07:18Z)
• Multi-Granularity Semantic Revision for Large Language Model Distillation [66.03746866578274]
  We propose a multi-granularity semantic revision method for LLM distillation. At the sequence level, we propose a sequence correction and re-generation strategy. At the token level, we design a distribution adaptive clipping Kullback-Leibler loss as the distillation objective function. At the span level, we leverage the span priors of a sequence to compute the probability correlations within spans, and constrain the teacher and student's probability correlations to be consistent.
  arXiv  Detail & Related papers  (2024-07-14T03:51:49Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Dense Unsupervised Learning for Video Segmentation [49.46930315961636]
  We present a novel approach to unsupervised learning for video object segmentation (VOS) Unlike previous work, our formulation allows to learn dense feature representations directly in a fully convolutional regime. Our approach exceeds the segmentation accuracy of previous work despite using significantly less training data and compute power.
  arXiv  Detail & Related papers  (2021-11-11T15:15:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.