Three Creates All: You Only Sample 3 Steps

• URL: http://arxiv.org/abs/2603.22375v1
• Date: Mon, 23 Mar 2026 09:13:53 GMT
• Title: Three Creates All: You Only Sample 3 Steps
• Abstract summary: We propose Multi-layer Time Embedding Optimization (MTEO), which freeze the pretrained diffusion backbone and distill a small set of step-wise, layer-wise time embeddings from reference trajectories.<n>Experiments across diverse datasets and backbones show state-of-the-art performance in the few-step sampling and substantially narrow the gap between distillation-based and lightweight methods.
• Score: 8.044338005450347
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models deliver high-fidelity generation but remain slow at inference time due to many sequential network evaluations. We find that standard timestep conditioning becomes a key bottleneck for few-step sampling. Motivated by layer-dependent denoising dynamics, we propose Multi-layer Time Embedding Optimization (MTEO), which freeze the pretrained diffusion backbone and distill a small set of step-wise, layer-wise time embeddings from reference trajectories. MTEO is plug-and-play with existing ODE solvers, adds no inference-time overhead, and trains only a tiny fraction of parameters. Extensive experiments across diverse datasets and backbones show state-of-the-art performance in the few-step sampling and substantially narrow the gap between distillation-based and lightweight methods. Code will be available.

Related papers

• DRiffusion: Draft-and-Refine Process Parallelizes Diffusion Models with Ease [3.5025591191253933]
  DRiffusion is a parallel sampling framework that parallelizes diffusion inference through a draft-and-refine process.<n> Empirically, DRiffusion attains 1.4$times$-3.7$times$ speedup across multiple diffusion models while incur minimal degradation in generation quality.
  arXiv  Detail & Related papers  (2026-03-26T19:58:42Z)
• Ada3Drift: Adaptive Training-Time Drifting for One-Step 3D Visuomotor Robotic Manipulation [53.750389076941396]
  Diffusion-based visuomotor policies capture multimodal action distributions through iterative denoising, but their high inference latency limits real-time robotic control.<n>We propose Ada3Drift, which learns a training-time drifting field that attracts predicted actions toward expert demonstration modes.<n>Ada3Drift achieves state-of-the-art performance while requiring $10times$ fewer function evaluations than diffusion-based alternatives.
  arXiv  Detail & Related papers  (2026-03-12T14:36:53Z)
• Operator Flow Matching for Timeseries Forecasting [2.406359246841227]
  Existing autoregressive and diffusion-based approaches often suffer cumulative errors and discretisation artifacts that limit long, physically consistent forecasts.<n>We prove an upper bound on FNO error and propose TempO, a latent flow matching model leveraging sparse conditioning with channel folding.
  arXiv  Detail & Related papers  (2025-10-16T19:40:56Z)
• Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Models [53.087070073434845]
  Diffusion models (DMs) have achieved state-of-the-art generative performance but suffer from high sampling latency due to their sequential denoising nature.<n>Existing solver-based acceleration methods often face image quality degradation under a low-latency budget.<n>We propose the Ensemble Parallel Direction solver (dubbed as ours), a novel ODE solver that mitigates truncation errors by incorporating multiple parallel gradient evaluations in each ODE step.
  arXiv  Detail & Related papers  (2025-07-20T03:08:06Z)
• ODE$_t$(ODE$_l$): Shortcutting the Time and Length in Diffusion and Flow Models for Faster Sampling [33.87434194582367]
  In this work, we explore a complementary direction in which the quality-complexity tradeoff can be dynamically controlled.<n>We employ time- and length-wise consistency terms during flow matching training, and as a result, the sampling can be performed with an arbitrary number of time steps.<n>Compared to the previous state of the art, image generation experiments on CelebA-HQ and ImageNet show a latency reduction of up to 3$times$ in the most efficient sampling mode.
  arXiv  Detail & Related papers  (2025-06-26T18:59:59Z)
• Quantizing Diffusion Models from a Sampling-Aware Perspective [43.95032520555463]
  We propose a sampling-aware quantization strategy, wherein a Mixed-Order Trajectory Alignment technique is devised.<n>Experiments on sparse-step fast sampling across multiple datasets demonstrate that our approach preserves the rapid convergence characteristics of high-speed samplers.
  arXiv  Detail & Related papers  (2025-05-04T20:50:44Z)
• Taming Flow Matching with Unbalanced Optimal Transport into Fast Pansharpening [10.23957420290553]
  We propose the Optimal Transport Flow Matching framework to achieve one-step, high-quality pansharpening.<n>The OTFM framework enables simulation-free training and single-step inference while maintaining strict adherence to pansharpening constraints.
  arXiv  Detail & Related papers  (2025-03-19T08:10:49Z)
• Self-Refining Diffusion Samplers: Enabling Parallelization via Parareal Iterations [53.180374639531145]
  Self-Refining Diffusion Samplers (SRDS) retain sample quality and can improve latency at the cost of additional parallel compute.<n>We take inspiration from the Parareal algorithm, a popular numerical method for parallel-in-time integration of differential equations.
  arXiv  Detail & Related papers  (2024-12-11T11:08:09Z)
• One Step Diffusion via Shortcut Models [109.72495454280627]
  We introduce shortcut models, a family of generative models that use a single network and training phase to produce high-quality samples.<n>Shortcut models condition the network on the current noise level and also on the desired step size, allowing the model to skip ahead in the generation process.<n>Compared to distillation, shortcut models reduce complexity to a single network and training phase and additionally allow varying step budgets at inference time.
  arXiv  Detail & Related papers  (2024-10-16T13:34:40Z)
• Towards More Accurate Diffusion Model Acceleration with A Timestep Tuner [112.99126045081046]
  A diffusion model, which is formulated to produce an image using thousands of denoising steps, usually suffers from a slow inference speed.<n>We propose a textbftimestep tuner that helps find a more accurate integral direction for a particular interval at the minimum cost.<n>Experiments show that our plug-in design can be trained efficiently and boost the inference performance of various state-of-the-art acceleration methods.
  arXiv  Detail & Related papers  (2023-10-14T02:19:07Z)
• Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement [51.55157852647306]
  Time series forecasting has been a widely explored task of great importance in many applications. It is common that real-world time series data are recorded in a short time period, which results in a big gap between the deep model and the limited and noisy time series. We propose to address the time series forecasting problem with generative modeling and propose a bidirectional variational auto-encoder equipped with diffusion, denoise, and disentanglement.
  arXiv  Detail & Related papers  (2023-01-08T12:20:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.