Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment

• URL: http://arxiv.org/abs/2506.02221v1
• Date: Mon, 02 Jun 2025 20:05:05 GMT
• Title: Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment
• Authors: Johannes Schusterbauer, Ming Gui, Frank Fundel, Björn Ommer,
• Abstract summary: Diffusion models have revolutionized generative tasks through high-fidelity outputs, yet flow matching (FM) offers faster inference and empirical performance gains.<n>This work addresses the critical challenge of efficiently transferring knowledge from pre-trained diffusion models to flow matching.<n>We propose Diff2Flow, a novel framework that systematically bridges diffusion and FM paradigms by rescaling timesteps, aligning interpolants, and deriving FM-compatible velocity fields from diffusion predictions.
• Score: 22.661660797545164
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models have revolutionized generative tasks through high-fidelity outputs, yet flow matching (FM) offers faster inference and empirical performance gains. However, current foundation FM models are computationally prohibitive for finetuning, while diffusion models like Stable Diffusion benefit from efficient architectures and ecosystem support. This work addresses the critical challenge of efficiently transferring knowledge from pre-trained diffusion models to flow matching. We propose Diff2Flow, a novel framework that systematically bridges diffusion and FM paradigms by rescaling timesteps, aligning interpolants, and deriving FM-compatible velocity fields from diffusion predictions. This alignment enables direct and efficient FM finetuning of diffusion priors with no extra computation overhead. Our experiments demonstrate that Diff2Flow outperforms na\"ive FM and diffusion finetuning particularly under parameter-efficient constraints, while achieving superior or competitive performance across diverse downstream tasks compared to state-of-the-art methods. We will release our code at https://github.com/CompVis/diff2flow.

Related papers

• FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systems [51.15230303652732]
  FLEX (F Low EXpert) is a backbone architecture for generative modeling of-temporal physical systems.<n>It reduces the variance of the velocity field in the diffusion model, which helps stabilize training.<n>It achieves accurate predictions for super-resolution and forecasting tasks using as few features as two reverse diffusion steps.
  arXiv  Detail & Related papers  (2025-05-23T00:07:59Z)
• Local Flow Matching Generative Models [19.859984725284896]
  Local Flow Matching is a computational framework for density estimation based on flow-based generative models.<n>$textttLFM$ employs a simulation-free scheme and incrementally learns a sequence of Flow Matching sub-models.<n>We demonstrate the improved training efficiency and competitive generative performance of $textttLFM$ compared to FM.
  arXiv  Detail & Related papers  (2024-10-03T14:53:10Z)
• Text-to-Image Rectified Flow as Plug-and-Play Priors [52.586838532560755]
  Rectified flow is a novel class of generative models that enforces a linear progression from the source to the target distribution.<n>We show that rectified flow approaches surpass in terms of generation quality and efficiency, requiring fewer inference steps.<n>Our method also displays competitive performance in image inversion and editing.
  arXiv  Detail & Related papers  (2024-06-05T14:02:31Z)
• Diffusion Tuning: Transferring Diffusion Models via Chain of Forgetting [41.75432332679519]
  We present Diff-Tuning, a frustratingly simple transfer approach that leverages the chain of forgetting tendency. D Diff-Tuning achieves a 26% improvement over standard fine-tuning and enhances the convergence speed of ControlNet by 24%.
  arXiv  Detail & Related papers  (2024-06-02T15:20:59Z)
• Guided Diffusion from Self-Supervised Diffusion Features [49.78673164423208]
  Guidance serves as a key concept in diffusion models, yet its effectiveness is often limited by the need for extra data annotation or pretraining. We propose a framework to extract guidance from, and specifically for, diffusion models.
  arXiv  Detail & Related papers  (2023-12-14T11:19:11Z)
• Guided Flows for Generative Modeling and Decision Making [55.42634941614435]
  We show that Guided Flows significantly improves the sample quality in conditional image generation and zero-shot text synthesis-to-speech. Notably, we are first to apply flow models for plan generation in the offline reinforcement learning setting ax speedup in compared to diffusion models.
  arXiv  Detail & Related papers  (2023-11-22T15:07:59Z)
• The Missing U for Efficient Diffusion Models [3.712196074875643]
  Diffusion Probabilistic Models yield record-breaking performance in tasks such as image synthesis, video generation, and molecule design. Despite their capabilities, their efficiency, especially in the reverse process, remains a challenge due to slow convergence rates and high computational costs. We introduce an approach that leverages continuous dynamical systems to design a novel denoising network for diffusion models.
  arXiv  Detail & Related papers  (2023-10-31T00:12:14Z)
• Improving and generalizing flow-based generative models with minibatch optimal transport [90.01613198337833]
  We introduce the generalized conditional flow matching (CFM) technique for continuous normalizing flows (CNFs) CFM features a stable regression objective like that used to train the flow in diffusion models but enjoys the efficient inference of deterministic flow models. A variant of our objective is optimal transport CFM (OT-CFM), which creates simpler flows that are more stable to train and lead to faster inference.
  arXiv  Detail & Related papers  (2023-02-01T14:47:17Z)
• Diffusion-GAN: Training GANs with Diffusion [135.24433011977874]
  Generative adversarial networks (GANs) are challenging to train stably. We propose Diffusion-GAN, a novel GAN framework that leverages a forward diffusion chain to generate instance noise. We show that Diffusion-GAN can produce more realistic images with higher stability and data efficiency than state-of-the-art GANs.
  arXiv  Detail & Related papers  (2022-06-05T20:45:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.