FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation

• URL: http://arxiv.org/abs/2605.20199v1
• Date: Mon, 06 Apr 2026 10:36:22 GMT
• Title: FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation
• Abstract summary: FlowLM is a flow matching language model transformed from pre-trained diffusion language models.<n>Finetuned FlowLM reaches performance saturation with only half as many training epochs as training from scratch.
• Score: 40.51046085293661
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present FlowLM, a flow matching language model transformed from pre-trained diffusion language models via efficient fine-tuning. By re-aligning the curved sampling trajectories of diffusion models into straight-line flows, FlowLM enables high quality few-step generation that rivals or even outperforms the quality of 2,000-step diffusion sampling with very few training epochs. Remarkably, finetuned FlowLM reaches performance saturation with only half as many training epochs as training from scratch, both approaches greatly outperforming the original diffusion model, thereby validating our method. Furthermore, we validate a more effective training objective for flow matching: predicting clean data to consistently guide the sampling process towards the true data distribution. Empirical results demonstrate that our approach is highly effective for high-quality, few-step text generation.

Related papers

• One-Step Generative Modeling via Wasserstein Gradient Flows [56.266175641614986]
  W-Flow is a framework for training a generator that transforms samples from a simple reference distribution into samples from a target data distribution in a single step.<n>We set a new state of the art for one-step ImageNet 256$times$256 generation, achieving 1.29 FID, with improved mode coverage and domain transfer.
  arXiv  Detail & Related papers  (2026-05-12T08:29:44Z)
• Discrete Meanflow Training Curriculum [11.664282029505353]
  One-step generative models can produce high quality image samples but can be difficult to optimize.<n>Meanflow models exhibit excellent few-step sampling performance and tantalizing one-step sampling performance.
  arXiv  Detail & Related papers  (2026-04-10T00:25:44Z)
• One-step Language Modeling via Continuous Denoising [36.18484491074519]
  We show that language models leveraging flow-based continuous denoising can outperform discrete diffusion in both quality and speed.<n>Our work calls into question the widely held hypothesis that discrete diffusion processes are necessary for generative modeling over discrete modalities.
  arXiv  Detail & Related papers  (2026-02-18T19:23:07Z)
• Align Your Flow: Scaling Continuous-Time Flow Map Distillation [63.927438959502226]
  Flow maps connect any two noise levels in a single step and remain effective across all step counts.<n>We extensively validate our flow map models, called Align Your Flow, on challenging image generation benchmarks.<n>We show text-to-image flow map models that outperform all existing non-adversarially trained few-step samplers in text-conditioned synthesis.
  arXiv  Detail & Related papers  (2025-06-17T15:06:07Z)
• ProReflow: Progressive Reflow with Decomposed Velocity [52.249464542399636]
  Flow matching aims to reflow the diffusion process of diffusion models into a straight line for a few-step and even one-step generation.<n>We introduce progressive reflow, which progressively reflows the diffusion models in local timesteps until the whole diffusion progresses.<n>We also introduce aligned v-prediction, which highlights the importance of direction matching in flow matching over magnitude matching.
  arXiv  Detail & Related papers  (2025-03-05T04:50:53Z)
• Self-Corrected Flow Distillation for Consistent One-Step and Few-Step Text-to-Image Generation [3.8959351616076745]
  Flow matching has emerged as a promising framework for training generative models.<n>We introduce a self-corrected flow distillation method that integrates consistency models and adversarial training.<n>This work is a pioneer in achieving consistent generation quality in both few-step and one-step sampling.
  arXiv  Detail & Related papers  (2024-12-22T07:48:49Z)
• Your Diffusion Model is Secretly a Noise Classifier and Benefits from Contrastive Training [20.492630610281658]
  Diffusion models learn to denoise data and the trained denoiser is then used to generate new samples from the data distribution. We introduce a new self-supervised training objective that differentiates the levels of noise added to a sample. We show by diverse experiments that the proposed contrastive diffusion training is effective for both sequential and parallel settings.
  arXiv  Detail & Related papers  (2024-07-12T03:03:50Z)
• Language Rectified Flow: Advancing Diffusion Language Generation with Probabilistic Flows [53.31856123113228]
  This paper proposes Language Rectified Flow (ours) Our method is based on the reformulation of the standard probabilistic flow models. Experiments and ablation studies demonstrate that our method can be general, effective, and beneficial for many NLP tasks.
  arXiv  Detail & Related papers  (2024-03-25T17:58:22Z)
• Guided Flows for Generative Modeling and Decision Making [55.42634941614435]
  We show that Guided Flows significantly improves the sample quality in conditional image generation and zero-shot text synthesis-to-speech. Notably, we are first to apply flow models for plan generation in the offline reinforcement learning setting ax speedup in compared to diffusion models.
  arXiv  Detail & Related papers  (2023-11-22T15:07:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.