ELF: Embedded Language Flows

• URL: http://arxiv.org/abs/2605.10938v1
• Date: Mon, 11 May 2026 17:59:29 GMT
• Title: ELF: Embedded Language Flows
• Abstract summary: We show that continuous language models can be made effective with minimal adaptation to the discrete domain.<n>We propose Embedded Language Flows (ELF), a class of diffusion models in continuous embedding space.<n>ELF substantially outperforms leading discrete and continuous DLMs, achieving better generation quality with fewer sampling steps.
• Score: 76.80499600969769
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion and flow-based models have become the de facto approaches for generating continuous data, e.g., in domains such as images and videos. Their success has attracted growing interest in applying them to language modeling. Unlike their image-domain counterparts, today's leading diffusion language models (DLMs) primarily operate over discrete tokens. In this paper, we show that continuous DLMs can be made effective with minimal adaptation to the discrete domain. We propose Embedded Language Flows (ELF), a class of diffusion models in continuous embedding space based on continuous-time Flow Matching. Unlike existing DLMs, ELF predominantly stays within the continuous embedding space until the final time step, where it maps to discrete tokens using a shared-weight network. This formulation makes it straightforward to adapt established techniques from image-domain diffusion models, e.g., classifier-free guidance (CFG). Experiments show that ELF substantially outperforms leading discrete and continuous DLMs, achieving better generation quality with fewer sampling steps. These results suggest that ELF offers a promising path toward effective continuous DLMs.

Related papers

• Continuous Diffusion Scales Competitively with Discrete Diffusion for Language [48.290726603587444]
  We construct RePlaid as a likelihood-based continuous diffusion language model (DLM)<n>We show that RePlaid exhibits a compute gap of only $20times$ compared to autoregressive models.<n>We also offer theoretical insights to understand the advantage of likelihood-based training.
  arXiv  Detail & Related papers  (2026-05-18T15:15:24Z)
• How to Train Your Latent Diffusion Language Model Jointly With the Latent Space [76.9057986588963]
  Latent diffusion models offer an attractive alternative to discrete diffusion for non-autoregressive text generation.<n>We present the Latent Diffusion Language Model (LDLM), in which the latent encoder, diffusion model, and decoder are trained jointly.<n>On OpenWebText and LM1B, LDLM achieves better generation performance than existing discrete and continuous diffusion language models.
  arXiv  Detail & Related papers  (2026-05-08T16:05:19Z)
• LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling [39.014850192441656]
  Continuous diffusion language models (DLMs) lag behind discrete counterparts due to sparse data space and underexplored design space.<n>LangFlow connects embedding-space DLMs to Flow Matching via Bregman divergence.<n>LangFlow rivals top discrete DLMs on both the perplexity (PPL) and the generative perplexity (Gen. PPL)<n>It even exceeds autoregressive baselines in zero-shot transfer on 4 out of 7 benchmarks.
  arXiv  Detail & Related papers  (2026-04-13T17:21:41Z)
• Diffusion-CAM: Faithful Visual Explanations for dMLLMs [8.78213424930081]
  Diffusion-CAM is the first interpretability method specifically tailored for dMLLMs.<n>We derive raw activation maps by differentiably probing intermediate representations in the backbone.<n>Our experiments demonstrate that Diffusion-CAM significantly outperforms SoTA methods in both localization accuracy and visual fidelity.
  arXiv  Detail & Related papers  (2026-04-13T05:14:18Z)
• One-step Language Modeling via Continuous Denoising [36.18484491074519]
  We show that language models leveraging flow-based continuous denoising can outperform discrete diffusion in both quality and speed.<n>Our work calls into question the widely held hypothesis that discrete diffusion processes are necessary for generative modeling over discrete modalities.
  arXiv  Detail & Related papers  (2026-02-18T19:23:07Z)
• Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner [66.86440230599656]
  We argue that diffusion language models do not necessarily need to be in the discrete space.<n>In particular, we prove that continuous diffusion models have stronger expressivity than discrete diffusions and looped transformers.<n>We propose Coevolutionary Continuous Diffusion (CCDD), which defines a joint multimodal diffusion process on the union of a continuous representation space and a discrete token space.
  arXiv  Detail & Related papers  (2025-10-03T17:44:41Z)
• DLM-One: Diffusion Language Models for One-Step Sequence Generation [63.43422118066493]
  DLM-One is a score-distillation-based framework for one-step sequence generation with continuous diffusion language models.<n>We investigate whether DLM-One can achieve substantial gains in sampling efficiency for language modeling.
  arXiv  Detail & Related papers  (2025-05-30T22:42:23Z)
• Multimodal Latent Language Modeling with Next-Token Diffusion [111.93906046452125]
  Multimodal generative models require a unified approach to handle both discrete data (e.g., text and code) and continuous data (e.g., image, audio, video)<n>We propose Latent Language Modeling (LatentLM), which seamlessly integrates continuous and discrete data using causal Transformers.
  arXiv  Detail & Related papers  (2024-12-11T18:57:32Z)
• A Cheaper and Better Diffusion Language Model with Soft-Masked Noise [62.719656543880596]
  Masked-Diffuse LM is a novel diffusion model for language modeling, inspired by linguistic features in languages. Specifically, we design a linguistic-informed forward process which adds corruptions to the text through strategically soft-masking to better noise the textual data. We demonstrate that our Masked-Diffuse LM can achieve better generation quality than the state-of-the-art diffusion models with better efficiency.
  arXiv  Detail & Related papers  (2023-04-10T17:58:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.