Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference

• URL: http://arxiv.org/abs/2508.02193v1
• Date: Mon, 04 Aug 2025 08:43:01 GMT
• Title: Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference
• Authors: Yuxuan Song, Zheng Zhang, Cheng Luo, Pengyang Gao, Fan Xia, Hao Luo, Zheng Li, Yuehang Yang, Hongli Yu, Xingwei Qu, Yuwei Fu, Jing Su, Ge Zhang, Wenhao Huang, Mingxuan Wang, Lin Yan, Xiaoying Jia, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Yonghui Wu, Hao Zhou,
• Abstract summary: We present Seed Diffusion Preview, a large-scale language model based on discrete-state diffusion, offering remarkably fast inference speed.<n>Thanks to non-sequential, parallel generation, discrete diffusion models provide a notable speedup to mitigate the inherent latency of token-by-token decoding.
• Score: 58.06027151683975
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Seed Diffusion Preview, a large-scale language model based on discrete-state diffusion, offering remarkably fast inference speed. Thanks to non-sequential, parallel generation, discrete diffusion models provide a notable speedup to mitigate the inherent latency of token-by-token decoding, as demonstrated recently (e.g., Mercury Coder, Gemini Diffusion). Seed Diffusion Preview achieves an inference speed of 2,146 token/s over H20 GPUs while maintaining competitive performance across a sweep of standard code evaluation benchmarks, significantly faster than contemporary Mercury and Gemini Diffusion, establishing new state of the art on the speed-quality Pareto frontier for code models.

Related papers

• Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding [36.74241893088594]
  Diffusion Large Language Models (dLLMs) offer a compelling paradigm for natural language generation.<n>Recent works have accelerated inference via KV cache reuse or decoding, but overlook the intrinsic inefficiencies within the block-wise diffusion process.<n>We propose Streaming-dLLM, a training-free framework that streamlines inference across both spatial and temporal dimensions.
  arXiv  Detail & Related papers  (2026-01-25T17:36:04Z)
• DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation [93.6273078684831]
  We propose a frequency-DeCoupled pixel diffusion framework to pursue a more efficient pixel diffusion paradigm.<n>With the intuition to decouple the generation of high and low frequency components, we leverage a lightweight pixel decoder to generate high-frequency details conditioned on semantic guidance.<n>Experiments show that DeCo achieves superior performance among pixel diffusion models, attaining FID of 1.62 (256x256) and 2.22 (512x512) on ImageNet.
  arXiv  Detail & Related papers  (2025-11-24T17:59:06Z)
• Breaking the Bottleneck with DiffuApriel: High-Throughput Diffusion LMs with Mamba Backbone [6.76700377196741]
  We introduce DiffuApriel, a masked diffusion language model built on a bidirectional Mamba backbone.<n>Our results demonstrate that bidirectional state-space architectures serve as strong denoisers in masked diffusion LMs.
  arXiv  Detail & Related papers  (2025-11-19T23:23:49Z)
• READ: Real-time and Efficient Asynchronous Diffusion for Audio-driven Talking Head Generation [55.58089937219475]
  We propose READ, the first real-time diffusion-transformer-based talking head generation framework.<n>Our approach first learns highly compressed video latent space via a VAE, significantly reducing the token count to speech generation.<n>We show that READ outperforms state-of-the-art methods by generating competitive talking head videos with significantly reduced runtime.
  arXiv  Detail & Related papers  (2025-08-05T13:57:03Z)
• CHORDS: Diffusion Sampling Accelerator with Multi-core Hierarchical ODE Solvers [72.23291099555459]
  Diffusion-based generative models have become dominant generators of high-fidelity images and videos but remain limited by their computationally expensive inference procedures.<n>This paper explores a general, training-free, and model-agnostic acceleration strategy via multi-core parallelism.<n>ChoRDS significantly accelerates sampling across diverse large-scale image and video diffusion models, yielding up to 2.1x speedup with four cores, improving by 50% over baselines, and 2.9x speedup with eight cores, all without quality degradation.
  arXiv  Detail & Related papers  (2025-07-21T05:48:47Z)
• Diffusion Buffer: Online Diffusion-based Speech Enhancement with Sub-Second Latency [14.001679439460359]
  We adapt a sliding window diffusion framework to the speech enhancement task.<n>Our approach corrupts speech signals through time, assigning more noise to frames close to the present in a buffer.<n>This marks the first practical diffusion-based solution for online speech enhancement.
  arXiv  Detail & Related papers  (2025-06-03T14:14:28Z)
• Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion [55.0194604505437]
  Speculative decoding has emerged as a widely adopted method to accelerate large language model inference.<n>This paper proposes an adaptation of speculative decoding which uses discrete diffusion models to generate draft sequences.
  arXiv  Detail & Related papers  (2024-08-10T21:24:25Z)
• Diffusion-TS: Interpretable Diffusion for General Time Series Generation [6.639630994040322]
  Diffusion-TS is a novel diffusion-based framework that generates time series samples of high quality. We train the model to directly reconstruct the sample instead of the noise in each diffusion step, combining a Fourier-based loss term. Results show that Diffusion-TS achieves the state-of-the-art results on various realistic analyses of time series.
  arXiv  Detail & Related papers  (2024-03-04T05:39:23Z)
• StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation [52.56469577812338]
  We introduce StreamDiffusion, a real-time diffusion pipeline for interactive image generation.<n>Existing diffusion models are adept at creating images from text or image prompts, yet they often fall short in real-time interaction.<n>We present a novel approach that transforms the original sequential denoising into the denoising process.
  arXiv  Detail & Related papers  (2023-12-19T18:18:33Z)
• DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models [58.450152413700586]
  We introduce a soft absorbing state that facilitates the diffusion model in learning to reconstruct discrete mutations based on the underlying Gaussian space. We employ state-of-the-art ODE solvers within the continuous space to expedite the sampling process. Our proposed method effectively accelerates the training convergence by 4x and generates samples of similar quality 800x faster.
  arXiv  Detail & Related papers  (2023-10-09T15:29:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.