Serving Masked Diffusion LLMs: Characterization and Design Principles from Real Hardware

• URL: http://arxiv.org/abs/2608.23807v1
• Date: Mon, 24 Aug 2026 20:16:48 GMT
• Title: Serving Masked Diffusion LLMs: Characterization and Design Principles from Real Hardware
• Abstract summary: Masked diffusion language models (dLLMs) can generate text faster than autoregressive (AR) models, since they denoise many tokens at once.<n>Recent systems have begun building serving infrastructure for dLLMs, but none first measure how these models behave under real, concurrent serving.<n>We characterize dLLM serving to close this gap, using LLaDA-8B-Instruct with a D2F (Discrete Diffusion Forcing) LoRA adapter on a single NVIDIA H200 GPU.
• Score: 1.7286026533711227
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Masked diffusion language models (dLLMs) can in principle generate text faster than autoregressive (AR) models, since they denoise many tokens at once. Recent systems have begun building serving infrastructure for dLLMs, but none first measure how these models behave under real, concurrent serving load. Serving systems built without this grounding risk carrying over assumptions from AR serving that may not hold for dLLMs. We characterize dLLM serving to close this gap, using LLaDA-8B-Instruct with a D2F (Discrete Diffusion Forcing) LoRA adapter on a single NVIDIA H200 GPU, evaluated on GSM8K and HumanEval. We report three findings. First, request difficulty, the number of denoising steps a request needs, is discrete rather than continuous: requests fall into 11 fixed step-count levels (178 + 29k), and no signal we test predicts the level before generation starts (best R2 = 0.150). Second, benchmarks with short generation budgets below 320 tokens understate serving variance, since requests are cut off before the latency spread appears. Third, only 24% of single-request wall-clock time is GPU computation; the rest is CPU-side dispatch overhead. Batching mainly helps by amortizing this overhead: sharing one forward pass per denoising step improves throughput by 16.0x at batch size 16 over a per-request-dispatch baseline. We also argue structurally that output quality should not degrade with batch size, stating three assumptions this rests on; we measure 74 to 76% GSM8K accuracy at single-request scale. Finally, we derive a batch-timeout rule for fixed-fill synchronized batching under Poisson arrivals. Together, these results show that serving diffusion language models needs parallelism at the level of each denoising step, which differs from AR serving in how admission and eviction interact with an already shared forward pass.

Related papers

• Unlocking Lossless Speedups in LLMs via Discrete Diffusion [67.4336730000437]
  Large Language Models (LLMs) owe much of their success to next-token prediction (NTP)<n>We introduce diffusion-augmented LLMs, a new class of models that defines an AR model distribution.<n>We decouple the parameters of these models into two sets: AR weights, trained using the standard NTP objective, and lightweight diffusion weights, trained to generate multiple tokens simultaneously.
  arXiv  Detail & Related papers  (2026-09-03T15:48:43Z)
• Retrofitting Linear Attention into Diffusion Language Models [8.41282954657158]
  Diffusion language models (dLLMs) offer a promising alternative to autoregressive models by accelerating inference through parallel decoding.<n>Recent dLLMs commonly use blockwise semi-autoregressive decoding, generating blocks autoregressively while denoising tokens within each active block in parallel.<n>We introduce block-hybrid attention, which retains exact softmax attention within the active denoising block while applying linear attention over previous blocks.
  arXiv  Detail & Related papers  (2026-08-06T22:34:20Z)
• An Efficient vLLM-Based Inference Pipeline for Unified Audio Understanding and Generation [63.50393485685279]
  We present a vLLM-based inference pipeline for unified speech understanding and generation.<n>We overcome the shared intuition-Free Guidance (CFG) halves throughput.<n>By co-scheduling paired conditional and unconditional requests within a continuous batch, our CFG implementation sustains 80% of non-CFG throughput.
  arXiv  Detail & Related papers  (2026-07-02T12:55:11Z)
• ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping [26.560813832545563]
  Diffusion large language models (dLLMs) are emerging as a promising alternative to autoregressive models (ARMs)<n>We analyze the generation dynamics of dLLMs and find that intermediate representations, including key, value, and hidden states, change only subtly across successive iterations.<n>We propose textbfES-dLLM, a training-free inference acceleration framework for dLLM that reduces computation by skipping tokens in early layers based on the estimated importance.
  arXiv  Detail & Related papers  (2026-03-10T14:31:19Z)
• EDIT: Early Diffusion Inference Termination for dLLMs Based on Dynamics of Training Gradients [6.736735746633275]
  Diffusion-based large language models (dLLMs) refine token generations through iterative denoising, but answers often stabilize before all steps complete.<n>We propose EDIT, an inference-time criterion that adaptively stops denoising once sufficient reasoning stability relative to training-time reasoning is detected.
  arXiv  Detail & Related papers  (2025-11-29T23:47:47Z)
• Diffusion Language Models are Super Data Learners [61.721441061210896]
  When unique data is limited, diffusion language models (DLMs) consistently surpass autoregressive (AR) models by training for more epochs.<n>We attribute the gains to three compounding factors: (1) any-order modeling, (2) super-dense compute from iterative bidirectional denoising, and (3) built-in Monte Carlo augmentation.
  arXiv  Detail & Related papers  (2025-11-05T08:17:42Z)
• dParallel: Learnable Parallel Decoding for dLLMs [77.24184219948337]
  Diffusion large language models (dLLMs) offer parallel token prediction and lower inference latency.<n>Existing open-source models still require nearly token-length decoding steps to ensure performance.<n>We introduce dParallel, a simple and effective method that unlocks the inherent parallelism of dLLMs for fast sampling.
  arXiv  Detail & Related papers  (2025-09-30T16:32:52Z)
• SparseD: Sparse Attention for Diffusion Language Models [98.05780626106555]
  diffusion language models (DLMs) offer a promising alternative to autoregressive models (ARs)<n>Existing open-source DLMs suffer from high inference latency.<n>We propose SparseD, a novel sparse attention method for DLMs.
  arXiv  Detail & Related papers  (2025-09-28T18:10:10Z)
• Diffusion Language Models Know the Answer Before Decoding [56.96815863705218]
  Diffusion language models (DLMs) have emerged as an alternative to autoregressive approaches.<n>Our work highlights and leverage an overlooked property of DLMs early answer convergence.<n>We introduce Prophet, a training-free fast decoding paradigm that enables early commit decoding.
  arXiv  Detail & Related papers  (2025-08-27T15:40:25Z)
• Efficient Depth Estimation for Unstable Stereo Camera Systems on AR Glasses [1.086544864007391]
  We develop approaches to the rectification and cost volume that consider ML acceleration (GPU and NPUs) in recent hardware.<n>Based on our approaches, we develop MultiHeadDepth and HomoDepth.<n>HomoDepth, which can directly process unrectified images, reduces the end-to-end latency by 44.5%.
  arXiv  Detail & Related papers  (2024-11-15T07:43:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.