LLaDA2.1: Speeding Up Text Diffusion via Token Editing

• URL: http://arxiv.org/abs/2602.08676v3
• Date: Fri, 13 Feb 2026 13:19:06 GMT
• Title: LLaDA2.1: Speeding Up Text Diffusion via Token Editing
• Abstract summary: We unveil LLaDA2.1, a paradigm shift designed to transcend the trade-off between decoding speed and generation quality.<n>By seamlessly weaving Token-to-Token (T2T) editing into the conventional Mask-to-Token (M2T) scheme, we introduce a joint, threshold-decoding scheme.<n>This structural innovation gives rise to two distinct personas: the Speedy Mode (S Mode), which audaciously lowers the M2T threshold to bypass traditional constraints while relying on T2T to refine the output; and the Quality Mode (Q Mode), which leans into conservative thresholds to secure superior benchmark
• Score: 72.92893150459909
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While LLaDA2.0 showcased the scaling potential of 100B-level block-diffusion models and their inherent parallelization, the delicate equilibrium between decoding speed and generation quality has remained an elusive frontier. Today, we unveil LLaDA2.1, a paradigm shift designed to transcend this trade-off. By seamlessly weaving Token-to-Token (T2T) editing into the conventional Mask-to-Token (M2T) scheme, we introduce a joint, configurable threshold-decoding scheme. This structural innovation gives rise to two distinct personas: the Speedy Mode (S Mode), which audaciously lowers the M2T threshold to bypass traditional constraints while relying on T2T to refine the output; and the Quality Mode (Q Mode), which leans into conservative thresholds to secure superior benchmark performances with manageable efficiency degrade. Furthering this evolution, underpinned by an expansive context window, we implement the first large-scale Reinforcement Learning (RL) framework specifically tailored for dLLMs, anchored by specialized techniques for stable gradient estimation. This alignment not only sharpens reasoning precision but also elevates instruction-following fidelity, bridging the chasm between diffusion dynamics and complex human intent. We culminate this work by releasing LLaDA2.1-Mini (16B) and LLaDA2.1-Flash (100B). Across 33 rigorous benchmarks, LLaDA2.1 delivers strong task performance and lightning-fast decoding speed. Despite its 100B volume, on coding tasks it attains an astounding 892 TPS on HumanEval+, 801 TPS on BigCodeBench, and 663 TPS on LiveCodeBench.

Related papers

• Trajectory-Level Speculative Decoding for Diffusion Language Models [16.97093186372498]
  Diffusion-based language models (dLLMs) enable parallel token generation through iterative denoising, but existing decoding strategies collapse to single-token generation under low confidence.<n>We develop a trajectory-level speculative framework that constructs draft denoising trajectories via confidence-stratified tree exploration.<n>Our framework reduces denoising by 30-40% and increases tokens-per-step from 2.6 to 4.3, achieving 7-14x speedup over vanilla dLLMs and 1.3x over Fast-dLLM with less than 1% accuracy change across reasoning and code benchmarks.
  arXiv  Detail & Related papers  (2026-08-27T09:42:20Z)
• DiffusionGemma Technical Report [72.10570774996219]
  We introduce DiffusionGemma, a language model that uses discrete diffusion to generate text at exceptionally high speed.<n>Rather than decoding one token at a time, DiffusionGemma iteratively refines blocks of 256 tokens in parallel.<n>Our compute-efficient two-stage training pipeline uses fewer than 10% of the starting AR model's total training token budget.
  arXiv  Detail & Related papers  (2026-07-31T16:11:46Z)
• SAID: Accelerating Diffusion-Based Language Models via Scaffold-Aware Iterative Decoding [11.233923872369202]
  Diffusion large language models (DLLMs) enable non-autoregressive computation generation by iteratively denoising corrupted tokens with bidirectional context.<n>We propose SAID, a Scaffold-Aware Iterative Decoding framework that acceleratesDLLMs by reallocating across tokens.<n>Experiments on LLaDA-8B and LLaDA 1.5 across math, coding, and knowledge benchmarks show that SAID significantly accelerates inference with a maximum speedup of 9.1x while maintaining competitive performance.
  arXiv  Detail & Related papers  (2026-06-03T14:56:42Z)
• DMax: Aggressive Parallel Decoding for dLLMs [77.24184219948337]
  We present DMax, a new paradigm for efficient diffusion language models (dLLMs)<n>It mitigates error accumulation in parallel decoding, enabling aggressive decoding while preserving generation quality.<n>At the core of our approach is On-Policy Uniform Training, a novel training strategy that efficiently unifies masked and uniform dLLMs.
  arXiv  Detail & Related papers  (2026-04-09T14:35:42Z)
• S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation [22.303253139413286]
  Block-diffusion language models combine block-wise autoregressive decoding with within-block parallel denoising.<n>Existing approaches that address this issue either require additional training or incur extra test-time compute.<n>We present S2D2, a training-free self-speculative decoding framework for block-diffusion language models.
  arXiv  Detail & Related papers  (2026-03-26T17:48:50Z)
• Mixture-of-Depths Attention [65.80640499676542]
  Scaling depth is a key driver for large language models (LLMs)<n>We introduce mixture-of-depths attention (MoDA)<n>MoDA allows each attention head to attend to sequence KV pairs at the current layer and depth KV pairs from preceding layers.
  arXiv  Detail & Related papers  (2026-03-16T17:59:55Z)
• DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference [27.204773545145326]
  DART is a speculative decoding framework for large language models (dLLMs)<n>It leverages parallel generation to reduce drafting latency.<n>It achieves a 2.03x--3.44x wall-clock time speedup across multiple datasets.
  arXiv  Detail & Related papers  (2026-01-27T07:04:24Z)
• LLaDA2.0: Scaling Up Diffusion Language Models to 100B [96.84156938318931]
  We present LLaDA2.0 -- a discrete diffusion large language models (dLLM) scaling up to 100B total parameters.<n>LLaDA2.0 upholds knowledge inheritance, progressive adaption and efficiency-aware design principle.<n>We obtain LLaDA2.0-mini (16B) and LLaDA2.0-flash (100B), two instruction-tuned Mixture-of-Experts (MoE) variants optimized for practical deployment.
  arXiv  Detail & Related papers  (2025-12-10T09:26:18Z)
• Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model [98.35868970993232]
  Diffusion language models (DLMs) are emerging as a powerful and promising alternative to the dominant autoregressive paradigm.<n>We introduce efficient Sampling with Adaptive acceleration and Backtracking Enhanced Remasking (i.e., Saber) to achieve better inference speed and output quality in code generation.
  arXiv  Detail & Related papers  (2025-10-20T23:38:12Z)
• dParallel: Learnable Parallel Decoding for dLLMs [77.24184219948337]
  Diffusion large language models (dLLMs) offer parallel token prediction and lower inference latency.<n>Existing open-source models still require nearly token-length decoding steps to ensure performance.<n>We introduce dParallel, a simple and effective method that unlocks the inherent parallelism of dLLMs for fast sampling.
  arXiv  Detail & Related papers  (2025-09-30T16:32:52Z)
• Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs [57.69190972274813]
  Diffusion Large Language Models (DLLMs) have emerged as a compelling alternative to Autoregressive models.<n>ExistingDLLMs are plagued by a severe quality-speed trade-off, where faster parallel decoding leads to significant performance degradation.<n>We introduce Wide-In, Narrow-Out (WINO), a training-free decoding algorithm that enables revokable decoding inDLLMs.
  arXiv  Detail & Related papers  (2025-07-24T16:51:33Z)
• Falcon: Faster and Parallel Inference of Large Language Models through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree [7.438117410146904]
  Falcon is an innovative speculative decoding framework fashioned to augment both the drafter's parallelism and output quality.<n>Falcon incorporates the Coupled Sequential Glancing Distillation technique, which fortifies inter-token dependencies within the same block, leading to increased speculation accuracy.
  arXiv  Detail & Related papers  (2024-12-17T08:02:08Z)
• Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion [55.0194604505437]
  Speculative decoding has emerged as a widely adopted method to accelerate large language model inference.<n>This paper proposes an adaptation of speculative decoding which uses discrete diffusion models to generate draft sequences.
  arXiv  Detail & Related papers  (2024-08-10T21:24:25Z)
• BiTA: Bi-Directional Tuning for Lossless Acceleration in Large Language Models [37.09385961422664]
  Large language models (LLMs) commonly employ autoregressive generation during inference, leading to high memory bandwidth demand and extended latency. We present Bi-directional Tuning for lossless Acceleration (BiTA), an innovative method expediting LLMs via streamlined semi-autoregressive generation and draft verification. The proposed BiTA, LLaMA-2-70B-Chat achieves a 2.7$times$ speedup on the MT-Bench benchmark.
  arXiv  Detail & Related papers  (2024-01-23T06:36:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.