CAI-DLLM: Convergence Aware Inference for Diffusion Language Models

• URL: http://arxiv.org/abs/2608.22646v1
• Date: Sun, 23 Aug 2026 23:03:59 GMT
• Title: CAI-DLLM: Convergence Aware Inference for Diffusion Language Models
• Abstract summary: Diffusion language models can generate many tokens in parallel, but they still require repeated denoising steps during inference.<n>We propose CAI-DLLM, a training-free inference method that uses first-step confidence to guide denoising and reduce inference time.<n>We evaluate CAI-DLLM on LLaDA-8B-Instruct and Dream-7B-Instruct across math, code, reasoning, commonsense, and long-context tasks.
• Score: 1.8883742168400344
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion language models can generate many tokens in parallel, but they still require repeated denoising steps during inference. This makes generation costly, especially when the model continues to recompute tokens that are already stable. To address these limitations, we propose CAI-DLLM, a training-free inference method that uses first-step confidence to guide denoising and reduce inference time. Specifically, CAI-DLLM commits easy tokens earlier, allocates more denoising steps to harder tokens, and adjusts decoding schedules across output blocks. As it relies only on first-step confidence signals, it does not require retraining, extra predictors, or weight updates. We evaluate CAI-DLLM on LLaDA-8B-Instruct and Dream-7B-Instruct across math, code, reasoning, commonsense, and long-context tasks. CAI-DLLM achieves up to 18.2x wall clock inference speedup on LLaDA GSM8K while improving accuracy from 76.27% to 77.41%, and up to 13.1x speedup on Dream HumanEval while achieving higher pass@1 than no-cache inference, 48.17% compared with 46.95%. On harder reasoning tasks, speedups reach 44.8x, with a largest accuracy drop of 4.4 points, while energy consumption is reduced by up to 95.3%.

Related papers

• Retrofitting Linear Attention into Diffusion Language Models [8.41282954657158]
  Diffusion language models (dLLMs) offer a promising alternative to autoregressive models by accelerating inference through parallel decoding.<n>Recent dLLMs commonly use blockwise semi-autoregressive decoding, generating blocks autoregressively while denoising tokens within each active block in parallel.<n>We introduce block-hybrid attention, which retains exact softmax attention within the active denoising block while applying linear attention over previous blocks.
  arXiv  Detail & Related papers  (2026-08-06T22:34:20Z)
• CORA-Diff: Confidence-Oriented Residual Acceptance for Efficient Diffusion Language Model Inference [24.176666448303095]
  Diffusion language models update many tokens in parallel, yet practical decoders often use a fixed denoising horizon.<n>We propose CORA-Diff, a training-free method that preserves the original transfer rule and applies confidence-and-persistence gating only to positions that rule leaves unresolved.<n>Results show that native confidence and persistence enable reliable residual acceptance, reducing repeated denoising while preserving task quality.
  arXiv  Detail & Related papers  (2026-07-31T07:32:10Z)
• Improved Large Language Diffusion Models [115.83169414681767]
  We present emphiLLaDA, an 8B masked diffusion language model trained from scratch with fully bidirectional attention.<n>iLLaDA keeps the masked diffusion objective throughout pre-training and supervised fine-tuning (SFT), scaling pre-training to 12T tokens and fine-tuning on a 25B-token instruction corpus for 12 epochs.
  arXiv  Detail & Related papers  (2026-06-24T02:51:36Z)
• SAID: Accelerating Diffusion-Based Language Models via Scaffold-Aware Iterative Decoding [11.233923872369202]
  Diffusion large language models (DLLMs) enable non-autoregressive computation generation by iteratively denoising corrupted tokens with bidirectional context.<n>We propose SAID, a Scaffold-Aware Iterative Decoding framework that acceleratesDLLMs by reallocating across tokens.<n>Experiments on LLaDA-8B and LLaDA 1.5 across math, coding, and knowledge benchmarks show that SAID significantly accelerates inference with a maximum speedup of 9.1x while maintaining competitive performance.
  arXiv  Detail & Related papers  (2026-06-03T14:56:42Z)
• Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers [76.15132587294862]
  Wide-In, Narrow-Out (WINO) is a training-free decoding algorithm that enables revokable parallel generation.<n>WINO+ injects the verified denoising trajectories produced by WINO into model parameters, aligning training with efficient inference.<n>Experiments on LLaDA and MMaDA show that WINO improves both quality and efficiency, while WINO+ further strengthens this progression.
  arXiv  Detail & Related papers  (2026-05-16T11:27:40Z)
• LoSA: Locality Aware Sparse Attention for Block-Wise Diffusion Language Models [49.93891888238178]
  Block-wise diffusion language models (DLMs) generate multiple tokens in any order, offering a promising alternative to the autoregressive decoding pipeline.<n>Naive sparse attention fails on DLMs due to a KV Inflation problem, where different queries select different prefix positions.<n>We propose LOSA (Locality-aware Sparse Attention), which reuses cached prefix-attention results for stable tokens and applies sparse attention only to active tokens.
  arXiv  Detail & Related papers  (2026-04-13T20:53:51Z)
• dParallel: Learnable Parallel Decoding for dLLMs [77.24184219948337]
  Diffusion large language models (dLLMs) offer parallel token prediction and lower inference latency.<n>Existing open-source models still require nearly token-length decoding steps to ensure performance.<n>We introduce dParallel, a simple and effective method that unlocks the inherent parallelism of dLLMs for fast sampling.
  arXiv  Detail & Related papers  (2025-09-30T16:32:52Z)
• Diffusion Language Models Know the Answer Before Decoding [56.96815863705218]
  Diffusion language models (DLMs) have emerged as an alternative to autoregressive approaches.<n>Our work highlights and leverage an overlooked property of DLMs early answer convergence.<n>We introduce Prophet, a training-free fast decoding paradigm that enables early commit decoding.
  arXiv  Detail & Related papers  (2025-08-27T15:40:25Z)
• R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning [80.104336426172]
  Chain-of-thought (CoT) enhances problem-solving ability of large language models.<n>CoT incurs substantial inference cost due to long autoregressive trajectories.<n>We introduce R-Stitch, a training-free hybrid decoding framework.
  arXiv  Detail & Related papers  (2025-07-23T08:14:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.