Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference

• URL: http://arxiv.org/abs/2608.26574v1
• Date: Thu, 27 Aug 2026 03:34:47 GMT
• Title: Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference
• Abstract summary: Dependency-Aware Revocable Decoding (DARD) is a training-free framework that separates tokens into masked, candidate, and unmasked states.<n>DARD consistently improves the speed-quality frontier over recent decoding methods.
• Score: 26.756607509798382
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion large language models (dLLMs) offer a promising alternative to autoregressive generation by decoding multiple tokens in parallel through iterative denoising. However, increasing decoding parallelism often degrades generation quality, as early errors can contaminate later contexts. Revocable decoding mitigates this issue by re-evaluating decoded tokens and remasking unreliable ones, but existing methods overlook that unreliable tokens may also corrupt the verification context itself. We identify this failure mode and propose Dependency-Aware Revocable Decoding (DARD), a training-free framework that separates tokens into masked, candidate, and unmasked states. DARD verifies candidate tokens using a selective context that excludes less reliable tokens and adaptively regulates their influence on subsequent decoding. Experiments across 12 textual and multimodal benchmarks on 3 open-source dLLMs show that DARD consistently improves the speed-quality Pareto frontier over recent revocable decoding methods, achieving a 2.71$\times$ speedup and a 4.35-point CIDEr score gain over Saber on Flickr30K.

Related papers

• Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens [22.687236570529844]
  We propose ASRD (Anchor Supervised Revocable Decoding), a training-free framework that operates within the embedding space.<n>We show that ASRD outperforms recent remasking baselines, achieving accuracy improvements of up to 6.4% while accelerating inference throughput by up to 7.2times$.
  arXiv  Detail & Related papers  (2026-06-15T15:23:47Z)
• Supportive Token Revealing for Fast Diffusion Language Model Decoding [10.022716564968695]
  AXON is a training-free module that can be added on top of existing parallel decoding strategies for diffusion language models.<n>We show that AXON improves the quality-latency trade-off of existing parallel decoders, often reducing the number of function evaluations while maintaining or improving accuracy.
  arXiv  Detail & Related papers  (2026-06-02T21:35:54Z)
• SimSD: Simple Speculative Decoding in Diffusion Language Models [61.33773959352141]
  Diffusion large language models (dLLMs) offer faster inference through parallel or blockwise decoding.<n>Their masked language modeling formulation remains incompatible with standard token-level speculative decoding.<n>We propose a speculative decoding algorithm for dLLMs, named SimSD, that equips dLLMs with temporally valid token-level contexts.<n>Our method achieves up to 7.46x higher decoding throughput while maintaining and even improving average generation quality.
  arXiv  Detail & Related papers  (2026-06-01T17:46:46Z)
• Diffusion Large Language Models for Visual Speech Recognition [56.81307584718608]
  We propose the first Diffusion Large Language Model (DLLM)-based Visual Speech Recognition (VSR) framework.<n>DLLM-VSR commits high-confidence positions early and uses the committed tokens as bidirectional context to refine ambiguous ones.<n>We develop length-guided candidate decoding, which uses video duration to construct plausible transcript-length hypotheses.<n>The proposed method achieves a state-of-the-art WER of 19.5% on LRS3 using only its labeled training data.
  arXiv  Detail & Related papers  (2026-05-27T13:22:08Z)
• Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers [76.15132587294862]
  Wide-In, Narrow-Out (WINO) is a training-free decoding algorithm that enables revokable parallel generation.<n>WINO+ injects the verified denoising trajectories produced by WINO into model parameters, aligning training with efficient inference.<n>Experiments on LLaDA and MMaDA show that WINO improves both quality and efficiency, while WINO+ further strengthens this progression.
  arXiv  Detail & Related papers  (2026-05-16T11:27:40Z)
• PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding [32.667256256847246]
  Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences.<n>We propose Parallel Speculative Decoding (PSD), a training-free framework that jointly improves inference along both axes.
  arXiv  Detail & Related papers  (2026-05-15T04:43:02Z)
• Residual Context Diffusion Language Models [90.07635240595926]
  Residual Context Diffusion (RCD) is a module that converts discarded token representations into contextual residuals and injects them back for the next denoising step.<n>RCD consistently improves frontier dLLMs by 5-10 points in accuracy with minimal extra computation overhead.
  arXiv  Detail & Related papers  (2026-01-30T13:16:32Z)
• Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models [47.5976588836299]
  Diffusion large language models (dLLMs) offer advantages such as accelerated parallel decoding and bidirectional context modeling.<n>The vanilla decoding strategy in discrete dLLMs suffers from a critical limitation: once a token is accepted, it can no longer be revised in subsequent steps.<n>We propose Tolerator, a training-free decoding strategy that leverages cross-validation among predicted tokens.
  arXiv  Detail & Related papers  (2025-10-06T17:56:46Z)
• Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMs [57.27982780697922]
  Large language models have demonstrated exceptional capability in natural language understanding and generation. However, their generation speed is limited by the inherently sequential nature of their decoding process. This paper introduces Lexical Unit Decoding, a novel decoding methodology implemented in a data-driven manner.
  arXiv  Detail & Related papers  (2024-05-24T04:35:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.