Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens

• URL: http://arxiv.org/abs/2606.16847v2
• Date: Sat, 20 Jun 2026 11:33:42 GMT
• Title: Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens
• Abstract summary: We propose ASRD (Anchor Supervised Revocable Decoding), a training-free framework that operates within the embedding space.<n>We show that ASRD outperforms recent remasking baselines, achieving accuracy improvements of up to 6.4% while accelerating inference throughput by up to 7.2times$.
• Score: 22.687236570529844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion Large Language Models (dLLMs) offer a promising avenue for parallel generation but face a trade-off between decoding speed and quality. While revocable decoding strategies attempt to mitigate errors by verifying and remasking tokens, they typically operate within a mixed-quality context. This leads to two critical failures: \textit{Error Propagation}, where new tokens absorb toxic information from erroneous context, and \textit{Local Error Reinforcement}, where errors mutually reinforce each other to evade detection. To alleviate these challenges, we propose ASRD (Anchor Supervised Revocable Decoding), a training-free framework that operates within the embedding space. ASRD explicitly decouples the decoding context into trusted \textit{Anchor Tokens}, which are identified via temporal consistency, and uncertain candidates. Leveraging a dynamic Anchor Tokens Cache, we introduce two complementary mechanisms: (1) Anchor-Guided Generation, which injects entropy-weighted anchor signals into masked positions to implicitly rectify attention toward the reliable global skeleton; and (2) Anchor-Perturbed Verification, which applies orthogonal perturbations to uncertain candidate tokens, destabilizing and remasking errors driven by fragile local consensus. Extensive experiments on math and coding benchmarks demonstrate that ASRD outperforms recent remasking baselines, achieving accuracy improvements of up to 6.4\% while accelerating inference throughput by up to 7.2$\times$.

Related papers

• Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference [26.756607509798382]
  Dependency-Aware Revocable Decoding (DARD) is a training-free framework that separates tokens into masked, candidate, and unmasked states.<n>DARD consistently improves the speed-quality frontier over recent decoding methods.
  arXiv  Detail & Related papers  (2026-08-27T03:34:47Z)
• Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMs [7.894740505622926]
  Diffusion multimodal large language models (dMLLMs) produce long-form outputs marred by semantic drift and repetition.<n>We propose ours (textbfContext-textbfAware textbfCluster textbfDecoding), a training-free decoding method that scores each masked position by a multiplicative composite of softmax confidence and neighbor proximity.
  arXiv  Detail & Related papers  (2026-08-23T11:16:04Z)
• Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs [69.52853771994451]
  We analyze the decoding trajectories of LLaDA 2.0 and identify a recurring diffusion confidence trap.<n>We propose Evolutionary Decoding, a training-free test-time scaling framework that views diffusion decoding as an evolutionary process.
  arXiv  Detail & Related papers  (2026-08-01T11:48:25Z)
• NAVIRA: Decoupled Stochastic Remasking for Masked Diffusion Language Models [0.0]
  PRISM addresses this by learning token-level quality scores and remasking unreliable tokens.<n>We propose NAVIRA, an inference-time decoding policy that separates these two operations.<n>Temperature-controlled remasking reduces repeated correction of the same positions.
  arXiv  Detail & Related papers  (2026-06-04T11:24:47Z)
• Supportive Token Revealing for Fast Diffusion Language Model Decoding [10.022716564968695]
  AXON is a training-free module that can be added on top of existing parallel decoding strategies for diffusion language models.<n>We show that AXON improves the quality-latency trade-off of existing parallel decoders, often reducing the number of function evaluations while maintaining or improving accuracy.
  arXiv  Detail & Related papers  (2026-06-02T21:35:54Z)
• SimSD: Simple Speculative Decoding in Diffusion Language Models [61.33773959352141]
  Diffusion large language models (dLLMs) offer faster inference through parallel or blockwise decoding.<n>Their masked language modeling formulation remains incompatible with standard token-level speculative decoding.<n>We propose a speculative decoding algorithm for dLLMs, named SimSD, that equips dLLMs with temporally valid token-level contexts.<n>Our method achieves up to 7.46x higher decoding throughput while maintaining and even improving average generation quality.
  arXiv  Detail & Related papers  (2026-06-01T17:46:46Z)
• Stability-Weighted Decoding for Diffusion Language Models [8.670026899042483]
  Existing decoding strategies rely on static confidence metrics computed at a single denoising step, ignoring temporal history and often leading to premature unmasking of unstable tokens.<n>We propose Stability-Weighted Decoding (SWD), a training-free, plug-and-play strategy that incorporates temporal stability into token scoring and acts as a universal modulator for arbitrary score-based decoding policies.
  arXiv  Detail & Related papers  (2026-04-18T17:04:10Z)
• Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference [58.189320101488725]
  DLLMs promise fast non-autoregressive inference but suffer a severe quality-speed trade-off in parallel decoding.<n>We address this by integrating continuous representations into the discrete decoding process, as they preserve rich inter-position dependency.<n>We propose ReMix, a framework that introduces a novel Continuous Mixing State as an intermediate between the initial masked state and the final decoded token state.
  arXiv  Detail & Related papers  (2026-02-26T11:08:11Z)
• Anchored Decoding: Provably Reducing Copyright Risk for Any Language Model [99.16364381244445]
  Modern language models (LMs) tend to memorize portions of their training data and emit verbatim spans.<n>We propose Anchored Decoding, a plug-and-play inference-time method for suppressing verbatim copying.<n>We evaluate our methods across six model pairs on long-form evaluations of copyright risk and utility.
  arXiv  Detail & Related papers  (2026-02-06T19:00:14Z)
• DAWN: Dependency-Aware Fast Inference for Diffusion LLMs [25.499655803741103]
  Diffusion large language models (dLLMs) have shown advantages in text generation.<n>Existing inference solutions adopt conservative parallel strategies, leaving substantial efficiency potential underexplored.<n>We propose DAWN, a training-free, dependency-aware decoding method for fast dLLM inference.
  arXiv  Detail & Related papers  (2026-02-06T18:51:29Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• Residual Context Diffusion Language Models [90.07635240595926]
  Residual Context Diffusion (RCD) is a module that converts discarded token representations into contextual residuals and injects them back for the next denoising step.<n>RCD consistently improves frontier dLLMs by 5-10 points in accuracy with minimal extra computation overhead.
  arXiv  Detail & Related papers  (2026-01-30T13:16:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.