STDec: Spatio-Temporal Stability Guided Decoding for dLLMs

• URL: http://arxiv.org/abs/2604.06330v1
• Date: Tue, 07 Apr 2026 18:13:31 GMT
• Title: STDec: Spatio-Temporal Stability Guided Decoding for dLLMs
• Abstract summary: Large Diffusion Language Models (dLLMs) have achieved rapid progress, viewed as a promising alternative to the autoregressive paradigm.<n>Most dLLM decoders still adopt a global confidence threshold, and do not model local context from neighboring decoded states or temporal consistency of predicted token IDs.<n>We observe strong stability neartemporal stability in dLLM decoding, and our approach includes spatial-aware decoding and temporal-aware decoding.
• Score: 49.55447757907809
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion Large Language Models (dLLMs) have achieved rapid progress, viewed as a promising alternative to the autoregressive paradigm. However, most dLLM decoders still adopt a global confidence threshold, and do not explicitly model local context from neighboring decoded states or temporal consistency of predicted token IDs across steps. To address this issue, we propose a simple spatio-temporal stability guided decoding approach, named STDec. We observe strong spatio-temporal stability in dLLM decoding: newly decoded tokens tend to lie near decoded neighbors, and their predicted IDs often remain consistent across several denoising steps. Inspired by this stability, our STDec includes spatial-aware decoding and temporal-aware decoding. The spatial-aware decoding dynamically generates the token-adaptive threshold by aggregating the decoded states of nearby tokens. The temporal-aware decoding relaxes the decoding thresholds for tokens whose predicted token IDs remain consistent over denoising steps. Our STDec is training-free and remains compatible with cache-based acceleration methods. Across textual reasoning and multimodal understanding benchmarks, STDec substantially improves throughput while maintaining comparable task performance score. Notably, on MBPP with LLaDA, STDec achieves up to 14.17x speedup with a comparable score. Homepage: https://yzchen02.github.io/STDec.

Related papers

• SAID: Accelerating Diffusion-Based Language Models via Scaffold-Aware Iterative Decoding [11.233923872369202]
  Diffusion large language models (DLLMs) enable non-autoregressive computation generation by iteratively denoising corrupted tokens with bidirectional context.<n>We propose SAID, a Scaffold-Aware Iterative Decoding framework that acceleratesDLLMs by reallocating across tokens.<n>Experiments on LLaDA-8B and LLaDA 1.5 across math, coding, and knowledge benchmarks show that SAID significantly accelerates inference with a maximum speedup of 9.1x while maintaining competitive performance.
  arXiv  Detail & Related papers  (2026-06-03T14:56:42Z)
• SimSD: Simple Speculative Decoding in Diffusion Language Models [61.33773959352141]
  Diffusion large language models (dLLMs) offer faster inference through parallel or blockwise decoding.<n>Their masked language modeling formulation remains incompatible with standard token-level speculative decoding.<n>We propose a speculative decoding algorithm for dLLMs, named SimSD, that equips dLLMs with temporally valid token-level contexts.<n>Our method achieves up to 7.46x higher decoding throughput while maintaining and even improving average generation quality.
  arXiv  Detail & Related papers  (2026-06-01T17:46:46Z)
• Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation [15.129362213838974]
  We propose a trace-aware decoding framework with two components.<n>First, Temporal-Spatial Parallel Decoding (TSPD) uses a lightweight temporalspatial controller that consumes per-token trajectory features, including confidence, entropy, and momentum, together with token position, to decide when a token has converged and can be safely fixed.<n>Second, we introduce Confidence Extrapolation (CE), a training-free state-space module that forecasts future logit trends with uncertainty to support proactive decisions.
  arXiv  Detail & Related papers  (2026-05-29T02:29:28Z)
• Stability-Weighted Decoding for Diffusion Language Models [8.670026899042483]
  Existing decoding strategies rely on static confidence metrics computed at a single denoising step, ignoring temporal history and often leading to premature unmasking of unstable tokens.<n>We propose Stability-Weighted Decoding (SWD), a training-free, plug-and-play strategy that incorporates temporal stability into token scoring and acts as a universal modulator for arbitrary score-based decoding policies.
  arXiv  Detail & Related papers  (2026-04-18T17:04:10Z)
• Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models [56.91801348360746]
  Large audio-language models (LALMs) generalize across speech, sound, and music.<n> unified decoders can exhibit a emphtemporal smoothing bias<n>We propose emphTemporal Contrastive Decoding (TCD), a training-free decoding method for unified LALMs.
  arXiv  Detail & Related papers  (2026-04-16T02:30:41Z)
• Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models [54.468024377748]
  We propose Anchor-based History-stable Decoding (AHD) as a training-free, plug-and-play dynamic decoding strategy.<n>AHD monitors the stability trend of tokens in real time through dynamic anchors.<n>Our approach reduces decoding steps by 80% while improving performance by 3.67%.
  arXiv  Detail & Related papers  (2026-04-10T05:08:39Z)
• Slow-Fast Inference: Training-Free Inference Acceleration via Within-Sentence Support Stability [61.28049899686214]
  We propose a training-free decoding framework that decouples generation into frequent low-cost fast steps and occasional dense-attention slow steps.<n>Across the evaluated context lengths, SFI delivers approximately $1.6times$--$14.4times$ higher decoding throughput.
  arXiv  Detail & Related papers  (2026-03-12T15:14:48Z)
• DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention [2.7422645382944935]
  We present DyLLM, a training-free inference framework that accelerates decoding by selectively computing only salient tokens.<n>Across diverse reasoning and code-generation benchmarks, DyLLM achieves up to 9.6x higher throughput.
  arXiv  Detail & Related papers  (2026-03-09T07:02:01Z)
• Beyond Scattered Acceptance: Fast and Coherent Inference for DLMs via Longest Stable Prefixes [10.877713536966601]
  Longestahead Prefix (LSP) scheduler is a training-free and model-agnostic inference paradigm based on monolithic prefix absorption.<n>LSP evaluates token stability via a single forward pass, dynamically identifies a contiguous left-aligned block of stable predictions.<n>It snaps its boundary to natural linguistic or structural acceptances before an atomic commitment.
  arXiv  Detail & Related papers  (2026-03-05T18:25:26Z)
• Beyond Fixed Frames: Dynamic Character-Aligned Speech Tokenization [27.32235541083431]
  We introduce DyCAST, a Dynamic Character-Aligned Speech Tokenizer.<n>DyCAST learns to associate tokens with character-level linguistic units during training.<n>We also introduce a retrieval-augmented decoding mechanism that enhances reconstruction without increasing fidelity.
  arXiv  Detail & Related papers  (2026-01-30T16:58:40Z)
• Residual Context Diffusion Language Models [90.07635240595926]
  Residual Context Diffusion (RCD) is a module that converts discarded token representations into contextual residuals and injects them back for the next denoising step.<n>RCD consistently improves frontier dLLMs by 5-10 points in accuracy with minimal extra computation overhead.
  arXiv  Detail & Related papers  (2026-01-30T13:16:32Z)
• CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Models [27.070045950001532]
  CD4LM is a framework that decouples training from inference.<n>On GSM8K, CD4LM matches the LLaDA baseline with a 5.18x wall-clock speedup.
  arXiv  Detail & Related papers  (2026-01-05T16:09:22Z)
• Accelerating Diffusion LLM Inference via Local Determinism Propagation [27.751279909685604]
  LocalLeap is a training-free adaptive parallel decoding strategy.<n>It achieves 6.94$times$ throughput improvements and reduces decoding steps to just 14.2% of the original requirement.
  arXiv  Detail & Related papers  (2025-10-08T14:39:34Z)
• StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMs [54.229363096087866]
  Speech tokenizers are not robust to meaning-irrelevant acoustic perturbations.<n>This instability stems from two flaws: a brittle single-path quantization architecture and a distant training signal.<n>We introduce StableToken, a tokenizer that achieves stability through a consensus-driven mechanism.
  arXiv  Detail & Related papers  (2025-09-26T11:32:51Z)
• Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction [72.27673320976933]
  Diffusion Large Language Models (dLLMs) enable breakthroughs in reasoning and parallel decoding.<n>Current caching techniques accelerate decoding by storing full-layer states, yet impose substantial memory usage.<n>We propose Sparse-dLLM, the first training-free framework integrating dynamic cache eviction with sparse attention.
  arXiv  Detail & Related papers  (2025-08-04T16:14:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.