DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding

• URL: http://arxiv.org/abs/2608.05448v1
• Date: Wed, 05 Aug 2026 22:49:54 GMT
• Title: DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding
• Abstract summary: We show that the accepted draft length degrades as the entropy of the target sampling distribution increases.<n>We propose a dependent block drafter based on a low-rank latent mixture over token positions, complemented by an acceptance-oriented training objective.
• Score: 26.72775347772003
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Speculative decoding accelerates large language models' inference by using a lightweight drafter to propose multiple future tokens and a target model to verify them. While recent block and diffusion-style drafters can predict several positions in a single pass, their training and sampling procedures are typically optimized for greedy decoding or assume that positions in the draft block are conditionally independent. This assumption becomes brittle in non-greedy speculative decoding, where the target distribution is deliberately stochastic and multiple continuations become plausible. We study this mismatch for block diffusion drafters and show that the accepted draft length degrades as the entropy of the target sampling distribution increases. We propose a dependent block drafter based on a low-rank latent mixture over token positions, complemented by an acceptance-oriented training objective that directly targets the expected verified length. Experiments with Qwen3-4B and Qwen3-8B on GSM8K, MT-Bench, HumanEval, and creative-writing benchmarks show that our approach, namely DBLast, consistently improves accepted length over independent block sampling, especially in higher-entropy decoding regimes.

Related papers

• ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping [79.1905186547363]
  ResiSpec is a framework that strategically reforms the proposal distribution during verification.<n>It prevents candidate obsolescence and achieves up to 1.92$times$ speedup over state-of-the-art multi-candidate methods.
  arXiv  Detail & Related papers  (2026-08-25T11:25:55Z)
• PRESTO: Prefix-Aligned Tree Drafting for Diffusion Speculative Decoding [98.71600061506206]
  We propose PRESTO, a principled framework that extends tree-based drafting to diffusion drafters.<n> PRESTO achieves up to an average of $1.5times$ end-to-end throughput speedup on the state-of-the-art dedicated diffusion drafter SD.
  arXiv  Detail & Related papers  (2026-06-20T14:32:14Z)
• Accelerating Speculative Diffusions via Block Verification [77.23389794645801]
  Speculative decoding speeds up LLM inference by using a draft model to generate tokens, with an acceptance-rejection scheme.<n>Existing diffusion adaptations either use inefficient sampling techniques or rely on an alternative scheme.<n>We introduce a novel scheme that efficiently implements the original speculative sampling mechanism for diffusion models.
  arXiv  Detail & Related papers  (2026-06-11T14:54:13Z)
• Teaching Diffusion to Speculate Left-to-Right [3.205247598097648]
  Large language models (LLMs) achieve remarkable performance across a wide range of tasks, but their autoregressive decoding process incurs substantial inference costs.<n> Speculative decoding addresses this bottleneck by employing a lightweight draft model to propose multiple future tokens that are subsequently verified in parallel by a larger target model.<n>Recent work has demonstrated that diffusion language models are well suited for this setting, as they can generate entire blocks of draft tokens in parallel.
  arXiv  Detail & Related papers  (2026-06-10T01:21:56Z)
• D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models [27.44373450962651]
  Recent diffusion-based drafters generate an entire block of tokens in parallel but usually commit to a single draft sequence per verification.<n>We propose D2SD, a dual diffusion draft speculative decoding framework that organizes candidates into a confidence-guided prefix tree.<n>The resulting shared-guided candidates are jointly verified via cascade attention.
  arXiv  Detail & Related papers  (2026-06-03T04:48:00Z)
• Draft-OPD: On-Policy Distillation for Speculative Draft Models [49.23782868133977]
  Speculative decoding accelerates large language model inference by pairing a target model with a lightweight draft model whose proposed tokens are verified in parallel.<n>We propose Draft-OPD, which uses target-assisted rollout for stable continuations and replays drafting from the verification-exposed error positions.
  arXiv  Detail & Related papers  (2026-05-28T04:30:22Z)
• D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting [59.204113363599994]
  Speculative decoding accelerates inference by having a small drafter propose tokens that a larger target model verifies in parallel.<n>Recent diffusion-based parallel drafters such as DFlash predict the full B-token block in one forward pass, enabling deeper drafters and longer accepted blocks.<n>We derive per-position training weights from a differentiable surrogate of expected accepted draft length, matching the weight of each position to its log-probability contribution.<n>Across six benchmarks, two Qwen3-4B draft depths, two decoding temperatures, and two additional target models, D-PACE consistently improves both wall-clock speedup and average
  arXiv  Detail & Related papers  (2026-05-12T06:27:57Z)
• Training-free Dropout Sampling for Semantic Token Acceptance in Speculative Decoding [13.249778063956917]
  Speculative decoding accelerates large language model inference by proposing tokens with a lightweight draft model.<n>This work introduces DropMatch, a novel approach that matches draft tokens to the predictive distribution of the target model.<n> Experiments across multiple benchmarks demonstrate that our approach increases acceptance length while maintaining competitive task performance.
  arXiv  Detail & Related papers  (2026-02-11T04:53:33Z)
• DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding [66.40658898418316]
  We present DiffuSpec, a training-free drop-in framework that uses a pretrained diffusion language model (DLM) to produce multi-token drafts in a single forward pass.<n>Across benchmarks, DiffuSpec yields up to 3x wall-clock speedup, establishing diffusion-based drafting as a robust alternative to autoregressive drafters for speculative decoding.
  arXiv  Detail & Related papers  (2025-09-28T07:00:15Z)
• Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation [64.59292053188264]
  Experimental results on mainstream SD benchmarks as well as reasoning-heavy benchmarks demonstrate the superior performance of SVIP.<n>SVIP is a training-free dynamic length policy for speculative decoding systems that adaptively determines the lengths of draft sequences by referring to the draft entropy.
  arXiv  Detail & Related papers  (2024-11-27T15:53:17Z)
• AdaEDL: Early Draft Stopping for Speculative Decoding of Large Language Models via an Entropy-based Lower Bound on Token Acceptance Probability [5.421949344085942]
  We show that AdaEDL consistently outperforms static draft-length speculative decoding by 10%-57%. We also show that AdaEDL is more robust than these techniques and preserves performance in high-temperature scenarios.
  arXiv  Detail & Related papers  (2024-10-24T01:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.