Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding

• URL: http://arxiv.org/abs/2608.30427v1
• Date: Mon, 31 Aug 2026 08:23:14 GMT
• Title: Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding
• Abstract summary: Speculative decoding speeds up generation with an efficient draft model (drafter)<n>High-acceptance block-diffusion drafters such as DFlash and DFlare fill an entire block in one parallel pass.<n>We post-train the drafter on a longer block with a short curriculum that emphasizes the newly exposed positions.
• Score: 0.008640713300591884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, preserving the target's output distribution. High-acceptance block-diffusion drafters such as DFlash and DFlare fill an entire block in one parallel pass. In many cycles, the target accepts the whole block, so the drafter exhausts its trained block horizon before verification fails. We call this unrealized acceptance stranded speed-up. A mean committed length, per prompt or per cycle, hides it, whereas the acceptance histogram exposes it as a spike in the ceiling bin, the fraction of cycles that accept the entire block. We recommend the histogram as a preflight check before spending training compute. Naively widening the block at inference does not recover the speed-up, because once the block outgrows its training size, the drafter's bidirectional attention shifts its distribution even at early positions and erodes front-of-block verification. Instead, we post-train the drafter on a longer block with a short curriculum that emphasizes the newly exposed positions, a method we call DBloom. Expanding the pretrained DFlash and DFlare drafters from block size 16 to 24 across Qwen3-8B and Qwen3-4B targets raises the per-prompt committed length on the high-ceiling benchmarks by a median of +0.8 tokens (up to +1.1). Once continuation fine-tuning precedes expansion, the increase reaches 1.37 tokens. The same expansion also lifts committed length on all seven benchmarks for Gemma-4-12B-IT, a different model family, by a median of +0.41 tokens (Arm A), and the full continuation-then-expand pipeline (Arm B) adds +0.29 to +0.98 tokens over the same B16 drafter. In a prompt-matched comparison against JetSpec, a contemporary tree-based drafter not used in our design, DBloom commits more tokens on every benchmark at tree budgets up to 64 nodes.

Related papers

• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• DARTree: Speculative Diffusion Decoding with Autoregressive Draft Trees [40.61855455828842]
  Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel.<n>Dartree is a training-free speculative decoding method that extends a pretrained AR correction head from chains to trees.
  arXiv  Detail & Related papers  (2026-08-13T17:43:44Z)
• From Chains to Trees: Parent-Conditioned Drafting for Semi-Autoregressive Speculative Decoding [14.328430707715114]
  Speculative decoding LLM accelerates inference only when drafted continuations survive target-model verification.<n>We show that parent-conditioned branching can turn conditional capacity already present in a semi-autoregressive drafter into end-to-end inference gains.
  arXiv  Detail & Related papers  (2026-08-03T12:15:26Z)
• Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafters [3.9513930662588863]
  Block (DLM-style) drafters predict the whole block in parallel, which is fast but trained with a full-block cross-entropy.<n>Recent acceptance-aware objectives patch this by reweighting the full-block loss.<n>AUF approximates that prefix-sensitive supervision on the loss side by keeping the cross-entropy support only through the drafter's first predicted failure.
  arXiv  Detail & Related papers  (2026-07-02T08:44:04Z)
• PRESTO: Prefix-Aligned Tree Drafting for Diffusion Speculative Decoding [98.71600061506206]
  We propose PRESTO, a principled framework that extends tree-based drafting to diffusion drafters.<n> PRESTO achieves up to an average of $1.5times$ end-to-end throughput speedup on the state-of-the-art dedicated diffusion drafter SD.
  arXiv  Detail & Related papers  (2026-06-20T14:32:14Z)
• TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding [32.75269650141292]
  One-shot block drafters for speculative decoding generate the full draft in a single forward pass.<n>They predict each draft token conditioned only on the prefix context, with no dependence on previously drafted tokens.<n>This non-autoregressive conditioning causes the drafter's distribution to diverge from the verifier's true autoregressive distribution as draft depth grows.<n>We propose TreeFlash, which addresses this by incorporating a layer conditioned on the drafter's hidden state and the previous token to approximate an autoregressive distribution.
  arXiv  Detail & Related papers  (2026-06-02T16:00:18Z)
• D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting [59.204113363599994]
  Speculative decoding accelerates inference by having a small drafter propose tokens that a larger target model verifies in parallel.<n>Recent diffusion-based parallel drafters such as DFlash predict the full B-token block in one forward pass, enabling deeper drafters and longer accepted blocks.<n>We derive per-position training weights from a differentiable surrogate of expected accepted draft length, matching the weight of each position to its log-probability contribution.<n>Across six benchmarks, two Qwen3-4B draft depths, two decoding temperatures, and two additional target models, D-PACE consistently improves both wall-clock speedup and average
  arXiv  Detail & Related papers  (2026-05-12T06:27:57Z)
• SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting [25.273024580844346]
  SpecBlock is a block-iterative drafter that combines path dependence with cheap drafting.<n>A cost-aware bandit at deployment uses free verifier feedback to update the drafter selectively.
  arXiv  Detail & Related papers  (2026-05-08T04:59:48Z)
• Pipeline Parallelism is All You Need for Optimized Early-Exit Based Self-Speculative Decoding [73.67253077506672]
  Large language models (LLMs) deliver impressive generation quality, but incur very high inference cost.<n>Early-exit based self-speculative decoding (EESD) has emerged to mitigate this cost.<n>We propose Pipeline-Parallel Self-Speculative Decoding (PPSD) that fully pipelines the draft and verification work.
  arXiv  Detail & Related papers  (2025-09-19T04:51:41Z)
• Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding [60.06816407728172]
  Discrete diffusion language models have shown strong potential for text generation.<n>Standard supervised fine-tuning misaligns with semi-autoregressive inference.<n>We propose Blockwise SFT, which partitions responses into fixed-size blocks.
  arXiv  Detail & Related papers  (2025-08-27T02:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.