PRESTO: Prefix-Aligned Tree Drafting for Diffusion Speculative Decoding

• URL: http://arxiv.org/abs/2607.22634v1
• Date: Sat, 20 Jun 2026 14:32:14 GMT
• Title: PRESTO: Prefix-Aligned Tree Drafting for Diffusion Speculative Decoding
• Abstract summary: We propose PRESTO, a principled framework that extends tree-based drafting to diffusion drafters.<n> PRESTO achieves up to an average of $1.5times$ end-to-end throughput speedup on the state-of-the-art dedicated diffusion drafter SD.
• Score: 98.71600061506206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs, generating tokens in parallel. This makes them effective draft models for speculative decoding (SD), producing an entire block of draft tokens in a single forward pass. Yet existing diffusion-based drafting methods rely on linear drafting, even though dLLMs emit multiple candidate tokens across positions, inducing a large combinatorial space of decoding paths. Consequently, they limit acceptance length and decoding efficiency. To exploit this multi-candidate structure, we apply tree-based drafting to diffusion drafters, enabling exploration of diverse candidate paths. However, we find that naive tree drafting is suboptimal: diffusion marginals are prefix-blind, mismatching the prefix-based AR verification and yielding unreliable path ranking. We propose PRESTO, a principled framework that extends tree-based drafting to diffusion drafters while resolving the fundamental mismatch between diffusion draft confidence and prefix-based AR verification through PREfix-aligned Scoring and priority-based Tree search for diffusion speculative decOding. The key principles behind PRESTO are that (1) candidate ranking should align with the prefix-based nature of AR verification, and (2) tree construction should prioritize candidate paths with high verification potential to maximize acceptance length. Extensive experiments show that PRESTO achieves up to an average of $1.5\times$ end-to-end throughput speedup on the state-of-the-art dedicated diffusion drafter SD and an average of $1.12\times$ on self-speculative diffusion LLMs across diverse benchmarks.

Related papers

• ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping [79.1905186547363]
  ResiSpec is a framework that strategically reforms the proposal distribution during verification.<n>It prevents candidate obsolescence and achieves up to 1.92$times$ speedup over state-of-the-art multi-candidate methods.
  arXiv  Detail & Related papers  (2026-08-25T11:25:55Z)
• DARTree: Speculative Diffusion Decoding with Autoregressive Draft Trees [40.61855455828842]
  Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel.<n>Dartree is a training-free speculative decoding method that extends a pretrained AR correction head from chains to trees.
  arXiv  Detail & Related papers  (2026-08-13T17:43:44Z)
• Diffusion Language Model Parallel Decoding via Product-of-Experts Bridge [93.37920675145553]
  Diffusion language models (DLMs) offer substantial speed advantages through parallel decoding.<n>The lack of token dependencies limits generation quality compared to autoregressive (AR) models.<n>Recent progress attempts to bridge the gap via importance sampling, with DLM being the proposal and AR being the target.<n>We introduce PoE-Bridge, a novel decoding framework that drastically improves generation speed and accuracy.
  arXiv  Detail & Related papers  (2026-06-06T08:21:06Z)
• D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models [27.44373450962651]
  Recent diffusion-based drafters generate an entire block of tokens in parallel but usually commit to a single draft sequence per verification.<n>We propose D2SD, a dual diffusion draft speculative decoding framework that organizes candidates into a confidence-guided prefix tree.<n>The resulting shared-guided candidates are jointly verified via cascade attention.
  arXiv  Detail & Related papers  (2026-06-03T04:48:00Z)
• TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding [4.5881678566958985]
  Using a diffusion model for parallel drafting is a promising approach for speculative decoding.<n>This shifts the bottleneck to verification: verifying a single sequence limits acceptance length, while verifying large draft trees incurs excessive target-model latency.<n>We propose TAPS, a target-aware prefix selection method that turns diffusion marginals into path-conditioned acceptance estimates.
  arXiv  Detail & Related papers  (2026-05-30T02:39:40Z)
• Dynamic Delayed Tree Expansion For Improved Multi-Path Speculative Decoding [35.984745508100595]
  We present a systematic evaluation of verification strategies across model families, tasks, and sampling regimes.<n>Traversal Verification dominates consistently, with OT-based methods lagging far behind.<n>We propose delayed tree expansion, which drafts a partial single path, delaying the i.i.d. branching point.
  arXiv  Detail & Related papers  (2026-02-19T01:41:58Z)
• DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding [66.40658898418316]
  We present DiffuSpec, a training-free drop-in framework that uses a pretrained diffusion language model (DLM) to produce multi-token drafts in a single forward pass.<n>Across benchmarks, DiffuSpec yields up to 3x wall-clock speedup, establishing diffusion-based drafting as a robust alternative to autoregressive drafters for speculative decoding.
  arXiv  Detail & Related papers  (2025-09-28T07:00:15Z)
• Traversal Verification for Speculative Tree Decoding [15.720388162422978]
  Speculative decoding is a promising approach for accelerating large language models.<n>This paper introduces Traversal Verification, a novel speculative decoding algorithm.<n>We show that our method consistently improves acceptance length and throughput over existing methods.
  arXiv  Detail & Related papers  (2025-05-18T12:51:55Z)
• Towards Optimal Multi-draft Speculative Decoding [102.67837141152232]
  Multi-Draft Speculative Decoding (MDSD) is a recent approach where, when generating each token, a small draft model generates multiple drafts.<n>This paper discusses the dual of the optimal transport problem, providing a way to efficiently compute the optimal acceptance rate.
  arXiv  Detail & Related papers  (2025-02-26T03:22:44Z)
• FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling [59.8051705468084]
  Speculative sampling has emerged as an important technique for accelerating the auto-regressive generation process of large language models.<n>We present FR-Spec, a frequency-ranked speculative sampling framework that optimize draft candidate selection through vocabulary space compression.
  arXiv  Detail & Related papers  (2025-02-20T18:58:10Z)
• Graph-Structured Speculative Decoding [52.94367724136063]
  Speculative decoding has emerged as a promising technique to accelerate the inference of Large Language Models. We introduce an innovative approach utilizing a directed acyclic graph (DAG) to manage the drafted hypotheses. We observe a remarkable speedup of 1.73$times$ to 1.96$times$, significantly surpassing standard speculative decoding.
  arXiv  Detail & Related papers  (2024-07-23T06:21:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.