From Chains to Trees: Parent-Conditioned Drafting for Semi-Autoregressive Speculative Decoding

• URL: http://arxiv.org/abs/2608.02123v1
• Date: Mon, 03 Aug 2026 12:15:26 GMT
• Title: From Chains to Trees: Parent-Conditioned Drafting for Semi-Autoregressive Speculative Decoding
• Abstract summary: Speculative decoding LLM accelerates inference only when drafted continuations survive target-model verification.<n>We show that parent-conditioned branching can turn conditional capacity already present in a semi-autoregressive drafter into end-to-end inference gains.
• Score: 14.328430707715114
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speculative decoding accelerates LLM inference only when drafted continuations survive target-model verification. Semi-autoregressive drafters such as DSpark predict an entire token block with one backbone forward and refine it with a lightweight Markov head. However, DSpark decodes this block as a single chain, so an early mismatch invalidates the remaining suffix and limits the benefit of large draft blocks. We show that the conditional structure already learned by DSpark can support multiple parent-consistent continuations without retraining or additional backbone passes. We introduce Parent-Conditioned Drafting Tree (PCTree), which uses the pretrained Markov head to score alternative children separately for each concrete parent and allocates a fixed verification budget to the most probable paths. This converts DSpark's linear draft into a tree while preserving its one-pass parallel backbone. Across Qwen3-{4B,8B,14B} and nine benchmarks, at $B{=}7$, measured speedup gains over autoregressive (AR) decoding, relative to matched DSpark, range from $3.1\%$ to $29.5\%$. On Qwen3-4B GSM8K at $B{=}16$, PCTree increases mean acceptance length from $9.41$ to $11.16$ and three-run mean AR speedup from $6.14{\times}$ to $6.60{\times}$. These show that parent-conditioned branching can turn conditional capacity already present in a semi-autoregressive drafter into end-to-end inference gains through an inference-only change.

Related papers

• Fork Where the Model Changes Its Mind: Belief-Shift Branching for Tree-Structured Reinforcement Learning [62.24941741019998]
  We formalize fork placement as locating the emphss of the chain's value curve, where the expected outcome turns.<n>Three instantiations, none needing step-level supervision, span access levels: a black-box probe, a logit-lens depth profile, and a learned activation direction.
  arXiv  Detail & Related papers  (2026-09-10T04:06:45Z)
• Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding [0.008640713300591884]
  Speculative decoding speeds up generation with an efficient draft model (drafter)<n>High-acceptance block-diffusion drafters such as DFlash and DFlare fill an entire block in one parallel pass.<n>We post-train the drafter on a longer block with a short curriculum that emphasizes the newly exposed positions.
  arXiv  Detail & Related papers  (2026-08-31T08:23:14Z)
• DARTree: Speculative Diffusion Decoding with Autoregressive Draft Trees [40.61855455828842]
  Speculative decoding losslessly accelerates autoregressive language models by verifying multiple draft tokens in parallel.<n>Dartree is a training-free speculative decoding method that extends a pretrained AR correction head from chains to trees.
  arXiv  Detail & Related papers  (2026-08-13T17:43:44Z)
• Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting [46.66100714934569]
  We propose textbfProgressive Tree Drafting (PTD), which employs a structured, guided parallel drafting strategy to harness the model's parallel potential.<n> Experiments demonstrate that PTD achieves up to $2times$ decoding speedup across various benchmarks while remaining training-free and model-agnostic.
  arXiv  Detail & Related papers  (2026-07-12T09:10:34Z)
• TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding [32.75269650141292]
  One-shot block drafters for speculative decoding generate the full draft in a single forward pass.<n>They predict each draft token conditioned only on the prefix context, with no dependence on previously drafted tokens.<n>This non-autoregressive conditioning causes the drafter's distribution to diverge from the verifier's true autoregressive distribution as draft depth grows.<n>We propose TreeFlash, which addresses this by incorporating a layer conditioned on the drafter's hidden state and the previous token to approximate an autoregressive distribution.
  arXiv  Detail & Related papers  (2026-06-02T16:00:18Z)
• Cost-Aware Diffusion Draft Trees for Speculative Decoding [17.900184934329666]
  We introduce textbfCaDDTree (Cost-aware Diffusion Draft Tree), a method that directly optimize token throughput (expected tokens generated per unit time) by jointly selecting the tree structure and node budget.<n>Experiments on Qwen3-4B and Qwen3-8B across eight benchmarks spanning reasoning, coding, and instruction-following tasks show that caDDTree matches or surpasses DDTree with oracle budget selection on nearly all tasks.
  arXiv  Detail & Related papers  (2026-06-01T07:29:56Z)
• Copy-as-Decode: Grammar-Constrained Parallel Prefill for LLM Editing [2.6382975801439836]
  LLMs edit text and code by autoregressively regenerating the full output, even when most tokens appear verbatim in the input.<n>We study Copy-as-Decode, a decoding-layer mechanism that recasts edit generation as structured decoding over a two-primitive grammar.
  arXiv  Detail & Related papers  (2026-04-20T12:29:53Z)
• Accelerating Speculative Decoding with Block Diffusion Draft Trees [20.28933257827737]
  Speculative decoding accelerates autoregressive language models by using a lightweight drafter to propose multiple future tokens.<n>DFlash shows that a block diffusion drafter can generate an entire draft block in a single forward pass.<n> Vanilla DFlash verifies only a single drafted trajectory per round, potentially limiting its acceptance length.<n>We introduce DDTree, a method that constructs a draft tree directly from the per-position distributions of a block diffusion drafter.
  arXiv  Detail & Related papers  (2026-04-14T17:23:14Z)
• SMART: When is it Actually Worth Expanding a Speculative Tree? [26.986851887229054]
  Tree-based speculative decoding accelerates autoregressive generation by verifying a branching tree draft tokens in a single targetmodel forward pass.<n>Existing methods prioritize maximizing token-level likelihood of the number of accepted tokens while ignoring a critical efficiency paradox''<n>We propose SMART, a system-aware marginal analysis framework for runtime tree construction.
  arXiv  Detail & Related papers  (2026-04-09T13:17:56Z)
• Diffusion Language Models Know the Answer Before Decoding [56.96815863705218]
  Diffusion language models (DLMs) have emerged as an alternative to autoregressive approaches.<n>Our work highlights and leverage an overlooked property of DLMs early answer convergence.<n>We introduce Prophet, a training-free fast decoding paradigm that enables early commit decoding.
  arXiv  Detail & Related papers  (2025-08-27T15:40:25Z)
• Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass [72.07642648108849]
  Superposed Decoding is a new decoding algorithm that generates $k$ drafts at the cost of one autoregressive inference pass. Superposed Decoding can be combined with other decoding strategies, resulting in universal coverage gains when scaling inference time compute.
  arXiv  Detail & Related papers  (2024-05-28T17:40:48Z)
• Strongly Incremental Constituency Parsing with Graph Neural Networks [70.16880251349093]
  Parsing sentences into syntax trees can benefit downstream applications in NLP. Transition-baseds build trees by executing actions in a state transition system. Existing transition-baseds are predominantly based on the shift-reduce transition system.
  arXiv  Detail & Related papers  (2020-10-27T19:19:38Z)
• Please Mind the Root: Decoding Arborescences for Dependency Parsing [67.71280539312536]
  We analyze the output of state-of-the-arts on many languages from the Universal Dependency Treebank. The worst constraint-violation rate we observe is 24%.
  arXiv  Detail & Related papers  (2020-10-06T08:31:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.