ReTrace: Rejected-Trajectory Conditioning for Speculative Decoding

• URL: http://arxiv.org/abs/2608.29748v2
• Date: Sat, 05 Sep 2026 16:06:40 GMT
• Title: ReTrace: Rejected-Trajectory Conditioning for Speculative Decoding
• Abstract summary: We introduce ReTrace, a conditioning method that conditions each draft block on the rejected suffix from the previous round.<n>ReTrace consistently improves average acceptance length and end-to-end decoding speed over its DFlash backbone.
• Score: 46.17292752074153
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which are then verified in parallel by a larger target model. However, after the first rejection, standard prefix-based verification discards the remaining draft suffix, so the computation spent generating and verifying those positions does not contribute to decoding progress. Focusing on DFlash, we show that rejected positions in a rejected suffix may still align with the target continuation, indicating that the draft model can retain useful semantic and structural information despite local token-level errors. Motivated by this observation and inspired by conditional diffusion, we introduce ReTrace, a rejected-trajectory conditioning method that conditions each draft block on the rejected suffix from the previous round rather than generating it from fresh mask placeholders alone. ReTrace retains the hidden representations of the rejected suffixes, aligns them with the next draft block, refines them using target-aware correction signals from the same verification pass, and admits them into the drafter's input embeddings through gated residual fusion. Because rejected tokens are never committed and target-side verification remains unchanged, ReTrace preserves the lossless property of speculative decoding without requiring an additional model forward pass. Experiments with Qwen3 models across mathematical reasoning, code generation, and open-ended dialogue demonstrate that ReTrace consistently improves average acceptance length and end-to-end decoding speed over its DFlash backbone. By introducing cross-round conditioning without modifying within-round proposal generation, ReTrace is largely orthogonal to existing drafting improvements and might be combined with them for further gains.

Related papers

• DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decoding [25.203742152821548]
  We propose DFlow, a framework that enables verifier information to flow across drafting rounds.<n>DFlow reuses the hidden states produced by the target verifier for the rejected suffix to guide subsequent drafting.<n>Experiments on Qwen3 models across diverse benchmarks demonstrate that DFlow consistently improves draft quality and acceptance length over DFlash.
  arXiv  Detail & Related papers  (2026-09-06T09:25:56Z)
• Verification-Aware Training for Speculative Decoding [57.84976863792784]
  We introduce Verification-Aware Training (VAT), a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision.<n>VAT modifies only the training objective, so it can be layered on top of existing methods without changing the draft architecture, the target model, or the inference procedure.<n>VAT improves average acceptance length by up to 11.4% and wall-clock speedup by up to 8.7%, with consistent gains across math, code, and chat benchmarks.
  arXiv  Detail & Related papers  (2026-08-31T01:42:10Z)
• ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping [79.1905186547363]
  ResiSpec is a framework that strategically reforms the proposal distribution during verification.<n>It prevents candidate obsolescence and achieves up to 1.92$times$ speedup over state-of-the-art multi-candidate methods.
  arXiv  Detail & Related papers  (2026-08-25T11:25:55Z)
• Encoder-Decoder Manifold Alignment for Idempotent Generation [51.959208893243776]
  We argue that a key reason for this failure is a geometric mismatch between the manifold learned by the encoder and decoder.<n>We propose a new training framework that explicitly closes this gap by forcing the encoder and decoder to learn consistent representations of the same underlying data manifold.
  arXiv  Detail & Related papers  (2026-06-21T02:14:37Z)
• Teaching Diffusion to Speculate Left-to-Right [3.205247598097648]
  Large language models (LLMs) achieve remarkable performance across a wide range of tasks, but their autoregressive decoding process incurs substantial inference costs.<n> Speculative decoding addresses this bottleneck by employing a lightweight draft model to propose multiple future tokens that are subsequently verified in parallel by a larger target model.<n>Recent work has demonstrated that diffusion language models are well suited for this setting, as they can generate entire blocks of draft tokens in parallel.
  arXiv  Detail & Related papers  (2026-06-10T01:21:56Z)
• D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models [27.44373450962651]
  Recent diffusion-based drafters generate an entire block of tokens in parallel but usually commit to a single draft sequence per verification.<n>We propose D2SD, a dual diffusion draft speculative decoding framework that organizes candidates into a confidence-guided prefix tree.<n>The resulting shared-guided candidates are jointly verified via cascade attention.
  arXiv  Detail & Related papers  (2026-06-03T04:48:00Z)
• Draft-OPD: On-Policy Distillation for Speculative Draft Models [49.23782868133977]
  Speculative decoding accelerates large language model inference by pairing a target model with a lightweight draft model whose proposed tokens are verified in parallel.<n>We propose Draft-OPD, which uses target-assisted rollout for stable continuations and replays drafting from the verification-exposed error positions.
  arXiv  Detail & Related papers  (2026-05-28T04:30:22Z)
• Make Every Draft Count: Hidden State based Speculative Decoding [7.373716645300893]
  We propose a novel system that transforms discarded drafts into reusable tokens.<n>Our key insight is to perform auto-regressive prediction at the hidden states level.<n>We demonstrate up to a 3.3x speedup against standard speculative decoding.
  arXiv  Detail & Related papers  (2026-02-02T08:25:21Z)
• DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding [66.40658898418316]
  We present DiffuSpec, a training-free drop-in framework that uses a pretrained diffusion language model (DLM) to produce multi-token drafts in a single forward pass.<n>Across benchmarks, DiffuSpec yields up to 3x wall-clock speedup, establishing diffusion-based drafting as a robust alternative to autoregressive drafters for speculative decoding.
  arXiv  Detail & Related papers  (2025-09-28T07:00:15Z)
• Think Before You Accept: Semantic Reflective Verification for Faster Speculative Decoding [48.52389201779425]
  Speculative decoding accelerates inference by generating multiple draft tokens using a lightweight model and verifying them in parallel.<n>Existing verification methods rely heavily on distributional consistency while overlooking semantic correctness.<n>We propose Reflective Verification, a training-free and semantics-aware approach that achieves a better trade-off between correctness and efficiency.
  arXiv  Detail & Related papers  (2025-05-24T10:26:27Z)
• Traversal Verification for Speculative Tree Decoding [15.720388162422978]
  Speculative decoding is a promising approach for accelerating large language models.<n>This paper introduces Traversal Verification, a novel speculative decoding algorithm.<n>We show that our method consistently improves acceptance length and throughput over existing methods.
  arXiv  Detail & Related papers  (2025-05-18T12:51:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.