PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding

• URL: http://arxiv.org/abs/2605.08632v1
• Date: Sat, 09 May 2026 02:50:58 GMT
• Title: PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding
• Abstract summary: Speculative decoding accelerates Large Language Models (LLMs) inference by using a lightweight draft model.<n>This paper builds upon PARD to propose PARD-2, a dual-mode speculative decoding framework with Confidence-Adaptive Token (CAT) optimization.
• Score: 10.569335087583655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speculative decoding accelerates Large Language Models (LLMs) inference by using a lightweight draft model to propose candidate tokens that are verified in parallel by the target model. However, existing draft model training objectives are not directly aligned with the inference-time goal of maximizing consecutive token acceptance. To address this issue, we reformulate the draft model optimization objective, shifting the focus from token prediction accuracy to the overall acceptance length. In this paper, we build upon PARD to propose PARD-2, a dual-mode speculative decoding framework with Confidence-Adaptive Token (CAT) optimization. This approach adaptively reweights each token to better align with the verification process. Notably, PARD-2 enables a single draft model to support both target-dependent and target-independent modes. Experiments across diverse models and tasks demonstrate that PARD-2 achieves up to 6.94$\times$ lossless acceleration, surpassing EAGLE-3 by 1.9$\times$ and PARD by 1.3$\times$ on Llama3.1-8B. Our code is available at https://github.com/AMD-AGI/PARD.

Related papers

• Verification-Aware Training for Speculative Decoding [57.84976863792784]
  We introduce Verification-Aware Training (VAT), a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision.<n>VAT modifies only the training objective, so it can be layered on top of existing methods without changing the draft architecture, the target model, or the inference procedure.<n>VAT improves average acceptance length by up to 11.4% and wall-clock speedup by up to 8.7%, with consistent gains across math, code, and chat benchmarks.
  arXiv  Detail & Related papers  (2026-08-31T01:42:10Z)
• Draft-OPD: On-Policy Distillation for Speculative Draft Models [49.23782868133977]
  Speculative decoding accelerates large language model inference by pairing a target model with a lightweight draft model whose proposed tokens are verified in parallel.<n>We propose Draft-OPD, which uses target-assisted rollout for stable continuations and replays drafting from the verification-exposed error positions.
  arXiv  Detail & Related papers  (2026-05-28T04:30:22Z)
• Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation [82.04574413609187]
  Speculative decoding suffers from performance degradation when target models are fine-tuned for specific domains.<n>A naive solution is to retrain draft models for every target model, which is costly and inefficient.<n>We introduce a parameter- and data-efficient framework named Efficient Draft Adaptation, abbreviated as EDA.
  arXiv  Detail & Related papers  (2026-03-10T11:35:58Z)
• Training-free Dropout Sampling for Semantic Token Acceptance in Speculative Decoding [13.249778063956917]
  Speculative decoding accelerates large language model inference by proposing tokens with a lightweight draft model.<n>This work introduces DropMatch, a novel approach that matches draft tokens to the predictive distribution of the target model.<n> Experiments across multiple benchmarks demonstrate that our approach increases acceptance length while maintaining competitive task performance.
  arXiv  Detail & Related papers  (2026-02-11T04:53:33Z)
• DEER: Draft with Diffusion, Verify with Autoregressive Models [33.19684425811274]
  Speculative decoding mitigates the inherent latency of autoregressive decoding.<n>We introduce DEER, an efficient speculative decoding framework.<n>Experiments show DEER reaches draft acceptance lengths of up to 32 tokens.
  arXiv  Detail & Related papers  (2025-12-17T08:19:04Z)
• DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding [66.40658898418316]
  We present DiffuSpec, a training-free drop-in framework that uses a pretrained diffusion language model (DLM) to produce multi-token drafts in a single forward pass.<n>Across benchmarks, DiffuSpec yields up to 3x wall-clock speedup, establishing diffusion-based drafting as a robust alternative to autoregressive drafters for speculative decoding.
  arXiv  Detail & Related papers  (2025-09-28T07:00:15Z)
• CARD: A Cache-Assisted Parallel Speculative Decoding Framework via Query-and-Correct Paradigm for Accelerating LLM Inference [14.527697328189362]
  We propose a speculative decoding framework called CARD, which employs a novel query-and-correct paradigm.<n>Our approach decouples drafting from verification, effectively leveraging the draft model's efficiency without additional fine-tuning.<n> CARD significantly outperforms existing state-of-the-art methods, achieving up to a 4.83x acceleration over vanilla autoregressive decoding.
  arXiv  Detail & Related papers  (2025-08-06T14:02:10Z)
• ARM: Adaptive Reasoning Model [52.955386431182724]
  We propose Adaptive Reasoning Model (ARM), a reasoning model capable of adaptively selecting appropriate formats based on the task at hand.<n>Ada-GRPO enables ARM to achieve high token efficiency, reducing tokens by an average of 30%, and up to 70%, while maintaining performance comparable to the model that relies solely on Long CoT.
  arXiv  Detail & Related papers  (2025-05-26T17:38:50Z)
• PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation [4.031603850949324]
  We introduce a novel speculative decoding method that enables low-cost adaptation of autoregressive draft models into parallel draft models.<n>Our proposed conditional drop token method can improve draft model training efficiency by 3x.<n>On our optimized inference framework, PARD accelerates LLaMA3.1-8B inference by 4.08x, achieving 311.5 tokens per second.
  arXiv  Detail & Related papers  (2025-04-23T12:27:43Z)
• ParallelSpec: Parallel Drafter for Efficient Speculative Decoding [62.68430939686566]
  We present ParallelSpec, an alternative to auto-regressive drafting strategies in state-of-the-art speculative decoding approaches. In contrast to auto-regressive drafting in the speculative stage, we train a parallel drafter to serve as an efficient speculative model.
  arXiv  Detail & Related papers  (2024-10-08T01:05:08Z)
• PEARL: Parallel Speculative Decoding with Adaptive Draft Length [12.166703341906242]
  We propose a conceptually simple, flexible, and general framework to boost speculative decoding, namely Parallel spEculative decoding with Adaptive dRaft Length (PEARL)<n>PEARL proposes pre-verify to verify the first draft token in advance during the drafting phase, and post-verify to generate more draft tokens during the verification phase.<n> Experiments on various text generation benchmarks demonstrate the effectiveness of our PEARL, leading to a superior speed up performance up to 4.43$times$ and 1.50$times$, compared to auto-regressive decoding and vanilla speculative decoding, respectively.
  arXiv  Detail & Related papers  (2024-08-13T08:32:06Z)
• Decoding-Time Language Model Alignment with Multiple Objectives [116.42095026960598]
  Existing methods primarily focus on optimizing LMs for a single reward function, limiting their adaptability to varied objectives. Here, we propose $textbfmulti-objective decoding (MOD)$, a decoding-time algorithm that outputs the next token from a linear combination of predictions. We show why existing approaches can be sub-optimal even in natural settings and obtain optimality guarantees for our method.
  arXiv  Detail & Related papers  (2024-06-27T02:46:30Z)
• SDSAT: Accelerating LLM Inference through Speculative Decoding with Semantic Adaptive Tokens [4.5888031410244885]
  We propose an acceleration scheme for large language models (LLMs) through Speculative Decoding with Semantic Adaptive Tokens (SDSAT) The primary objective of this design is to enhance the LLM model's ability to generate draft tokens more accurately without compromising its accuracy. Experiments conducted on the CodeLlama-13B and 7B models have yielded speed increases of over 3.5X and 3.0X, respectively.
  arXiv  Detail & Related papers  (2024-03-27T14:54:27Z)
• DistillSpec: Improving Speculative Decoding via Knowledge Distillation [70.61777015900272]
  Speculative decoding (SD) accelerates large language model inference by employing a faster draft model for generating multiple tokens. We propose DistillSpec that uses knowledge distillation to better align the draft model with the target model, before applying SD. We show that DistillSpec yields impressive 10 - 45% speedups over standard SD on a range of standard benchmarks.
  arXiv  Detail & Related papers  (2023-10-12T16:21:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.