Multi-Drafter Speculative Decoding with Alignment Feedback

• URL: http://arxiv.org/abs/2604.05417v1
• Date: Tue, 07 Apr 2026 04:25:26 GMT
• Title: Multi-Drafter Speculative Decoding with Alignment Feedback
• Abstract summary: Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller model to draft future tokens.<n>We introduce textsc MetaSD, a unified framework that integrates multiple drafters into the SD process.
• Score: 37.869090398558505
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller model to draft future tokens, which are then verified by the target LLM. This preserves generation quality by accepting only aligned tokens. However, individual drafters, often trained for specific tasks or domains, exhibit limited effectiveness across diverse applications. To address this, we introduce \textsc{MetaSD}, a unified framework that integrates multiple drafters into the SD process. MetaSD dynamically allocates computational resources to heterogeneous drafters by leveraging alignment feedback and framing drafter selection as a multi-armed bandit problem. Extensive experiments show MetaSD consistently outperforms single-drafter approaches.

Related papers

• PRISMR: Overcoming Parse Collapse in Multimodal Listwise Ranking via Parameterized Representation Internalization [49.659739762240655]
  Generative listwise ranking with Large Multimodal Models (LMMs) aims to capture global list context in a single forward pass, but its effectiveness degrades in long-context multimodal scenarios.<n>We identify a recurring failure mode, parse collapse, where the autoregressive decoder produces fluent yet incomplete rankings by silently omitting candidates and terminating early.<n>We propose PRISMR, a framework that replaces transient in-context list processing with parametric structural conditioning.
  arXiv  Detail & Related papers  (2026-06-11T06:09:51Z)
• TABED: Test-Time Adaptive Ensemble Drafting for Robust Speculative Decoding in LVLMs [14.030784220154151]
  We propose Test-time Adaptive Batched Ensemble Drafting (TABED) for Large Vision-Language Models.<n> TABED ensembles multiple drafts obtained via batch inference by leveraging deviations from past ground truths available in the SD setting.<n>It achieves an average robust walltime speedup of 1.74x over autoregressive decoding and a 5% improvement over single drafting methods.
  arXiv  Detail & Related papers  (2026-01-28T08:16:57Z)
• DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding [66.40658898418316]
  We present DiffuSpec, a training-free drop-in framework that uses a pretrained diffusion language model (DLM) to produce multi-token drafts in a single forward pass.<n>Across benchmarks, DiffuSpec yields up to 3x wall-clock speedup, establishing diffusion-based drafting as a robust alternative to autoregressive drafters for speculative decoding.
  arXiv  Detail & Related papers  (2025-09-28T07:00:15Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decoding [11.167833073080612]
  Accelerating inference in Large Language Models (LLMs) is critical for real-time interactions.<n> Speculative decoding has gained attention for improving inference speed by drafting and verifying tokens, thereby generating multiple tokens in a single forward pass.<n>We propose Hierarchy Drafting (HD), a novel lossless drafting approach that organizes various token sources into multiple databases in a hierarchical framework based on temporal locality.<n>Our experiments on Spec-Bench using LLMs with 7B and 13B parameters demonstrate that HD outperforms existing database drafting methods, achieving robust inference speedups across model sizes, tasks, and temperatures.
  arXiv  Detail & Related papers  (2025-02-08T15:32:53Z)
• Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies [7.14946066475415]
  Speculative decoding (SD) methods offer substantial efficiency gains by generating multiple tokens using a single target forward pass.<n>Existing SD approaches require the drafter and target models to share the same vocabulary, thus limiting the pool of possible drafters.<n>We present three new SD methods that remove this shared-vocabulary constraint.<n>Our algorithms demonstrate significant speedups of up to 2.8x over standard autoregressive decoding.
  arXiv  Detail & Related papers  (2025-01-31T19:13:58Z)
• Improving Multi-candidate Speculative Decoding [1.6291177798903276]
  Speculative Decoding (SD) is a technique to accelerate the inference of Large Language Models (LLMs)<n>In this work, we introduce a new version of MCSD that includes a target model multi-candidate generation.<n>We also evaluate the effects of using the target model multi-candidate process with different draft models on output quality.
  arXiv  Detail & Related papers  (2024-09-16T18:20:38Z)
• Learning Multi-Aspect Item Palette: A Semantic Tokenization Framework for Generative Recommendation [55.99632509895994]
  We introduce LAMIA, a novel approach for multi-aspect semantic tokenization.<n>Unlike RQ-VAE, which uses a single embedding, LAMIA learns an item palette''--a collection of independent and semantically parallel embeddings.<n>Our results demonstrate significant improvements in recommendation accuracy over existing methods.
  arXiv  Detail & Related papers  (2024-09-11T13:49:48Z)
• Multi-Candidate Speculative Decoding [82.05519287513444]
  Large language models have shown impressive capabilities across a variety of NLP tasks, yet their generating text autoregressively is time-consuming. One way to speed them up is speculative decoding, which generates candidate segments from a fast draft model that is then verified in parallel by the target model. This paper proposes sampling multiple candidates from a draft model and then organising them in batches for verification. We design algorithms for efficient multi-candidate verification while maintaining the distribution of the target model.
  arXiv  Detail & Related papers  (2024-01-12T17:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.