SpecLA: Efficient Speculative Decoding for Linear-Attention Models

• URL: http://arxiv.org/abs/2607.16673v1
• Date: Sat, 18 Jul 2026 07:08:19 GMT
• Title: SpecLA: Efficient Speculative Decoding for Linear-Attention Models
• Abstract summary: SpecLA is a speculative runtime decoding for stateful linear-attention models.<n>On an NVIDIA H100 with a public GDN-1.3B target, SpecLA achieves up to 1.70x end-to-end speedup over autoregressive decoding.
• Score: 23.993269460318704
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Linear-attention models replace the growing KV cache with recurrent states, but autoregressive decoding still reads, updates, and writes these states one token at a time. Speculative decoding can reduce this cost by verifying several draft tokens in one target pass, yet existing speculative systems are designed for Transformer KV caches. For stateful linear-attention targets, verification must follow recurrent dependencies across chains and branches, acceptance must update only the accepted state trajectory, and the drafter must avoid submitting candidates that waste stateful verification work. This paper presents SpecLA, a speculative decoding runtime for stateful linear-attention models. SpecLA verifies chains and trees with topology-aware kernels, stores compact factors produced during verification to recover accepted states, and uses confidence pruning plus a target-aligned EAGLE-style drafter to feed useful candidates to the verifier. On an NVIDIA H100 with a public GDN-1.3B target, SpecLA achieves up to 1.70x end-to-end speedup over autoregressive decoding.

Related papers

• Verification-Aware Training for Speculative Decoding [57.84976863792784]
  We introduce Verification-Aware Training (VAT), a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision.<n>VAT modifies only the training objective, so it can be layered on top of existing methods without changing the draft architecture, the target model, or the inference procedure.<n>VAT improves average acceptance length by up to 11.4% and wall-clock speedup by up to 8.7%, with consistent gains across math, code, and chat benchmarks.
  arXiv  Detail & Related papers  (2026-08-31T01:42:10Z)
• CURE: Local Uncertainty Repair for Block-Parallel Speculative Decoding [6.268849049716121]
  CURE is a budget-aware dynamic repair tree designed to repair errors at uncertainty focal points.<n>We demonstrate that CURE increases the average accepted length by 4.2-7.5% over parallel baselines without repair.<n>We also provide a plug-and-play repair module compatible with standard parallel drafting frameworks.
  arXiv  Detail & Related papers  (2026-08-01T08:43:34Z)
• A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decoding [11.263048239748379]
  We introduce SparseSpec-L, a training-free self-speculative decoding framework for long-context inference.<n>SparseSpec-L generates lightweight drafts directly from the target model using a dynamically sparsified and recallable KV cache.<n> Experiments across multiple long-context tasks and model scales show consistent end-to-end acceleration.
  arXiv  Detail & Related papers  (2026-07-30T06:16:56Z)
• LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding [11.799175681800696]
  Key-value ( KV) caching accelerates inference of large language models (LLMs) by reusing past computations for generated tokens.<n>Existing solutions either restrict reuse to prefixes or require expensive memory materialization for positional re-encoding.<n>We introduce LazyAttention, a novel attention mechanism that kernelizes deferred positional encoding to enable zero-copy, position-agnostic KV reuse.
  arXiv  Detail & Related papers  (2026-06-03T00:12:22Z)
• Making Every Verified Token Count: Adaptive Verification for MoE Speculative Decoding [15.040000425985324]
  Tree-based speculative decoding accelerates autoregressive generation by verifying multiple draft candidates in parallel, but this advantage weakens for sparse Mixture-of-Experts (MoE) models.<n>We propose EVICT, a training-free, hyper parameter-free, and lossless adaptive verification method for MoE speculative decoding.<n> EVICT makes every verified token count by truncating the draft tree before target verification and retaining only the cost-effective prefix.
  arXiv  Detail & Related papers  (2026-05-01T01:52:01Z)
• SpecAttn: Co-Designing Sparse Attention with Self-Speculative Decoding [3.47505166101801]
  SpecAttn is a self-speculative decoding method with verification-guided sparse attention.<n>It achieves 2.81$times$ higher throughput over vanilla auto-regressive decoding and 1.29$times$ improvement over state-of-the-art sparsity-based decoding methods.
  arXiv  Detail & Related papers  (2026-02-06T22:12:52Z)
• SpecPV: Improving Self-Speculative Decoding for Long-Context Generation via Partial Verification [11.366541829206199]
  Speculative decoding is one of the most direct and effective approaches for accelerating generation.<n>We introduce SpecPV, a self-speculative decoding approach that performs fast verification using partial key-value states.<n>We validate SpecPV across multiple long-context benchmarks and models, including LLaMA-3.1-8B-Instruct and Qwen3-series.
  arXiv  Detail & Related papers  (2025-12-02T02:15:33Z)
• SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding [48.96349422252313]
  Speculative decoding has become the standard approach for accelerating Large Language Model (LLM) inference.<n>It exploits a lossless draft-then-verify procedure to circumvent the latency of autoregressive decoding, achieving impressive speed-ups.<n>This paper proposes SpecDiff-2, a novel framework to jointly address these two bottlenecks.
  arXiv  Detail & Related papers  (2025-11-01T16:12:56Z)
• Fast Inference via Hierarchical Speculative Decoding [65.40448210801763]
  We introduce Hierarchical Speculative Decoding (HSD), an algorithm that stacks draft models into a hierarchy, where each model proposes tokens, and the next larger model verifies them in a single forward pass.<n>HSD gives up to 1.2x speed-up over the best single-draft baseline.
  arXiv  Detail & Related papers  (2025-10-22T15:56:19Z)
• REAR: Rethinking Visual Autoregressive Models via Generator-Tokenizer Consistency Regularization [130.46612643194973]
  reAR is a simple training strategy introducing a token-wise regularization objective.<n>On ImageNet, it reduces gFID from 3.02 to 1.86 and improves IS to 316.9 using a standardization-based tokenizer.<n>When applied to advanced tokenizers, it achieves a gFID of 1.42 with only 177M parameters, matching the performance with larger state-of-the-art diffusion models (675M)
  arXiv  Detail & Related papers  (2025-10-06T02:48:13Z)
• DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding [66.40658898418316]
  We present DiffuSpec, a training-free drop-in framework that uses a pretrained diffusion language model (DLM) to produce multi-token drafts in a single forward pass.<n>Across benchmarks, DiffuSpec yields up to 3x wall-clock speedup, establishing diffusion-based drafting as a robust alternative to autoregressive drafters for speculative decoding.
  arXiv  Detail & Related papers  (2025-09-28T07:00:15Z)
• AutoJudge: Judge Decoding Without Manual Annotation [13.451750613294054]
  AutoJudge is a method that accelerates large language model (LLM) inference with task-specific lossy speculative decoding.<n>Our approach relies on a semi-greedy search algorithm to test which of the mismatches between target and draft models should be corrected.
  arXiv  Detail & Related papers  (2025-04-28T17:59:28Z)
• QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache [67.84112700032007]
  Large Language Models (LLMs) are increasingly being deployed on edge devices for long-context settings.<n>In these scenarios, the Key-Value ( KV) cache is the primary bottleneck in terms of both GPU memory and latency.<n>We propose a novel self-speculative decoding framework, QuantSpec, where the draft model shares the architecture of the target model but employs a hierarchical 4-bit quantized KV cache and 4-bit quantized weights for acceleration.
  arXiv  Detail & Related papers  (2025-02-05T20:43:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.