Component-Aware Self-Speculative Decoding in Hybrid Language Models

• URL: http://arxiv.org/abs/2605.01106v1
• Date: Fri, 01 May 2026 21:25:25 GMT
• Title: Component-Aware Self-Speculative Decoding in Hybrid Language Models
• Abstract summary: Speculative decoding accelerates autoregressive inference by drafting candidate tokens with a fast model.<n>We introduce component-aware self-speculative decoding, isolating the SSM/linear-attention subgraph as a zero-cost internal draft.<n>We show that generic LayerSkip achieves 12x higher acceptance rates than the component-aware strategy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speculative decoding accelerates autoregressive inference by drafting candidate tokens with a fast model and verifying them in parallel with the target. Self-speculative methods avoid the need for an external drafter but have been studied exclusively in homogeneous Transformer architectures. We introduce component-aware self-speculative decoding, the first method to exploit the internal architectural heterogeneity of hybrid language models, isolating the SSM/linear-attention subgraph as a zero-cost internal draft. We evaluate this on two architecturally distinct hybrid families: Falcon-H1 (parallel: Mamba-2 + attention per layer) and Qwen3.5 (sequential: interleaved linear and attention layers), with a pure Transformer control (Qwen2.5). Parallel hybrids achieve acceptance rates of alpha = 0.68 at draft length k=2 under greedy decoding, while sequential hybrids yield only alpha = 0.038 -- an 18x gap attributable to how each architecture integrates its components. The property is scale-invariant: Falcon-H1 at 3B reproduces the rates observed at 0.5B. We further show that perplexity degradation from a companion ablation study predicts speculative viability without running speculative decoding: a 3.15x ratio (Falcon) maps to alpha = 0.37 at k=4, while 81.96x (Qwen) maps to alpha = 0.019. For sequential hybrids, generic LayerSkip achieves 12x higher acceptance rates than the component-aware strategy. The composition pattern of hybrid models -- not merely the presence of alternative components -- determines whether component-level self-speculation is viable.

Related papers

• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• Priming: Hybrid State Space Models From Pre-trained Transformers [45.220597209553866]
  Priming is a method that turns Hybrid architecture design from a pre-training problem into a knowledge transfer one.<n>We evaluate, Gated KalmaNet (GKA), Gated DeltaNet (GDN), and Mamba-2, and show that their hierarchy, GKA>GDN>Mamba-2, directly predicts downstream performance on long-context reasoning tasks.<n>Our Hybrid GKA 32B improves over its source Qwen3-32B by +3.8 average reasoning points, while staying within 1% of a Transformer post-trained on the same data.
  arXiv  Detail & Related papers  (2026-05-08T11:43:51Z)
• Where Should LoRA Go? Component-Type Placement in Hybrid Language Models [0.0]
  We study component-type LoRA placement across two hybrid architectures.<n>We find that the attention pathway consistently outperforms full-model adaptation with 5-10x fewer trainable parameters.
  arXiv  Detail & Related papers  (2026-04-24T00:20:29Z)
• Functional Component Ablation Reveals Specialization Patterns in Hybrid Language Model Architectures [0.0]
  We present a functional component ablation framework applied to two sub-1B hybrid models.<n>We find that both component types are essential and neither is bypassed.<n>These results provide actionable guidance for hybrid model compression, architecture design, and fault-tolerant deployment.
  arXiv  Detail & Related papers  (2026-03-23T18:41:49Z)
• Cross-Family Speculative Decoding for Polish Language Models on Apple~Silicon: An Empirical Evaluation of Bielik~11B with UAG-Extended MLX-LM [0.0]
  We extend the MLX-LM framework with Universal Assisted Generation (UAG) to enable cross-tokenizer speculative decoding on Apple Silicon.<n>Experiments on three Polish-language datasets (Wikipedia, pl_alpaca, synthetic) use draft lengths k in 2, 4, 6 to compare naive and context-aware token translation.
  arXiv  Detail & Related papers  (2026-03-22T19:07:41Z)
• LINA: Linear Autoregressive Image Generative Models with Continuous Tokens [56.80443965097921]
  Autoregressive models with continuous tokens form a promising paradigm for visual generation, especially for text-to-image (T2I) synthesis.<n>We study how to design compute-efficient linear attention within this framework.<n>We present LINA, a simple and compute-efficient T2I model built entirely on linear attention, capable of generating high-fidelity 1024x1024 images from user instructions.
  arXiv  Detail & Related papers  (2026-01-30T06:44:33Z)
• PanFoMa: A Lightweight Foundation Model and Benchmark for Pan-Cancer [54.958921946378304]
  We introduce PanFoMa, a lightweight hybrid neural network that combines the strengths of Transformers and state-space models.<n>PanFoMa consists of a front-end local-context encoder with shared self-attention layers to capture complex, order-independent gene interactions.<n>We also construct a large-scale pan-cancer single-cell benchmark, PanFoMaBench, containing over 3.5 million high-quality cells.
  arXiv  Detail & Related papers  (2025-12-02T08:31:31Z)
• A Systematic Analysis of Hybrid Linear Attention [11.722015123070957]
  Linear models often suffer from limited recall performance.<n>Our study highlights selective gating, hierarchical recurrence, and controlled forgetting as critical for effective hybrid models.<n>Our models are open-sourced at https://huggingface.co/collections/m-hugging-a-p/hybrid-linear-attention-research-686c488a63d609d2f2 0e2b1e.
  arXiv  Detail & Related papers  (2025-07-08T23:54:11Z)
• Systems and Algorithms for Convolutional Multi-Hybrid Language Models at Scale [68.6602625868888]
  We introduce convolutional multi-hybrid architectures, with a design grounded on two simple observations.<n>Operators in hybrid models can be tailored to token manipulation tasks such as in-context recall, multi-token recall, and compression.<n>We train end-to-end 1.2 to 2.9 times faster than optimized Transformers, and 1.1 to 1.4 times faster than previous generation hybrids.
  arXiv  Detail & Related papers  (2025-02-25T19:47:20Z)
• Squeezeformer: An Efficient Transformer for Automatic Speech Recognition [99.349598600887]
  Conformer is the de facto backbone model for various downstream speech tasks based on its hybrid attention-convolution architecture. We propose the Squeezeformer model, which consistently outperforms the state-of-the-art ASR models under the same training schemes.
  arXiv  Detail & Related papers  (2022-06-02T06:06:29Z)
• Adaptive Split-Fusion Transformer [90.04885335911729]
  We propose an Adaptive Split-Fusion Transformer (ASF-former) to treat convolutional and attention branches differently with adaptive weights. Experiments on standard benchmarks, such as ImageNet-1K, show that our ASF-former outperforms its CNN, transformer counterparts, and hybrid pilots in terms of accuracy.
  arXiv  Detail & Related papers  (2022-04-26T10:00:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.