GLIDE: Guided Layerwise Hybrid Attention for Efficient LLM Inference

• URL: http://arxiv.org/abs/2607.24788v1
• Date: Fri, 26 Jun 2026 17:48:13 GMT
• Title: GLIDE: Guided Layerwise Hybrid Attention for Efficient LLM Inference
• Abstract summary: We propose GLIDE, a Guided Layerwise Hybrid Attention that integrates sliding-window softmax attention with linear recurrent aggregation.<n>Unlike uniform hybrid approaches, GLIDE non-uniformly compresses the softmax footprint across the model, reducing aggregate KV cache I/O while preserving expressive power where most vital.<n> Empirical evaluations demonstrate the GLIDE achieves superior performance-efficiency tradeoffs, reducing end-to-end latency for long-context generation without compromising quality.
• Score: 6.410595937323586
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: As Large Language Models scale to increasingly long contexts, the memory I/O and computational overhead of the Key-Value (KV) cache during decoding emerges as the primary throughput bottleneck. To address this, we propose GLIDE, a Guided Layerwise Hybrid Attention that strategically integrates sliding-window softmax attention with linear recurrent aggregation. GLIDE is motivated by layer-wise heterogeneity: early layers exhibit high sensitivity to softmax removal, while deeper layers demonstrate redundancy and tolerate aggressive replacement by linear alternatives. Leveraging this insight, GLIDE introduces a layer-wise adaptive mechanism wherein each layer balances an efficient linear recurrence with a variable-sized softmax window. Unlike uniform hybrid approaches, GLIDE non-uniformly compresses the softmax footprint across the model, reducing aggregate KV cache I/O while preserving expressive power where most vital. Empirical evaluations demonstrate the GLIDE achieves superior performance-efficiency tradeoffs, reducing end-to-end latency for long-context generation without compromising quality.

Related papers

• Morphing into Hybrid Attention Models [52.02140767770562]
  Hybrid attention models improve long-context efficiency by retaining only a subset of full-attention layers and replacing the remaining layers with linear attention.<n>Existing hybrid layer selection methods typically rely on fixed placement patterns or layerwise scoring, implicitly treating layer importance as isolated and overlooking interdependent layer effect under a global hybrid configuration.<n>We propose FlashMorph, an effective, efficient and scalable layer selection method for Transformer-to-hybrid conversion.
  arXiv  Detail & Related papers  (2026-06-29T17:02:34Z)
• LayerBoost: Layer-Aware Attention Reduction for Efficient LLMs [3.80555579179805]
  LayerBoost reduces inference latency and improves throughput by up to 68% at high.<n>It matches base model performance on several benchmarks, exhibits only minor degradations on others, and significantly outperforms state-of-the-art attention linearization methods.
  arXiv  Detail & Related papers  (2026-04-23T20:12:19Z)
• Curvature-Weighted Capacity Allocation: A Minimum Description Length Framework for Layer-Adaptive Large Language Model Optimization [8.029535985033485]
  Layer-wise capacity in large language models is non-uniform; some layers contribute disproportionately to loss reduction while others are near-redundant.<n>Existing methods for exploiting this non-uniformity, such as influence-function-based layer scoring, produce sensitivity estimates but offer no principled mechanism for translating them into allocation or pruning decisions.<n>We address this gap with a unified, curvature-aware framework grounded in the Minimum Description Length (MDL) principle.
  arXiv  Detail & Related papers  (2026-03-01T04:14:15Z)
• HyLRA: Hybrid Layer Reuse Attention for Efficient Long-Context Inference [11.718567830546538]
  Long-context inference in Large Language Models is bottlenecked by the quadratic computation complexity of attention.<n>We introduce bf HyLRA, a novel framework driven by layer-wise sparsity profiling.<n>We show that HyLRA improves inference throughput by 6%--46% while maintaining comparable performance.
  arXiv  Detail & Related papers  (2026-01-31T15:36:17Z)
• SoLA-Vision: Fine-grained Layer-wise Linear Softmax Hybrid Attention [50.99430451151184]
  Linear attention reduces the cost to O(N), yet its compressed state representations can impair modeling capacity and accuracy.<n>We present an analytical study that contrasts linear and softmax attention for visual representation learning.<n>We propose SoLA-Vision, a flexible layer-wise hybrid attention backbone.
  arXiv  Detail & Related papers  (2026-01-16T10:26:53Z)
• Distilling to Hybrid Attention Models via KL-Guided Layer Selection [66.06591032073744]
  This paper describes a simple and efficient recipe for layer selection that uses layer importance scores derived from a small amount of training on generic text data.<n>We find that this approach is more effective than existing approaches for layer selection, including approaches that uniformly interleave linear attentions based on a fixed ratio.
  arXiv  Detail & Related papers  (2025-12-23T18:12:22Z)
• FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping [49.66872823080736]
  Autoregressive Large Language Models (e.g., LLaMa, GPTs) are omnipresent achieving remarkable success in language understanding and generation. To mitigate overload incurred during generation, several early-exit and layer-dropping strategies have been proposed. We propose FFN-SkipLLM, which is an input-adaptive feed-forward skipping strategy.
  arXiv  Detail & Related papers  (2024-04-05T02:35:43Z)
• ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models [70.45441031021291]
  Large Vision-Language Models (LVLMs) can understand the world comprehensively by integrating rich information from different modalities. LVLMs are often problematic due to their massive computational/energy costs and carbon consumption. We propose Efficient Coarse-to-Fine LayerWise Pruning (ECoFLaP), a two-stage coarse-to-fine weight pruning approach for LVLMs.
  arXiv  Detail & Related papers  (2023-10-04T17:34:00Z)
• Layer-adaptive sparsity for the Magnitude-based Pruning [88.37510230946478]
  We propose a novel importance score for global pruning, coined layer-adaptive magnitude-based pruning (LAMP) score. LAMP consistently outperforms popular existing schemes for layerwise sparsity selection.
  arXiv  Detail & Related papers  (2020-10-15T09:14:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.