GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding

• URL: http://arxiv.org/abs/2605.15250v1
• Date: Thu, 14 May 2026 15:50:01 GMT
• Title: GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding
• Abstract summary: Group-Query Latent Attention (GQLA) is a minimal modification of Multi-head Latent Attention (MLA)<n>Group-Query Latent Attention (GQLA) is a minimal modification of Multi-head Latent Attention (MLA)
• Score: 5.930949606554186
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-head Latent Attention (MLA), the attention used in DeepSeek-V2/V3, jointly compresses keys and values into a low-rank latent and matches the H100 roofline almost perfectly. Its trained weights, however, expose only one decoding path - an absorbed MQA form - which ties efficient inference to H100-class compute-bandwidth ratios, forfeits tensor parallelism along the head axis, and yields no Multi-Token Prediction (MTP) gain on commodity inference GPUs such as the export-restricted H20. We propose Group-Query Latent Attention (GQLA), a minimal modification of MLA whose trained weights expose two algebraically equivalent decoding paths over the same parameters: an MQA-absorb path identical to MLA's, and a GQA path with a per-group expanded cache. The runtime picks the path that matches the target hardware - no retraining, no custom kernels - so a single set of GQLA weights pins the rooflines of both H100 (MQA-absorb, s_q=1) and H20 (GQA + MTP, s_q=2), while supporting up to 8-way zero-redundancy tensor parallelism on the GQA path. To avoid pretraining from scratch we extend TransMLA into TransGQLA, which converts a pretrained GQA checkpoint into a GQLA model; on LLaMA-3-8B it compresses the per-token KV cache to 28.125% of the GQA baseline on the MQA-absorb path while structurally preserving GQA-level traffic on the per-group path.

Related papers

• QK-Normed MLA: QK normalization without full key caching [65.51862319225098]
  We show that QK normalization is a practical stabilization option for MLA models without requiring full-key caching.<n>In our 400M runs trained for up to 100B tokens, QK-Normed MLA achieves lower training loss and better downstream accuracy than QK clipping.
  arXiv  Detail & Related papers  (2026-06-15T07:16:30Z)
• MiniMax Sparse Attention [16.750980846187343]
  MiniMax Sparse Attention (MSA) is a blockwise sparse attention built upon Grouped Query Attention (GQA)<n>A lightweight Index Branch scores key-value blocks and independently selects a Top-k subset for each GQA group.<n>The Main Branch then performs exact block-sparse attention over only the selected blocks.
  arXiv  Detail & Related papers  (2026-06-11T14:23:41Z)
• Minimizing the Hidden Cost of Scales: Graph-Guided Ultra-Low-Bit Quantization for Large Language Models [50.16014098038291]
  Post-training quantization (PTQ) is critical for the efficient deployment of large language models (LLMs)<n>We propose SAGE-PTQ, a novel ultra-low-bit quantization framework for LLMs that minimizes hidden scaling cost.<n>On LLaMA-3B, SAGE-PTQ achieves 6.74 WikiText2 perplexity, compared to 55.8 for BiLLM, while using less than 50% of BiLLM's GPU memory.
  arXiv  Detail & Related papers  (2026-06-03T20:51:52Z)
• Latent-Condensed Transformer for Efficient Long Context Modeling [60.72493959155964]
  We propose Latent-Condensed Attention (LCA) for large language models.<n>LCA condenses context within MLA's latent space, where representation is disentangled into semantic latent vectors and positional keys.<n>LCA achieves up to 2.5$times$ prefilling speedup and 90% KV cache reduction at 128K context.
  arXiv  Detail & Related papers  (2026-04-14T08:40:31Z)
• Compressed Convolutional Attention: Efficient Attention in a Compressed Latent Space [12.98205656003145]
  Multi-headed Attention's (MHA) quadratic compute and linearly growing KV-cache make long-context transformers expensive to train and serve.<n>We introduce Compressed Convolutional Attention (CCA), a novel attention method which down-projects queries, keys, and values and performs the entire attention operation inside the shared latent space.<n>Experiments show CCGQA consistently outperforms both Grouped Query Attention (GQA) and Multi-Latent Attention (MLA) at equal KV-cache compression on dense and MoE models.
  arXiv  Detail & Related papers  (2025-10-06T04:24:23Z)
• TPLA: Tensor Parallel Latent Attention for Efficient Disaggregated Prefill and Decode Inference [48.40143137402824]
  Multi-Head Latent Attention (MLA) compresses key-value states into a low-rank latent vector, caching only this vector to reduce memory.<n>In tensor parallelism (TP), however, attention heads are computed across multiple devices, and each device must load the full cache.<n>We propose-Parallel Latent Attention (TPLA), a scheme that partitions both the latent representation and each head's input dimension across devices, performs attention independently per shard, and then combines results with an all-reduce.
  arXiv  Detail & Related papers  (2025-08-21T15:25:40Z)
• TransMLA: Multi-Head Latent Attention Is All You Need [34.38934956358534]
  TransMLA is a framework that seamlessly converts GQA-based models to MLA-based models.<n>By compressing 93% of the KV cache in LLaMA-2-7B, TransMLA achieves a 10.6x inference speedup at an 8K context length.
  arXiv  Detail & Related papers  (2025-02-11T18:20:18Z)
• GWQ: Gradient-Aware Weight Quantization for Large Language Models [56.22507677736051]
  Large language models (LLMs) show impressive performance in solving complex language tasks.<n> compressing LLMs to low bits can enable to deploy on resource-constrained devices.<n>We propose gradient-aware weight quantization (GWQ), the first quantization approach for low-bit weight quantization.
  arXiv  Detail & Related papers  (2024-10-30T11:16:04Z)
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints [25.154477500940626]
  We propose a recipe for uptraining existing multi-head language model checkpoints into models with MQA using 5% of original pre-training compute. We show that uptrained GQA achieves quality close to multi-head attention with comparable speed to MQA.
  arXiv  Detail & Related papers  (2023-05-22T17:16:38Z)
• Pruning Self-attentions into Convolutional Layers in Single Path [89.55361659622305]
  Vision Transformers (ViTs) have achieved impressive performance over various computer vision tasks. We propose Single-Path Vision Transformer pruning (SPViT) to efficiently and automatically compress the pre-trained ViTs. Our SPViT can trim 52.0% FLOPs for DeiT-B and get an impressive 0.6% top-1 accuracy gain simultaneously.
  arXiv  Detail & Related papers  (2021-11-23T11:35:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.