Understanding the Energy Scaling of Large Language Model Inference Across Context Lengths and Attention Architectures

• URL: http://arxiv.org/abs/2608.25096v1
• Date: Tue, 25 Aug 2026 19:45:30 GMT
• Title: Understanding the Energy Scaling of Large Language Model Inference Across Context Lengths and Attention Architectures
• Abstract summary: We study how attention architecture influences decode-phase energy consumption under varying inference workloads.<n>We show that attention is the primary factor governing how decode energy scales with context length.<n>Model size determines absolute energy consumption, while reducing both energy per generated token and latency by up to 87%.
• Score: 0.2519906683279152
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The growing adoption of large language models (LLMs) has raised increasing concerns about the energy consumption and environmental impact of inference. This paper presents a systematic empirical study of decode-phase energy consumption across representative open-source LLMs employing Multi-Head Attention (MHA), Grouped Query Attention (GQA), and Grouped Query Attention with Sliding Window Attention (SWA) to characterize how attention architecture influences decode-phase energy consumption under varying inference workloads. We evaluate four models across different context lengths, batch sizes, and generation workloads while measuring GPU energy using NVIDIA hardware counters. We examine the effects of context length, attention mechanism, Key-Value (KV) cache growth, and batching on decode-phase energy consumption. Results show that attention mechanism is the primary factor governing how decode energy scales with context length. MHA models exhibit substantially steeper energy growth than GQA models, whereas GQA with SWA maintains nearly constant energy consumption. We further show that model size primarily determines absolute energy consumption, while batching reduces both energy per generated token and request latency by up to 87%. These findings provide practical guidance for selecting energy-efficient LLM architectures and inference configurations.

Related papers

• How Do Prompt Variations Affect Energy Consumption in On-Device LLMs? [15.386321055792209]
  Large language models (LLMs) are increasingly deployed on mobile devices, making energy efficiency a key deployment constraint.<n>This paper aims to understand how two prompt properties, cognitive load and phrasing pattern, shape the energy behavior of on-device LLM inference.
  arXiv  Detail & Related papers  (2026-09-01T19:08:04Z)
• The Energy Footprint of LLM-Based Environmental Analysis: LLMs and Domain Products [21.239678391166578]
  Large language models (LLMs) are increasingly used in domain-specific applications, including climate change and environmental research.<n>How does the energy consumption of domain-specific RAGs compare with that of direct generic LLMs?<n>We estimate energy use under actual user queries by decomposing each workflow into retrieval, generation, and hallucination-checking components.
  arXiv  Detail & Related papers  (2026-03-31T00:19:07Z)
• Towards Green AI: Decoding the Energy of LLM Inference in Software Development [46.879983975894135]
  AI-assisted tools are increasingly integrated into software development, but their reliance on large language models (LLMs) introduces substantial computational and energy costs.<n>We conduct a phase-level analysis of LLM inference energy consumption, distinguishing between the (1) prefill, where the model processes the input and builds internal representations, and (2) decoding, where output tokens are generated using the stored state.
  arXiv  Detail & Related papers  (2026-02-05T14:38:19Z)
• Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression [53.39128997308138]
  We introduce information capacity, a measure of model efficiency based on text compression performance.<n> Empirical evaluations on mainstream open-source models show that models of varying sizes within a series exhibit consistent information capacity.<n>A distinctive feature of information capacity is that it incorporates tokenizer efficiency, which affects both input and output token counts.
  arXiv  Detail & Related papers  (2025-11-11T10:07:32Z)
• From Prompts to Power: Measuring the Energy Footprint of LLM Inference [1.9116784879310027]
  We present a large-scale measurement-based study comprising over 32,500 measurements across 21 GPU configurations and 155 model architectures.<n>Using the vLLM inference engine, we quantify energy usage at the prompt level and identify how architectural and operational factors shape energy demand.<n>We develop a predictive model that accurately estimates inference energy consumption across unseen architectures and hardware, and implement it as a browser extension to raise awareness of the environmental impact of generative AI.
  arXiv  Detail & Related papers  (2025-11-05T15:06:46Z)
• Comparing energy consumption and accuracy in text classification inference [0.9208007322096533]
  This study systematically evaluates the trade-offs between model accuracy and energy consumption in text classification inference.<n>The best-performing model in terms of accuracy can also be energy-efficient, while larger LLMs tend to consume significantly more energy with lower classification accuracy.
  arXiv  Detail & Related papers  (2025-08-19T18:00:08Z)
• Green MLOps to Green GenOps: An Empirical Study of Energy Consumption in Discriminative and Generative AI Operations [2.2765705959685234]
  This study investigates the energy consumption of Discriminative and Generative AI models within real-world MLOps pipelines.<n>We employ software-based power measurements to ensure ease of replication across diverse configurations, models, and datasets.
  arXiv  Detail & Related papers  (2025-03-31T10:28:04Z)
• Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding [58.364933651703524]
  We show that concentrated massive values consistently emerge in specific regions of attention queries.<n>These massive values play a critical role in interpreting contextual knowledge.<n>We trace the emergence of massive values and find that such concentration is caused by Rotary Positional.
  arXiv  Detail & Related papers  (2025-02-03T17:47:03Z)
• Exploring the Privacy-Energy Consumption Tradeoff for Split Federated Learning [51.02352381270177]
  Split Federated Learning (SFL) has recently emerged as a promising distributed learning technology. The choice of the cut layer in SFL can have a substantial impact on the energy consumption of clients and their privacy. This article provides a comprehensive overview of the SFL process and thoroughly analyze energy consumption and privacy.
  arXiv  Detail & Related papers  (2023-11-15T23:23:42Z)
• Enhancing Energy-Awareness in Deep Learning through Fine-Grained Energy Measurement [11.37120215795946]
  This paper introduces FECoM (Fine-grained Energy Consumption Meter), a framework for fine-grained Deep Learning energy consumption measurement. FECoM addresses the challenges of measuring energy consumption at fine-grained level by using static instrumentation and considering various factors, including computational load stability and temperature.
  arXiv  Detail & Related papers  (2023-08-23T17:32:06Z)
• Energy Transformer [64.22957136952725]
  Our work combines aspects of three promising paradigms in machine learning, namely, attention mechanism, energy-based models, and associative memory. We propose a novel architecture, called the Energy Transformer (or ET for short), that uses a sequence of attention layers that are purposely designed to minimize a specifically engineered energy function.
  arXiv  Detail & Related papers  (2023-02-14T18:51:22Z)
• Attention Mechanism with Energy-Friendly Operations [61.58748425876866]
  We rethink attention mechanism from the energy consumption aspects. We build a novel attention model by replacing multiplications with either selective operations or additions. Empirical results on three machine translation tasks demonstrate that the proposed model achieves competitable accuracy.
  arXiv  Detail & Related papers  (2022-04-28T08:50:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.