From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

• URL: http://arxiv.org/abs/2607.26571v1
• Date: Wed, 29 Jul 2026 07:50:55 GMT
• Title: From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs
• Abstract summary: This report presents an analytically structured, empirically calibrated, GPU-level methodology for estimating large language model (LLM) inference energy on NVIDIA H100-class accelerators without direct measurement.<n>The proposed estimator combines parameter-scaled transformer FLOP accounting, memory-traffic factors, and hardware-specific energy coefficients for FP16/BF16 tensor-core computation and high-bandwidth-memory movement.<n>The methodology further decomposes total energy into compute, parameter-access, key-value-cache write, and attention-read components, allowing the scaling behavior with model size, context length, and generated-token count to
• Score: 6.041312734387813
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentation, or infrastructure-specific monitoring, limiting its applicability in comparative studies, early-stage system design, and sustainability reporting. This report presents an analytically structured, empirically calibrated, GPU-level methodology for estimating LLM inference energy on NVIDIA H100-class accelerators without direct runtime measurement. The proposed estimator combines parameter-scaled transformer FLOP accounting, calibrated memory-traffic factors, and hardware-specific energy coefficients for FP16/BF16 tensor-core computation and high-bandwidth-memory movement. It explicitly separates prompt prefill from autoregressive decoding, enabling energy estimates for input tokens, output tokens, and complete inference requests. The methodology further decomposes total energy into compute, parameter-access, key-value-cache write, and attention-read components, allowing the scaling behavior with model size, context length, and generated-token count to be analyzed. The resulting estimates are not intended to replace physical power measurements; rather, they provide transparent, reproducible, and assumption-explicit approximations suitable for model comparison, green-coding analysis, and design-time evaluation of LLM inference workloads.

Related papers

• Beyond Test-Time Compute Strategies: Advocating Energy-per-Token in LLM Inference [1.522424334864671]
  Large Language Models (LLMs) demonstrate exceptional performance across diverse tasks but come with substantial energy and computational costs.<n>Small Language Models (SLMs) can provide accurate responses for simpler text generation tasks.<n>These strategies can also introduce additional energy costs, creating an energy-accuracy trade-off.
  arXiv  Detail & Related papers  (2026-03-04T15:20:55Z)
• Towards Green AI: Decoding the Energy of LLM Inference in Software Development [46.879983975894135]
  AI-assisted tools are increasingly integrated into software development, but their reliance on large language models (LLMs) introduces substantial computational and energy costs.<n>We conduct a phase-level analysis of LLM inference energy consumption, distinguishing between the (1) prefill, where the model processes the input and builds internal representations, and (2) decoding, where output tokens are generated using the stored state.
  arXiv  Detail & Related papers  (2026-02-05T14:38:19Z)
• Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use [4.513690948889834]
  Large Language Models (LLMs) are increasingly deployed in production, contributing towards shifting the burden in terms of computational resources and energy demands from training to inference.<n>We show how emphsystem-level design choices can lead to orders-of-magnitude differences in energy consumption for the same model.<n>Our findings motivate phase-aware energy profiling and system-level optimizations for greener AI services.
  arXiv  Detail & Related papers  (2026-01-29T22:16:25Z)
• Energy Scaling Laws for Diffusion Models: Quantifying Compute and Carbon Emissions in Image Generation [50.21021246855702]
  We propose an adaptation of Kaplan scaling laws to predict GPU energy consumption for diffusion models based on computational complexity (FLOPs)<n>Our approach decomposes diffusion model inference into text encoding, iterative denoising, and decoding components, with the hypothesis that denoising operations dominate energy consumption due to their repeated execution across multiple inference steps.<n>Our results validate the compute-bound nature of diffusion inference and provide a foundation for sustainable AI deployment planning and carbon footprint estimation.
  arXiv  Detail & Related papers  (2025-11-21T08:12:47Z)
• Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression [53.39128997308138]
  We introduce information capacity, a measure of model efficiency based on text compression performance.<n> Empirical evaluations on mainstream open-source models show that models of varying sizes within a series exhibit consistent information capacity.<n>A distinctive feature of information capacity is that it incorporates tokenizer efficiency, which affects both input and output token counts.
  arXiv  Detail & Related papers  (2025-11-11T10:07:32Z)
• Quantifying Memory Utilization with Effective State-Size [73.52115209375343]
  We develop a measure of textitmemory utilization'<n>This metric is tailored to the fundamental class of systems with textitinput-invariant and textitinput-varying linear operators
  arXiv  Detail & Related papers  (2025-04-28T08:12:30Z)
• Energy Considerations of Large Language Model Inference and Efficiency Optimizations [28.55549828393871]
  As large language models (LLMs) scale in size and adoption, their computational and environmental costs continue to rise.<n>We systematically analyze the energy implications of common inference efficiency optimizations across diverse NLP and AI workloads.<n>Our findings reveal that the proper application of relevant inference efficiency optimizations can reduce total energy use by up to 73% from unoptimized baselines.
  arXiv  Detail & Related papers  (2025-04-24T15:45:05Z)
• Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency [6.306413686006502]
  We conduct a comprehensive analysis of 28 quantized Large Language Models (LLMs) from the Ollama library.<n>We evaluate energy efficiency, inference performance, and output accuracy across multiple quantization levels and task types.<n>Our findings reveal the trade-offs between energy efficiency, inference speed, and accuracy in different quantization settings.
  arXiv  Detail & Related papers  (2025-04-04T11:29:30Z)
• Latent Thought Models with Variational Bayes Inference-Time Computation [52.63299874322121]
  Latent Thought Models (LTMs) incorporate explicit latent thought vectors that follow an explicit prior model in latent space.<n>LTMs demonstrate superior sample and parameter efficiency compared to autoregressive models and discrete diffusion models.
  arXiv  Detail & Related papers  (2025-02-03T17:50:34Z)
• From Dense to Sparse: Event Response for Enhanced Residential Load Forecasting [48.22398304557558]
  We propose an Event-Response Knowledge Guided approach (ERKG) for residential load forecasting.<n>ERKG incorporates the estimation of electricity usage events for different appliances, mining event-related sparse knowledge from the load series.
  arXiv  Detail & Related papers  (2025-01-06T05:53:38Z)
• Computing Within Limits: An Empirical Study of Energy Consumption in ML Training and Inference [2.553456266022126]
  Machine learning (ML) has seen tremendous advancements, but its environmental footprint remains a concern. Acknowledging the growing environmental impact of ML this paper investigates Green ML.
  arXiv  Detail & Related papers  (2024-06-20T13:59:34Z)
• Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures [68.91874045918112]
  adapter-ALBERT is an efficient model optimization for maximal data reuse across different tasks. We demonstrate the advantage of mapping the model to a heterogeneous on-chip memory architecture by performing simulations on a validated NLP edge accelerator.
  arXiv  Detail & Related papers  (2023-03-25T14:40:59Z)
• Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data [66.11139091362078]
  We provide the first model selection results on large pretrained Transformers from Huggingface using generalization metrics. Despite their niche status, we find that metrics derived from the heavy-tail (HT) perspective are particularly useful in NLP tasks.
  arXiv  Detail & Related papers  (2022-02-06T20:07:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.