Reducing Compute Waste in LLMs through Kernel-Level DVFS

• URL: http://arxiv.org/abs/2601.08539v1
• Date: Tue, 13 Jan 2026 13:26:57 GMT
• Title: Reducing Compute Waste in LLMs through Kernel-Level DVFS
• Abstract summary: We propose a kernel-level, DVFS approach that explores new frequency configurations, and prove these save more energy than previous, pass- or iteration-level solutions.<n>For a GPT-3 training run, a pass-level approach could reduce energy consumption by 2% (without losing performance), while our kernel-level approach saves as much as 14.6%.
• Score: 0.4747334911695324
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid growth of AI has fueled the expansion of accelerator- or GPU-based data centers. However, the rising operational energy consumption has emerged as a critical bottleneck and a major sustainability concern. Dynamic Voltage and Frequency Scaling (DVFS) is a well-known technique used to reduce energy consumption, and thus improve energy-efficiency, since it requires little effort and works with existing hardware. Reducing the energy consumption of training and inference of Large Language Models (LLMs) through DVFS or power capping is feasible: related work has shown energy savings can be significant, but at the cost of significant slowdowns. In this work, we focus on reducing waste in LLM operations: i.e., reducing energy consumption without losing performance. We propose a fine-grained, kernel-level, DVFS approach that explores new frequency configurations, and prove these save more energy than previous, pass- or iteration-level solutions. For example, for a GPT-3 training run, a pass-level approach could reduce energy consumption by 2% (without losing performance), while our kernel-level approach saves as much as 14.6% (with a 0.6% slowdown). We further investigate the effect of data and tensor parallelism, and show our discovered clock frequencies translate well for both. We conclude that kernel-level DVFS is a suitable technique to reduce waste in LLM operations, providing significant energy savings with negligible slow-down.

Related papers

• Ge$^\ ext{2}$mS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformer [84.8831358775386]
  Spiking Neural Networks (SNNs) offer superior energy efficiency over Artificial Neural Networks (ANNs)<n>Existing paradigms including ANN-SNN Conversion and Spatial-Temporal Backpropagation (STBP) suffer from inherent limitations.<n>We propose Ge$text2$mS-T, a novel architecture implementing grouped computation across temporal, spatial and network structure dimensions.
  arXiv  Detail & Related papers  (2026-04-10T02:58:46Z)
• The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hacking [72.45765726160151]
  This work identifies the Energy Loss Phenomenon in Reinforcement Learning from Human Feedback and its connection to reward hacking.<n>We propose an Energy loss-aware PPO algorithm (EPPO) which penalizes the increase in energy loss in the final layer during reward calculation to prevent excessive energy loss.
  arXiv  Detail & Related papers  (2025-01-31T18:10:53Z)
• On the Effectiveness of Microservices Tactics and Patterns to Reduce Energy Consumption: An Experimental Study on Trade-Offs [3.928499292698212]
  Microservice-based systems have established themselves in the software industry.<n> sustainability-related legislation and the growing costs of energy-hungry software increase the importance of energy efficiency for these systems.<n>While some proposals for architectural tactics and patterns exist, their effectiveness as well as potential trade-offs on other quality attributes (QAs) remain unclear.
  arXiv  Detail & Related papers  (2025-01-24T11:15:23Z)
• TinyML NLP Scheme for Semantic Wireless Sentiment Classification with Privacy Preservation [49.801175302937246]
  This study provides insights into deploying privacy-preserving, energy-efficient NLP models on edge devices.<n>We introduce semantic split learning (SL) as an energy-efficient, privacy-preserving tiny machine learning (TinyML) framework.<n>Our results show that SL significantly reduces computational power and CO2 emissions while enhancing privacy, as evidenced by a fourfold increase in reconstruction error compared to FL and nearly eighteen times that of CL.
  arXiv  Detail & Related papers  (2024-11-09T21:26:59Z)
• Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloads [0.2389598109913753]
  Training and using Large Language Models (LLMs) require large amounts of energy. This paper addresses the challenge of reducing energy consumption in data centers running LLMs. We propose a hybrid data center model that uses a cost-based scheduling framework to dynamically allocate tasks across hardware accelerators.
  arXiv  Detail & Related papers  (2024-04-25T11:24:08Z)
• Energy-Efficient Power Control for Multiple-Task Split Inference in UAVs: A Tiny Learning-Based Approach [27.48920259431965]
  We present a two-timescale approach for energy minimization in split inference in unmanned aerial vehicles (UAVs) We replace the optimization of transmit power with that of transmission time to decrease the computational complexity of OP. Simulation results show that the proposed algorithm can achieve a higher probability of successful task completion with lower energy consumption.
  arXiv  Detail & Related papers  (2023-12-31T10:16:59Z)
• Multiagent Reinforcement Learning with an Attention Mechanism for Improving Energy Efficiency in LoRa Networks [52.96907334080273]
  As the network scale increases, the energy efficiency of LoRa networks decreases sharply due to severe packet collisions. We propose a transmission parameter allocation algorithm based on multiagent reinforcement learning (MALoRa) Simulation results demonstrate that MALoRa significantly improves the system EE compared with baseline algorithms.
  arXiv  Detail & Related papers  (2023-09-16T11:37:23Z)
• A Safe Genetic Algorithm Approach for Energy Efficient Federated Learning in Wireless Communication Networks [53.561797148529664]
  Federated Learning (FL) has emerged as a decentralized technique, where contrary to traditional centralized approaches, devices perform a model training in a collaborative manner. Despite the existing efforts made in FL, its environmental impact is still under investigation, since several critical challenges regarding its applicability to wireless networks have been identified. The current work proposes a Genetic Algorithm (GA) approach, targeting the minimization of both the overall energy consumption of an FL process and any unnecessary resource utilization.
  arXiv  Detail & Related papers  (2023-06-25T13:10:38Z)
• Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models [8.927248087602942]
  We investigate techniques that can be used to reduce the energy consumption of common NLP applications. These techniques can lead to significant reduction in energy consumption when training language models or their use for inference.
  arXiv  Detail & Related papers  (2022-05-19T16:03:55Z)
• Deep Reinforcement Learning Based Multidimensional Resource Management for Energy Harvesting Cognitive NOMA Communications [64.1076645382049]
  Combination of energy harvesting (EH), cognitive radio (CR), and non-orthogonal multiple access (NOMA) is a promising solution to improve energy efficiency. In this paper, we study the spectrum, energy, and time resource management for deterministic-CR-NOMA IoT systems.
  arXiv  Detail & Related papers  (2021-09-17T08:55:48Z)
• Compute and Energy Consumption Trends in Deep Learning Inference [67.32875669386488]
  We study relevant models in the areas of computer vision and natural language processing. For a sustained increase in performance we see a much softer growth in energy consumption than previously anticipated.
  arXiv  Detail & Related papers  (2021-09-12T09:40:18Z)
• Memory-Aware Partitioning of Machine Learning Applications for Optimal Energy Use in Batteryless Systems [17.072240411944914]
  We present Julienning: an automated method for optimizing the total energy cost of batteryless applications. Our method can reduce the required energy storage by over 94% while only incurring a 0.12% energy overhead.
  arXiv  Detail & Related papers  (2021-08-05T09:49:42Z)
• SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training [82.35376405568975]
  Deep neural networks (DNNs) come with heavy parameterization, leading to external dynamic random-access memory (DRAM) for storage. We present SmartDeal (SD), an algorithm framework to trade higher-cost memory storage/access for lower-cost computation. We show that SD leads to 10.56x and 4.48x reduction in the storage and training energy, with negligible accuracy loss compared to state-of-the-art training baselines.
  arXiv  Detail & Related papers  (2021-01-04T18:54:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.