EnerInfer: Energy-Aware On-Device LLM Inference

• URL: http://arxiv.org/abs/2606.23001v2
• Date: Wed, 24 Jun 2026 14:38:38 GMT
• Title: EnerInfer: Energy-Aware On-Device LLM Inference
• Abstract summary: EnerInfer is a framework that jointly manages energy efficiency, throughput, and thermal comfort for LLM workloads.<n>EnerInfer improves energy efficiency by up to 65%, 12%, and 24% on phones, a laptop, and a development board, respectively.
• Score: 26.942672033524826
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: On-device LLM inference is increasingly attractive for privacy-preserving, reliable, and cost-effective deployment, yet its energy and thermal costs remain a critical bottleneck. Existing systems primarily optimize for decoding speed, implicitly assuming that faster execution is always preferable. We show instead that on-device LLM inference often has exploitable configuration slack: modestly lowering NPU and memory frequencies preserves quality of experience (QoE) while substantially improving energy efficiency and reducing heat. Realizing this opportunity in production is challenging. The most energy-efficient NPU/DDR setting varies with the model, inference engine, platform, and runtime conditions, with no stable ranking across configurations. Commercial devices further lack component-level power sensing, and shell temperature evolves with request arrivals, response lengths, and thermal history. To address these challenges, we propose EnerInfer, the first on-device LLM inference framework that jointly manages energy efficiency, throughput, and thermal comfort for LLM workloads. EnerInfer replaces per-model profiling and sensor-heavy control with disaggregated, model-structure-aware prediction and ranking-driven online feedback. It predicts throughput and power for unseen LLMs across NPU/DDR frequency settings, selects QoE-satisfying efficient configurations under runtime interference, and uses lightweight limited-horizon thermal prediction to dynamically switch between energy-optimized and thermally constrained inference. Evaluations on real-world LLMs show that EnerInfer improves energy efficiency by up to 65%, 12%, and 24% on phones, a laptop, and a development board, respectively, without QoE violation.

Related papers

• LLM Inference in a Flash! [66.43896490006965]
  Large Language Models (LLMs) have shown impressive capabilities across a range of natural language processing tasks.<n>We present an end-to-end integer-only quantization approach to eliminate expensive floating-point computations.
  arXiv  Detail & Related papers  (2026-09-14T18:04:30Z)
• PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling [0.42015796373418257]
  Large Language Models (LLMs) can be deployed directly on mobile platforms at the edge.<n>LLMs have heavy computational requirements, which are difficult for resource-constrained mobile and edge platforms to fulfill.<n>We present PELM, a solution that augments traditional DVFS processor frequency tuning with two additional workload-specific knobs.
  arXiv  Detail & Related papers  (2026-09-09T03:27:51Z)
• PALS: Power-Aware LLM Serving for Mixture-of-Experts Models [7.28734124763028]
  Large language model (LLM) inference has become a dominant workload in modern data centers.<n>We present a power-aware runtime for LLM serving, PALS, that treats GPU power caps as a first-class control knob.<n> PALS improves energy efficiency by up to 26.3%, reduces runtime violations by 4x to 7x under power constraints, and tracks dynamic power budgets.
  arXiv  Detail & Related papers  (2026-05-20T17:19:20Z)
• Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use [4.513690948889834]
  Large Language Models (LLMs) are increasingly deployed in production, contributing towards shifting the burden in terms of computational resources and energy demands from training to inference.<n>We show how emphsystem-level design choices can lead to orders-of-magnitude differences in energy consumption for the same model.<n>Our findings motivate phase-aware energy profiling and system-level optimizations for greener AI services.
  arXiv  Detail & Related papers  (2026-01-29T22:16:25Z)
• Energy-Driven Steering: Reducing False Refusals in Large Language Models [80.09252175869858]
  We introduce Energy-Driven Steering (EDS), a novel, fine-tuning free framework designed to resolve this challenge through dynamic, inference-time intervention.<n>We trained a lightweight, external Energy-Based Model (EBM) to assign high energy to undesirable (false refusal or jailbreak) states and low energy to desirable (helpful response or safe reject) ones.<n>We use the gradient of the energy function to dynamically steer the LLM's hidden states to low energy regions, correcting the model to generate a desirable response in real-time without modifying its weights.
  arXiv  Detail & Related papers  (2025-10-09T06:01:41Z)
• TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs [67.55973229034319]
  This paper introduces TempSamp-R1, a new reinforcement fine-tuning framework designed to improve the effectiveness of adapting multimodal large language models (MLLMs) to video temporal grounding tasks.<n>We show that TempSamp-R1 outperforms GRPO-based baselines, establishing new state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2025-09-22T17:30:15Z)
• CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems [62.24576366776727]
  We propose a latency-aware scheduling framework to minimize total inference latency.<n>We show that the proposed method significantly reduces cold-start latency compared to baseline strategies.
  arXiv  Detail & Related papers  (2025-08-15T07:49:22Z)
• Dissecting the Impact of Mobile DVFS Governors on LLM Inference Performance and Energy Efficiency [20.904706759529237]
  Large Language Models (LLMs) are increasingly being integrated into various applications and services running on billions of mobile devices.<n>Currently, deploying LLMs on resource-limited mobile devices faces a significant challenge due to their high demand for computation, memory, and ultimately energy.
  arXiv  Detail & Related papers  (2025-07-02T20:47:40Z)
• TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms [9.36320423249322]
  The rising demand for generative large language models (LLMs) poses challenges for thermal and power management in the cloud.<n>We propose TAPAS, a thermal-aware framework designed for LLM inference clusters in the cloud.<n>Our evaluation on a large GPU cluster demonstrates significant reductions in thermal and power throttling events, boosting system efficiency.
  arXiv  Detail & Related papers  (2025-01-05T16:51:17Z)
• Age-Based Device Selection and Transmit Power Optimization in Over-the-Air Federated Learning [44.04728314657621]
  Over-the-air federated learning (FL) has attracted significant attention for its ability to enhance communication efficiency.<n>In particular, neglecting straggler devices in FL can lead to a decline in the fairness of model updates and amplify the global model's bias toward certain devices' data.<n>We propose a joint device selection and transmit power optimization framework that ensures the appropriate participation of straggler devices, maintains efficient training performance, and guarantees timely updates.
  arXiv  Detail & Related papers  (2025-01-03T14:27:13Z)
• TinyML NLP Scheme for Semantic Wireless Sentiment Classification with Privacy Preservation [49.801175302937246]
  This study provides insights into deploying privacy-preserving, energy-efficient NLP models on edge devices.<n>We introduce semantic split learning (SL) as an energy-efficient, privacy-preserving tiny machine learning (TinyML) framework.<n>Our results show that SL significantly reduces computational power and CO2 emissions while enhancing privacy, as evidenced by a fourfold increase in reconstruction error compared to FL and nearly eighteen times that of CL.
  arXiv  Detail & Related papers  (2024-11-09T21:26:59Z)
• MELTing point: Mobile Evaluation of Language Transformers [8.238355633015068]
  We explore the current state of mobile execution of Large Language Models (LLMs) We have created our own automation infrastructure, MELT, which supports the headless execution and benchmarking of LLMs on device. We evaluate popular instruction fine-tuned LLMs and leverage different frameworks to measure their end-to-end and granular performance.
  arXiv  Detail & Related papers  (2024-03-19T15:51:21Z)
• AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration [54.692405042065815]
  We propose Activation-aware Weight Quantization (AWQ), a hardware-friendly approach for LLM low-bit weight-only quantization. AWQ protects only 1% salient weights and achieves excellent quantization performance for instruction-tuned LMs and, for the first time, multi-modal LMs. We also implement TinyChat, an efficient and flexible inference framework tailored for 4-bit on-device LLM/VLMs.
  arXiv  Detail & Related papers  (2023-06-01T17:59:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.