Can Adjusting Hyperparameters Lead to Green Deep Learning: An Empirical Study on Correlations between Hyperparameters and Energy Consumption of Deep Learning Models

• URL: http://arxiv.org/abs/2603.06195v1
• Date: Fri, 06 Mar 2026 12:07:14 GMT
• Title: Can Adjusting Hyperparameters Lead to Green Deep Learning: An Empirical Study on Correlations between Hyperparameters and Energy Consumption of Deep Learning Models
• Abstract summary: This paper focuses on the effect of hyper parameters on the energy cost of Deep learning (DL) models.<n>We train the original and mutated models separately and gather energy information and run-time performance metrics.<n>In a parallel environment, energy consumption becomes more susceptible to change.
• Score: 11.512867515430058
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Context: Along with developing Deep learning (DL) models, larger datasets and more complex model structures are applied, leading to rising computing resources and energy consumption, which is an alert that green DL models should receive more attention. Objective: This paper focuses on a novel view to analyze DL energy consumption: the effect of hyperparameters on the energy cost of DL models. Method: Our approach involves using mutation operators to simulate how practitioners adjust hyperparameters, such as epochs and learning rates. We train the original and mutated models separately and gather energy information and run-time performance metrics. Moreover, we focus on the parallel scenario where multiple DL models are trained in parallel. Results: To examine the effect of hyperparameters on energy consumption, we conducted extensive experiments on five real-world DL models. The results show that (1) many hyperparameters studied have a (positive or negative) correlation with energy consumption, (2) adjusting hyperparameters can make DL models greener, i.e., lead to less energy consumption without performance damage, and (3) in a parallel environment, energy consumption becomes more susceptible to change. Conclusions: We suggest that hyperparameters need more attention in developing DL models, as appropriately adjusting hyperparameters would cause green DL models.

Related papers

• Hyperparameter Learning for Latent Factorization of Tensors for Representation Learning to Large-scale Dynamic Weighted Directed Network [0.0]
  A large-scale dynamic weighted directed networks (DWDNs) are widely used to model time-varying interactions among nodes.<n>Latent factorization of tensors (LFT) extracts target knowledge from DWDNs via low-rank embedding.<n>In practice, these parameters are often tuned manually or through grid search, which requires significant computational resources and human effort.
  arXiv  Detail & Related papers  (2026-06-03T04:17:46Z)
• High-Rank Structured Modulation for Parameter-Efficient Fine-Tuning [57.85676271833619]
  Low-rank Adaptation (LoRA) uses a low-rank update method to simulate full parameter fine-tuning.<n>We present textbfSMoA, a high-rank textbfStructured textbfMOdulation textbfAdapter that uses fewer trainable parameters while maintaining a higher rank.
  arXiv  Detail & Related papers  (2026-01-12T13:06:17Z)
• Deep Learning in Renewable Energy Forecasting: A Cross-Dataset Evaluation of Temporal and Spatial Models [7.286091036139208]
  This study deploys regularization approaches, including early stopping, neuron dropping, and L2 regularization, to reduce the overfitting problem associated with DL models.<n>The LSTM and models show superior performance. Their validation data exhibit exceptionally low root mean square error values.
  arXiv  Detail & Related papers  (2025-05-06T02:05:19Z)
• Green MLOps to Green GenOps: An Empirical Study of Energy Consumption in Discriminative and Generative AI Operations [2.2765705959685234]
  This study investigates the energy consumption of Discriminative and Generative AI models within real-world MLOps pipelines.<n>We employ software-based power measurements to ensure ease of replication across diverse configurations, models, and datasets.
  arXiv  Detail & Related papers  (2025-03-31T10:28:04Z)
• Renewable Energy Prediction: A Comparative Study of Deep Learning Models for Complex Dataset Analysis [6.857122511301731]
  Deep learning (DL) models capture complex, nonlinear relationships in renewable energy datasets.<n>Seven machine learning methods, LSTM, Stacked LSTM, CNN, CNN-LSTM, Time-DistributedDistributed (TD-MLP) and Autoencoder (AE) are evaluated.<n>Regularization techniques such as early stopping, neuron dropout, L1 and L2 regularization are applied to address overfitting.
  arXiv  Detail & Related papers  (2025-01-27T02:10:10Z)
• Optimization Hyper-parameter Laws for Large Language Models [52.49860340549727]
  We present Opt-Laws, a framework that captures the relationship between hyper- parameters and training outcomes.<n>Our validation across diverse model sizes and data scales demonstrates Opt-Laws' ability to accurately predict training loss.<n>This approach significantly reduces computational costs while enhancing overall model performance.
  arXiv  Detail & Related papers  (2024-09-07T09:37:19Z)
• SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models [85.67096251281191]
  We present an innovative approach to model fusion called zero-shot Sparse MIxture of Low-rank Experts (SMILE) construction. SMILE allows for the upscaling of source models into an MoE model without extra data or further training. We conduct extensive experiments across diverse scenarios, such as image classification and text generation tasks, using full fine-tuning and LoRA fine-tuning.
  arXiv  Detail & Related papers  (2024-08-19T17:32:15Z)
• ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflections [59.839926875976225]
  We propose the ETHER transformation family, which performs Efficient fineTuning via HypErplane Reflections. In particular, we introduce ETHER and its relaxation ETHER+, which match or outperform existing PEFT methods with significantly fewer parameters.
  arXiv  Detail & Related papers  (2024-05-30T17:26:02Z)
• Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models [109.06052781040916]
  We introduce a technique to enhance the inference efficiency of parameter-shared language models. We also propose a simple pre-training technique that leads to fully or partially shared models. Results demonstrate the effectiveness of our methods on both autoregressive and autoencoding PLMs.
  arXiv  Detail & Related papers  (2023-10-19T15:13:58Z)
• AutoRL Hyperparameter Landscapes [69.15927869840918]
  Reinforcement Learning (RL) has shown to be capable of producing impressive results, but its use is limited by the impact of its hyperparameters on performance. We propose an approach to build and analyze these hyperparameter landscapes not just for one point in time but at multiple points in time throughout training. This supports the theory that hyperparameters should be dynamically adjusted during training and shows the potential for more insights on AutoRL problems that can be gained through landscape analyses.
  arXiv  Detail & Related papers  (2023-04-05T12:14:41Z)
• A Transistor Operations Model for Deep Learning Energy Consumption Scaling [14.856688747814912]
  Deep Learning (DL) has transformed the automation of a wide range of industries and finds increasing ubiquity in society. The increasing complexity of DL models and its widespread adoption has led to the energy consumption doubling every 3-4 months. Current FLOPs and MACs based methods only consider the linear operations. We develop a bottom-level Transistor Operations (TOs) method to expose the role of activation functions and neural network structure in energy consumption scaling with DL model configuration.
  arXiv  Detail & Related papers  (2022-05-30T12:42:33Z)
• On the Sparsity of Neural Machine Translation Models [65.49762428553345]
  We investigate whether redundant parameters can be reused to achieve better performance. Experiments and analyses are systematically conducted on different datasets and NMT architectures.
  arXiv  Detail & Related papers  (2020-10-06T11:47:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.