IrEne: Interpretable Energy Prediction for Transformers

• URL: http://arxiv.org/abs/2106.01199v1
• Date: Wed, 2 Jun 2021 14:43:51 GMT
• Title: IrEne: Interpretable Energy Prediction for Transformers
• Authors: Qingqing Cao, Yash Kumar Lal, Harsh Trivedi, Aruna Balasubramanian, Niranjan Balasubramanian
• Abstract summary: Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. We present IrEne, an interpretable and energy prediction system that accurately predicts the inference energy consumption of a wide range of Transformer-based NLP models.
• Score: 15.677294441315535
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. We present IrEne, an interpretable and extensible energy prediction system that accurately predicts the inference energy consumption of a wide range of Transformer-based NLP models. IrEne constructs a model tree graph that breaks down the NLP model into modules that are further broken down into low-level machine learning (ML) primitives. IrEne predicts the inference energy consumption of the ML primitives as a function of generalizable features and fine-grained runtime resource usage. IrEne then aggregates these low-level predictions recursively to predict the energy of each module and finally of the entire model. Experiments across multiple Transformer models show IrEne predicts inference energy consumption of transformer models with an error of under 7% compared to the ground truth. In contrast, existing energy models see an error of over 50%. We also show how IrEne can be used to conduct energy bottleneck analysis and to easily evaluate the energy impact of different architectural choices. We release the code and data at https://github.com/StonyBrookNLP/irene.

Related papers

• Forecasting Auxiliary Energy Consumption for Electric Heavy-Duty Vehicles [6.375656754994484]
  Energy consumption prediction is crucial for optimizing the operation of electric commercial heavy-duty vehicles. In this paper, we demonstrate a potential solution by training multiple regression models on subsets of data. Experiments on both synthetic and real-world datasets show that such splitting of a complex problem into simpler ones yields better regression performance and interpretability.
  arXiv  Detail & Related papers  (2023-11-27T16:52:25Z)
• FedWOA: A Federated Learning Model that uses the Whale Optimization Algorithm for Renewable Energy Prediction [0.0]
  This paper introduces FedWOA, a novel federated learning model that aggregate global prediction models from the weights of local neural network models trained on prosumer energy data. The evaluation results on prosumers energy data have shown that FedWOA can effectively enhance the accuracy of energy prediction models accuracy by 25% for MSE and 16% for MAE compared to FedAVG.
  arXiv  Detail & Related papers  (2023-09-19T05:44:18Z)
• Predictable MDP Abstraction for Unsupervised Model-Based RL [93.91375268580806]
  We propose predictable MDP abstraction (PMA) Instead of training a predictive model on the original MDP, we train a model on a transformed MDP with a learned action space. We theoretically analyze PMA and empirically demonstrate that PMA leads to significant improvements over prior unsupervised model-based RL approaches.
  arXiv  Detail & Related papers  (2023-02-08T07:37:51Z)
• EVE: Environmental Adaptive Neural Network Models for Low-power Energy Harvesting System [8.16411986220709]
  Energy harvesting technology that harvests energy from ambient environment is a promising alternative to batteries for powering those devices. This paper proposes EVE, an automated machine learning framework to search for desired multi-models with shared weights for energy harvesting IoT devices. Experimental results show that the neural networks models generated by EVE is on average 2.5X faster than the baseline models without pruning and shared weights.
  arXiv  Detail & Related papers  (2022-07-14T20:53:46Z)
• A Comparative Study on Energy Consumption Models for Drones [4.660172505713055]
  We benchmark the five most popular energy consumption models for drones derived from their physical behaviours. We propose a novel data-driven energy model using the Long Short-Term Memory (LSTM) based deep learning architecture. Our experimental results have shown that the LSTM based approach can easily outperform other mathematical models for the dataset under study.
  arXiv  Detail & Related papers  (2022-05-30T23:05:32Z)
• Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction [51.80191416661064]
  We propose a novel vision transformer with latent variables following an informative energy-based prior for salient object detection. Both the vision transformer network and the energy-based prior model are jointly trained via Markov chain Monte Carlo-based maximum likelihood estimation. With the generative vision transformer, we can easily obtain a pixel-wise uncertainty map from an image, which indicates the model confidence in predicting saliency from the image.
  arXiv  Detail & Related papers  (2021-12-27T06:04:33Z)
• Prediction of liquid fuel properties using machine learning models with Gaussian processes and probabilistic conditional generative learning [56.67751936864119]
  The present work aims to construct cheap-to-compute machine learning (ML) models to act as closure equations for predicting the physical properties of alternative fuels. Those models can be trained using the database from MD simulations and/or experimental measurements in a data-fusion-fidelity approach. The results show that ML models can predict accurately the fuel properties of a wide range of pressure and temperature conditions.
  arXiv  Detail & Related papers  (2021-10-18T14:43:50Z)
• MoEfication: Conditional Computation of Transformer Models for Efficient Inference [66.56994436947441]
  Transformer-based pre-trained language models can achieve superior performance on most NLP tasks due to large parameter capacity, but also lead to huge computation cost. We explore to accelerate large-model inference by conditional computation based on the sparse activation phenomenon. We propose to transform a large model into its mixture-of-experts (MoE) version with equal model size, namely MoEfication.
  arXiv  Detail & Related papers  (2021-10-05T02:14:38Z)
• Efficient pre-training objectives for Transformers [84.64393460397471]
  We study several efficient pre-training objectives for Transformers-based models. We prove that eliminating the MASK token and considering the whole output during the loss are essential choices to improve performance.
  arXiv  Detail & Related papers  (2021-04-20T00:09:37Z)
• Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration [130.89746032163106]
  We propose ALOE, a new algorithm for learning conditional and unconditional EBMs for discrete structured data. We show that the energy function and sampler can be trained efficiently via a new variational form of power iteration. We present an energy model guided fuzzer for software testing that achieves comparable performance to well engineered fuzzing engines like libfuzzer.
  arXiv  Detail & Related papers  (2020-11-10T19:31:29Z)
• Energy Predictive Models for Convolutional Neural Networks on Mobile Platforms [0.0]
  Energy use is a key concern when deploying deep learning models on mobile devices. We build layer-type predictive models for the fully-connected and pooling layers using 12 representative Convolutional NeuralNetworks (ConvNets) on the Jetson TX1 and the Snapdragon 820. We obtain an accuracy between 76% to 85% and a model complexity of 1 for the overall energy prediction of the test ConvNets across different hardware-software combinations.
  arXiv  Detail & Related papers  (2020-04-10T17:35:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.