From RNNs to Foundation Models: An Empirical Study on Commercial Building Energy Consumption

• URL: http://arxiv.org/abs/2411.14421v2
• Date: Tue, 26 Nov 2024 14:55:52 GMT
• Title: From RNNs to Foundation Models: An Empirical Study on Commercial Building Energy Consumption
• Authors: Shourya Bose, Yijiang Li, Amy Van Sant, Yu Zhang, Kibaek Kim,
• Abstract summary: Short-term energy consumption forecasting for commercial buildings is crucial for smart grid operations. While smart meters and deep learning models enable forecasting using past data from multiple buildings, data heterogeneity from diverse buildings can reduce model performance. We tackle this issue using the ComStock dataset, which provides synthetic energy consumption data for U.S. commercial buildings.
• Score: 3.355907772736553
• License:
• Abstract: Accurate short-term energy consumption forecasting for commercial buildings is crucial for smart grid operations. While smart meters and deep learning models enable forecasting using past data from multiple buildings, data heterogeneity from diverse buildings can reduce model performance. The impact of increasing dataset heterogeneity in time series forecasting, while keeping size and model constant, is understudied. We tackle this issue using the ComStock dataset, which provides synthetic energy consumption data for U.S. commercial buildings. Two curated subsets, identical in size and region but differing in building type diversity, are used to assess the performance of various time series forecasting models, including fine-tuned open-source foundation models (FMs). The results show that dataset heterogeneity and model architecture have a greater impact on post-training forecasting performance than the parameter count. Moreover, despite the higher computational cost, fine-tuned FMs demonstrate competitive performance compared to base models trained from scratch.

Related papers

• Tackling Data Heterogeneity in Federated Time Series Forecasting [61.021413959988216]
  Time series forecasting plays a critical role in various real-world applications, including energy consumption prediction, disease transmission monitoring, and weather forecasting. Most existing methods rely on a centralized training paradigm, where large amounts of data are collected from distributed devices to a central cloud server. We propose a novel framework, Fed-TREND, to address data heterogeneity by generating informative synthetic data as auxiliary knowledge carriers.
  arXiv  Detail & Related papers  (2024-11-24T04:56:45Z)
• Exploring the design space of deep-learning-based weather forecasting systems [56.129148006412855]
  This paper systematically analyzes the impact of different design choices on deep-learning-based weather forecasting systems. We study fixed-grid architectures such as UNet, fully convolutional architectures, and transformer-based models. We propose a hybrid system that combines the strong performance of fixed-grid models with the flexibility of grid-invariant architectures.
  arXiv  Detail & Related papers  (2024-10-09T22:25:50Z)
• Creating synthetic energy meter data using conditional diffusion and building metadata [0.0]
  The study proposes a conditional diffusion model for generating high-quality synthetic energy data using relevant metadata. Using a dataset comprising 1,828 power meters from various buildings and countries, this model is compared with traditional methods. Results demonstrate the proposed diffusion model's superior performance, with a 36% reduction in Frechet Inception Distance (FID) score and a 13% decrease in Kullback-Leibler divergence (KL divergence)
  arXiv  Detail & Related papers  (2024-03-31T01:58:38Z)
• Impact of data for forecasting on performance of model predictive control in buildings with smart energy storage [0.0]
  The impact on forecast accuracy of measures to improve model data efficiency are quantified. The use of more than 2 years of training data for load prediction models provided no significant improvement in forecast accuracy. Reused models and those trained with 3 months of data had on average 10% higher error than baseline, indicating that deploying MPC systems without prior data collection may be economic.
  arXiv  Detail & Related papers  (2024-02-19T21:01:11Z)
• Grid Frequency Forecasting in University Campuses using Convolutional LSTM [0.0]
  This paper harnesses Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks to establish robust time forecasting models for grid frequency. Individual ConvLSTM models are trained on power consumption data for each campus building and forecast the grid frequency based on historical trends. An Ensemble Model is formulated to aggregate insights from the building-specific models, delivering comprehensive forecasts for the entire campus.
  arXiv  Detail & Related papers  (2023-10-24T13:53:51Z)
• DECODE: Data-driven Energy Consumption Prediction leveraging Historical Data and Environmental Factors in Buildings [1.2891210250935148]
  This paper introduces a Long Short-Term Memory (LSTM) model designed to forecast building energy consumption. The LSTM model provides accurate short, medium, and long-term energy predictions for residential and commercial buildings. It demonstrates exceptional prediction accuracy, boasting the highest R2 score of 0.97 and the most favorable mean absolute error (MAE) of 0.007.
  arXiv  Detail & Related papers  (2023-09-06T11:02:53Z)
• Benchmarks and Custom Package for Energy Forecasting [55.460452605056894]
  Energy forecasting aims to minimize the cost of subsequent tasks such as power grid dispatch. In this paper, we collected large-scale load datasets and released a new renewable energy dataset. We conducted extensive experiments with 21 forecasting methods in these energy datasets at different levels under 11 evaluation metrics.
  arXiv  Detail & Related papers  (2023-07-14T06:50:02Z)
• DA-VEGAN: Differentiably Augmenting VAE-GAN for microstructure reconstruction from extremely small data sets [110.60233593474796]
  DA-VEGAN is a model with two central innovations. A $beta$-variational autoencoder is incorporated into a hybrid GAN architecture. A custom differentiable data augmentation scheme is developed specifically for this architecture.
  arXiv  Detail & Related papers  (2023-02-17T08:49:09Z)
• A Comparative Study on Energy Consumption Models for Drones [4.660172505713055]
  We benchmark the five most popular energy consumption models for drones derived from their physical behaviours. We propose a novel data-driven energy model using the Long Short-Term Memory (LSTM) based deep learning architecture. Our experimental results have shown that the LSTM based approach can easily outperform other mathematical models for the dataset under study.
  arXiv  Detail & Related papers  (2022-05-30T23:05:32Z)
• Back2Future: Leveraging Backfill Dynamics for Improving Real-time Predictions in Future [73.03458424369657]
  In real-time forecasting in public health, data collection is a non-trivial and demanding task. 'Backfill' phenomenon and its effect on model performance has been barely studied in the prior literature. We formulate a novel problem and neural framework Back2Future that aims to refine a given model's predictions in real-time.
  arXiv  Detail & Related papers  (2021-06-08T14:48:20Z)
• Comparing Test Sets with Item Response Theory [53.755064720563]
  We evaluate 29 datasets using predictions from 18 pretrained Transformer models on individual test examples. We find that Quoref, HellaSwag, and MC-TACO are best suited for distinguishing among state-of-the-art models. We also observe span selection task format, which is used for QA datasets like QAMR or SQuAD2.0, is effective in differentiating between strong and weak models.
  arXiv  Detail & Related papers  (2021-06-01T22:33:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.