ReeM: Ensemble Building Thermodynamics Model for Efficient HVAC Control via Hierarchical Reinforcement Learning

• URL: http://arxiv.org/abs/2505.02439v1
• Date: Mon, 05 May 2025 08:09:36 GMT
• Title: ReeM: Ensemble Building Thermodynamics Model for Efficient HVAC Control via Hierarchical Reinforcement Learning
• Authors: Yang Deng, Yaohui Liu, Rui Liang, Dafang Zhao, Donghua Xie, Ittetsu Taniguchi, Dan Wang,
• Abstract summary: Building thermodynamics models predict real-time indoor temperature changes under potential HVAC control operations.<n>These models often require extensive data collection periods and rely heavily on expert knowledge, making the modeling process inefficient and limiting the reusability of the models.<n>This paper explores a model ensemble perspective that utilizes existing developed models as base models to serve a target building environment.
• Score: 8.266185862232225
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The building thermodynamics model, which predicts real-time indoor temperature changes under potential HVAC (Heating, Ventilation, and Air Conditioning) control operations, is crucial for optimizing HVAC control in buildings. While pioneering studies have attempted to develop such models for various building environments, these models often require extensive data collection periods and rely heavily on expert knowledge, making the modeling process inefficient and limiting the reusability of the models. This paper explores a model ensemble perspective that utilizes existing developed models as base models to serve a target building environment, thereby providing accurate predictions while reducing the associated efforts. Given that building data streams are non-stationary and the number of base models may increase, we propose a Hierarchical Reinforcement Learning (HRL) approach to dynamically select and weight the base models. Our approach employs a two-tiered decision-making process: the high-level focuses on model selection, while the low-level determines the weights of the selected models. We thoroughly evaluate the proposed approach through offline experiments and an on-site case study, and the experimental results demonstrate the effectiveness of our method.

Related papers

• Integrating Physics-Based and Data-Driven Approaches for Probabilistic Building Energy Modeling [5.437298646956505]
  Building energy modeling is a key tool for optimizing the performance of building energy systems.<n>Recently, hybrid approaches that combine the strengths of both paradigms have gained attention.
  arXiv  Detail & Related papers  (2025-07-23T14:07:33Z)
• On conditional diffusion models for PDE simulations [53.01911265639582]
  We study score-based diffusion models for forecasting and assimilation of sparse observations. We propose an autoregressive sampling approach that significantly improves performance in forecasting. We also propose a new training strategy for conditional score-based models that achieves stable performance over a range of history lengths.
  arXiv  Detail & Related papers  (2024-10-21T18:31:04Z)
• Improving Building Temperature Forecasting: A Data-driven Approach with System Scenario Clustering [3.2114754609864695]
  Heat, Ventilation and Air Conditioning systems cost approximately 40% of primary energy usage in the building sector. For large-scale HVAC system management, it is difficult to construct a detailed model for each subsystem. New data-driven room temperature prediction model is proposed based on the k-means clustering method.
  arXiv  Detail & Related papers  (2024-02-21T09:04:45Z)
• Deep encoder-decoder hierarchical convolutional neural networks for conjugate heat transfer surrogate modeling [0.0]
  Conjugate heat transfer (CHT) analyses are vital for the design of many energy systems.<n>High-fidelity CHT numerical simulations are computationally intensive.<n>We develop a modular deep encoder-decoder hierarchical (DeepEDH) convolutional neural network for CHT analyses.
  arXiv  Detail & Related papers  (2023-11-24T21:45:11Z)
• Data-driven HVAC Control Using Symbolic Regression: Design and Implementation [0.0]
  This study proposes a design and implementation methodology of data-driven heating, ventilation, and air conditioning () control. Building thermodynamics is modeled using a symbolic regression model (SRM) built from the collected data. The proposed framework reduces the peak power by 16.1% compared to the widely used thermostat controller.
  arXiv  Detail & Related papers  (2023-04-06T13:57:50Z)
• Towards Efficient Task-Driven Model Reprogramming with Foundation Models [52.411508216448716]
  Vision foundation models exhibit impressive power, benefiting from the extremely large model capacity and broad training data. However, in practice, downstream scenarios may only support a small model due to the limited computational resources or efficiency considerations. This brings a critical challenge for the real-world application of foundation models: one has to transfer the knowledge of a foundation model to the downstream task.
  arXiv  Detail & Related papers  (2023-04-05T07:28:33Z)
• Benchmarking Model Predictive Control Algorithms in Building Optimization Testing Framework (BOPTEST) [40.17692290400862]
  We present a data-driven modeling and control framework for physics-based building emulators. Our approach consists of: (a) Offline training of differentiable surrogate models that accelerate model evaluations, provide cost-effective gradients, and maintain good predictive accuracy for the receding horizon in Model Predictive Control (MPC) We extensively evaluate the modeling and control performance using multiple surrogate models and optimization frameworks across various test cases available in the Building Optimization Testing Framework (BOPTEST)
  arXiv  Detail & Related papers  (2023-01-31T06:55:19Z)
• Minimal Value-Equivalent Partial Models for Scalable and Robust Planning in Lifelong Reinforcement Learning [56.50123642237106]
  Common practice in model-based reinforcement learning is to learn models that model every aspect of the agent's environment. We argue that such models are not particularly well-suited for performing scalable and robust planning in lifelong reinforcement learning scenarios. We propose new kinds of models that only model the relevant aspects of the environment, which we call "minimal value-minimal partial models"
  arXiv  Detail & Related papers  (2023-01-24T16:40:01Z)
• When to Update Your Model: Constrained Model-based Reinforcement Learning [50.74369835934703]
  We propose a novel and general theoretical scheme for a non-decreasing performance guarantee of model-based RL (MBRL) Our follow-up derived bounds reveal the relationship between model shifts and performance improvement. A further example demonstrates that learning models from a dynamically-varying number of explorations benefit the eventual returns.
  arXiv  Detail & Related papers  (2022-10-15T17:57:43Z)
• Semi-analytical Industrial Cooling System Model for Reinforcement Learning [4.272330410469061]
  We present a hybrid industrial cooling system model that embeds analytical solutions within a multi-physics simulation. The model's fidelity is evaluated against real world data from a large scale cooling system.
  arXiv  Detail & Related papers  (2022-07-26T18:19:17Z)
• Your Autoregressive Generative Model Can be Better If You Treat It as an Energy-Based One [83.5162421521224]
  We propose a unique method termed E-ARM for training autoregressive generative models. E-ARM takes advantage of a well-designed energy-based learning objective. We show that E-ARM can be trained efficiently and is capable of alleviating the exposure bias problem.
  arXiv  Detail & Related papers  (2022-06-26T10:58:41Z)
• Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration [130.89746032163106]
  We propose ALOE, a new algorithm for learning conditional and unconditional EBMs for discrete structured data. We show that the energy function and sampler can be trained efficiently via a new variational form of power iteration. We present an energy model guided fuzzer for software testing that achieves comparable performance to well engineered fuzzing engines like libfuzzer.
  arXiv  Detail & Related papers  (2020-11-10T19:31:29Z)
• Generative Modeling for Atmospheric Convection [13.104272504735052]
  We explore the potential of generative modeling to cheaply recreate small-scale storms by designing and implementing a Variational Autoencoder (VAE) VAE performs structural replication, dimensionality reduction, and clustering of high-resolution vertical velocity fields on 6*106 samples spanning the globe. It successfully reconstructs the spatial structure of convection, performs unsupervised clustering of convective organization regimes, and identifies anomalous storm activity.
  arXiv  Detail & Related papers  (2020-07-03T00:24:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.