Reinforcement Learning (RL) Meets Urban Climate Modeling: Investigating the Efficacy and Impacts of RL-Based HVAC Control

• URL: http://arxiv.org/abs/2505.07045v1
• Date: Sun, 11 May 2025 16:33:42 GMT
• Title: Reinforcement Learning (RL) Meets Urban Climate Modeling: Investigating the Efficacy and Impacts of RL-Based HVAC Control
• Authors: Junjie Yu, John S. Schreck, David John Gagne, Keith W. Oleson, Jie Li, Yongtu Liang, Qi Liao, Mingfei Sun, David O. Topping, Zhonghua Zheng,
• Abstract summary: Reinforcement learning (RL)-based heating, ventilation, and air conditioning control has emerged as a promising technology.<n>This study proposes an integrated framework combining RL with an urban climate model that incorporates a building energy model.
• Score: 6.144467371827833
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning (RL)-based heating, ventilation, and air conditioning (HVAC) control has emerged as a promising technology for reducing building energy consumption while maintaining indoor thermal comfort. However, the efficacy of such strategies is influenced by the background climate and their implementation may potentially alter both the indoor climate and local urban climate. This study proposes an integrated framework combining RL with an urban climate model that incorporates a building energy model, aiming to evaluate the efficacy of RL-based HVAC control across different background climates, impacts of RL strategies on indoor climate and local urban climate, and the transferability of RL strategies across cities. Our findings reveal that the reward (defined as a weighted combination of energy consumption and thermal comfort) and the impacts of RL strategies on indoor climate and local urban climate exhibit marked variability across cities with different background climates. The sensitivity of reward weights and the transferability of RL strategies are also strongly influenced by the background climate. Cities in hot climates tend to achieve higher rewards across most reward weight configurations that balance energy consumption and thermal comfort, and those cities with more varying atmospheric temperatures demonstrate greater RL strategy transferability. These findings underscore the importance of thoroughly evaluating RL-based HVAC control strategies in diverse climatic contexts. This study also provides a new insight that city-to-city learning will potentially aid the deployment of RL-based HVAC control.

Related papers

• HVAC-DPT: A Decision Pretrained Transformer for HVAC Control [0.0]
  Building operations consume approximately 40% of global energy, with Heating, Ventilation, and Air Conditioning systems responsible for up to 50%.<n>Existing control strategies lack generalisation and require extensive training and data.<n>This paper introduces HVAC-DPT, a Decision-Pretrained Transformer using in-context Reinforcement Learning.
  arXiv  Detail & Related papers  (2024-11-29T14:46:37Z)
• Tackling extreme urban heat: a machine learning approach to assess the impacts of climate change and the efficacy of climate adaptation strategies in urban microclimates [0.0]
  High temperatures concentrated in urban heat can drive increased risk of heat-related death and illness. We present open-source, computationally efficient machine learning methods that can improve the accuracy of urban temperature estimates. We find that cooling demand is likely to increase substantially through midcentury, but engineered high-albedo surfaces could lessen this increase by more than 50%.
  arXiv  Detail & Related papers  (2024-11-08T20:29:11Z)
• Efficient Localized Adaptation of Neural Weather Forecasting: A Case Study in the MENA Region [62.09891513612252]
  We focus on limited-area modeling and train our model specifically for localized region-level downstream tasks. We consider the MENA region due to its unique climatic challenges, where accurate localized weather forecasting is crucial for managing water resources, agriculture and mitigating the impacts of extreme weather events. Our study aims to validate the effectiveness of integrating parameter-efficient fine-tuning (PEFT) methodologies, specifically Low-Rank Adaptation (LoRA) and its variants, to enhance forecast accuracy, as well as training speed, computational resource utilization, and memory efficiency in weather and climate modeling for specific regions.
  arXiv  Detail & Related papers  (2024-09-11T19:31:56Z)
• RAIN: Reinforcement Algorithms for Improving Numerical Weather and Climate Models [0.0]
  Current climate models rely on complex mathematical parameterisations to represent sub-grid scale processes.<n>This study explores integrating reinforcement learning with idealised climate models to address key parameterisation challenges.
  arXiv  Detail & Related papers  (2024-08-28T20:10:46Z)
• Employing Federated Learning for Training Autonomous HVAC Systems [3.4137115855910767]
  Buildings account for 40% of global energy consumption. implementing smart, energy-efficient HVAC systems has the potential to significantly impact the course of climate change.<n>In recent years, model-free reinforcement learning algorithms have been increasingly assessed for this purpose due to their ability to learn and adapt purely from experience.<n>We show that using federated learning to train the reinforcement learning controller of HVAC systems can improve the learning speed, as well as improve their ability to generalize.
  arXiv  Detail & Related papers  (2024-05-01T08:42:22Z)
• An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control [40.71019623757305]
  Recent studies have shown that Deep Reinforcement Learning (DRL) algorithms can outperform traditional reactive controllers. This paper provides a critical and reproducible evaluation of several state-of-the-art DRL algorithms for HVAC control.
  arXiv  Detail & Related papers  (2024-01-11T08:40:26Z)
• Optimizing Heat Alert Issuance with Reinforcement Learning [2.7262923206583136]
  Key strategy in societal adaptation to climate change is using alert systems to prompt preventative action and reduce the adverse health impacts of extreme heat events.<n>This paper implements and evaluates reinforcement learning as a tool to optimize the effectiveness of such systems.<n>First, we introduce a new publicly available RL environment enabling the evaluation of the effectiveness of heat alert policies to reduce heat-related hospitalizations.<n>Second, we use this environment to evaluate standard RL algorithms in the context of heat alert issuance.<n>Third, a post-hoc contrastive analysis provides insight into scenarios where our modified heat alert-RL policies yield significant gains/losses over the current
  arXiv  Detail & Related papers  (2023-12-21T00:50:21Z)
• Climate-sensitive Urban Planning through Optimization of Tree Placements [55.11389516857789]
  Climate change is increasing the intensity and frequency of many extreme weather events, including heatwaves. Among the most promising strategies is harnessing the benefits of urban trees in shading and cooling pedestrian-level environments. Physical simulations can estimate the radiative and thermal impact of trees on human thermal comfort but induce high computational costs. We employ neural networks to simulate the point-wise mean radiant temperatures--a driving factor of outdoor human thermal comfort--across various time scales.
  arXiv  Detail & Related papers  (2023-10-09T13:07:23Z)
• Low Emission Building Control with Zero-Shot Reinforcement Learning [70.70479436076238]
  Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency. We show it is possible to obtain emission-reducing policies without a priori--a paradigm we call zero-shot building control.
  arXiv  Detail & Related papers  (2022-08-12T17:13:25Z)
• Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training [71.7750435554693]
  We show that several state-of-the-art RL agents proposed for power system control are vulnerable to adversarial attacks. Specifically, we use an adversary Markov Decision Process to learn an attack policy, and demonstrate the potency of our attack. We propose to use adversarial training to increase the robustness of RL agent against attacks and avoid infeasible operational decisions.
  arXiv  Detail & Related papers  (2021-10-18T00:50:34Z)
• Data-driven control of micro-climate in buildings: an event-triggered reinforcement learning approach [56.22460188003505]
  We formulate the micro-climate control problem based on semi-Markov decision processes. We propose two learning algorithms for event-triggered control of micro-climate in buildings. We show the efficacy of our proposed approach via designing a smart learning thermostat.
  arXiv  Detail & Related papers  (2020-01-28T18:20:43Z)
• NeurOpt: Neural network based optimization for building energy management and climate control [58.06411999767069]
  We propose a data-driven control algorithm based on neural networks to reduce this cost of model identification. We validate our learning and control algorithms on a two-story building with ten independently controlled zones, located in Italy.
  arXiv  Detail & Related papers  (2020-01-22T00:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.