Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control

• URL: http://arxiv.org/abs/2403.00172v1
• Date: Thu, 29 Feb 2024 22:42:23 GMT
• Title: Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control
• Authors: Zhiyu An, Xianzhong Ding, Wan Du
• Abstract summary: We overcome the bottleneck by redesigning HVAC controllers using decision trees extracted from thermal dynamics models and historical data. Our method saves 68.4% more energy and increases human comfort gain by 14.8% compared to the state-of-the-art method.
• Score: 3.326392645107372
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent research has shown the potential of Model-based Reinforcement Learning (MBRL) to enhance energy efficiency of Heating, Ventilation, and Air Conditioning (HVAC) systems. However, existing methods rely on black-box thermal dynamics models and stochastic optimizers, lacking reliability guarantees and posing risks to occupant health. In this work, we overcome the reliability bottleneck by redesigning HVAC controllers using decision trees extracted from existing thermal dynamics models and historical data. Our decision tree-based policies are deterministic, verifiable, interpretable, and more energy-efficient than current MBRL methods. First, we introduce a novel verification criterion for RL agents in HVAC control based on domain knowledge. Second, we develop a policy extraction procedure that produces a verifiable decision tree policy. We found that the high dimensionality of the thermal dynamics model input hinders the efficiency of policy extraction. To tackle the dimensionality challenge, we leverage importance sampling conditioned on historical data distributions, significantly improving policy extraction efficiency. Lastly, we present an offline verification algorithm that guarantees the reliability of a control policy. Extensive experiments show that our method saves 68.4% more energy and increases human comfort gain by 14.8% compared to the state-of-the-art method, in addition to an 1127x reduction in computation overhead. Our code and data are available at https://github.com/ryeii/Veri_HVAC

Related papers

• Improving Building Temperature Forecasting: A Data-driven Approach with System Scenario Clustering [3.2114754609864695]
  Heat, Ventilation and Air Conditioning systems cost approximately 40% of primary energy usage in the building sector. For large-scale HVAC system management, it is difficult to construct a detailed model for each subsystem. New data-driven room temperature prediction model is proposed based on the k-means clustering method.
  arXiv  Detail & Related papers  (2024-02-21T09:04:45Z)
• Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks [55.591662978280894]
  This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs) Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees.
  arXiv  Detail & Related papers  (2023-10-13T21:26:16Z)
• Data-driven HVAC Control Using Symbolic Regression: Design and Implementation [0.0]
  This study proposes a design and implementation methodology of data-driven heating, ventilation, and air conditioning () control. Building thermodynamics is modeled using a symbolic regression model (SRM) built from the collected data. The proposed framework reduces the peak power by 16.1% compared to the widely used thermostat controller.
  arXiv  Detail & Related papers  (2023-04-06T13:57:50Z)
• Wasserstein Auto-encoded MDPs: Formal Verification of Efficiently Distilled RL Policies with Many-sided Guarantees [0.0]
  Variational Markov Decision Processes (VAE-MDPs) are discrete latent space models that provide a reliable framework for distilling verifiable controllers from any RL policy. We introduce the Wasserstein auto-encoded MDP (WAE-MDP), a latent space model that fixes those issues by minimizing a penalized form of the optimal transport between the behaviors of the agent executing the original policy and the distilled policy. Our experiments show that, besides distilling policies up to 10 times faster, the latent model quality is indeed better in general.
  arXiv  Detail & Related papers  (2023-03-22T13:41:42Z)
• Data-Driven Stochastic AC-OPF using Gaussian Processes [54.94701604030199]
  Integrating a significant amount of renewables into a power grid is probably the most a way to reduce carbon emissions from power grids slow down climate change. This paper presents an alternative data-driven approach based on the AC power flow equations that can incorporate uncertainty inputs. The GP approach learns a simple yet non-constrained data-driven approach to close this gap to the AC power flow equations.
  arXiv  Detail & Related papers  (2022-07-21T23:02:35Z)
• Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training [71.7750435554693]
  We show that several state-of-the-art RL agents proposed for power system control are vulnerable to adversarial attacks. Specifically, we use an adversary Markov Decision Process to learn an attack policy, and demonstrate the potency of our attack. We propose to use adversarial training to increase the robustness of RL agent against attacks and avoid infeasible operational decisions.
  arXiv  Detail & Related papers  (2021-10-18T00:50:34Z)
• Development of a Soft Actor Critic Deep Reinforcement Learning Approach for Harnessing Energy Flexibility in a Large Office Building [0.0]
  This research is concerned with the novel application and investigation of Soft Actor Critic' (SAC) based Deep Reinforcement Learning (DRL) SAC is a model-free DRL technique that is able to handle continuous action spaces.
  arXiv  Detail & Related papers  (2021-04-25T10:33:35Z)
• DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning [25.710523867709664]
  We develop a new data-driven AI system, namely DeepThermal, to optimize the combustion control strategy for thermal power generating units. At its core is a new model-based offline reinforcement learning framework, called MORE. More aims at simultaneously improving the long-term reward (increase combustion efficiency and reduce pollutant emission) and controlling operational risks (safety constraints satisfaction)
  arXiv  Detail & Related papers  (2021-02-23T04:55:12Z)
• SS-SFDA : Self-Supervised Source-Free Domain Adaptation for Road Segmentation in Hazardous Environments [54.22535063244038]
  We present a novel approach for unsupervised road segmentation in adverse weather conditions such as rain or fog. This includes a new algorithm for source-free domain adaptation (SFDA) using self-supervised learning. We have evaluated the performance on $6$ datasets corresponding to real and synthetic adverse weather conditions.
  arXiv  Detail & Related papers  (2020-11-27T09:19:03Z)
• Controlling Rayleigh-B\'enard convection via Reinforcement Learning [62.997667081978825]
  The identification of effective control strategies to suppress or enhance the convective heat exchange under fixed external thermal gradients is an outstanding fundamental and technological issue. In this work, we explore a novel approach, based on a state-of-the-art Reinforcement Learning (RL) algorithm. We show that our RL-based control is able to stabilize the conductive regime and bring the onset of convection up to a Rayleigh number.
  arXiv  Detail & Related papers  (2020-03-31T16:39:25Z)
• NeurOpt: Neural network based optimization for building energy management and climate control [58.06411999767069]
  We propose a data-driven control algorithm based on neural networks to reduce this cost of model identification. We validate our learning and control algorithms on a two-story building with ten independently controlled zones, located in Italy.
  arXiv  Detail & Related papers  (2020-01-22T00:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.