Laxity-Aware Scalable Reinforcement Learning for HVAC Control

• URL: http://arxiv.org/abs/2306.16619v1
• Date: Thu, 29 Jun 2023 01:28:14 GMT
• Title: Laxity-Aware Scalable Reinforcement Learning for HVAC Control
• Authors: Ruohong Liu, Yuxin Pan, Yize Chen
• Abstract summary: We tackle the curse of dimensionality issue in modeling and control by utilizing the concept of laxity to quantify the emergency level of each operation request. We propose a two-level approach to address energy optimization for a large population of HVAC systems.
• Score: 2.0625936401496237
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Demand flexibility plays a vital role in maintaining grid balance, reducing peak demand, and saving customers' energy bills. Given their highly shiftable load and significant contribution to a building's energy consumption, Heating, Ventilation, and Air Conditioning (HVAC) systems can provide valuable demand flexibility to the power systems by adjusting their energy consumption in response to electricity price and power system needs. To exploit this flexibility in both operation time and power, it is imperative to accurately model and aggregate the load flexibility of a large population of HVAC systems as well as designing effective control algorithms. In this paper, we tackle the curse of dimensionality issue in modeling and control by utilizing the concept of laxity to quantify the emergency level of each HVAC operation request. We further propose a two-level approach to address energy optimization for a large population of HVAC systems. The lower level involves an aggregator to aggregate HVAC load laxity information and use least-laxity-first (LLF) rule to allocate real-time power for individual HVAC systems based on the controller's total power. Due to the complex and uncertain nature of HVAC systems, we leverage a reinforcement learning (RL)-based controller to schedule the total power based on the aggregated laxity information and electricity price. We evaluate the temperature control and energy cost saving performance of a large-scale group of HVAC systems in both single-zone and multi-zone scenarios, under varying climate and electricity market conditions. The experiment results indicate that proposed approach outperforms the centralized methods in the majority of test scenarios, and performs comparably to model-based method in some scenarios.

Related papers

• An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control [40.71019623757305]
  Recent studies have shown that Deep Reinforcement Learning (DRL) algorithms can outperform traditional reactive controllers. This paper provides a critical and reproducible evaluation of several state-of-the-art DRL algorithms for HVAC control.
  arXiv  Detail & Related papers  (2024-01-11T08:40:26Z)
• Global Transformer Architecture for Indoor Room Temperature Forecasting [49.32130498861987]
  This work presents a global Transformer architecture for indoor temperature forecasting in multi-room buildings. It aims at optimizing energy consumption and reducing greenhouse gas emissions associated with HVAC systems. Notably, this study is the first to apply a Transformer architecture for indoor temperature forecasting in multi-room buildings.
  arXiv  Detail & Related papers  (2023-10-31T14:09:32Z)
• A Comparison of Classical and Deep Reinforcement Learning Methods for HVAC Control [0.0]
  We benchmark two popular classical and deep RL methods (Q-Learning and Deep-Q-Networks) across multiple HVAC environments. The findings provide insight for configuring RL agents in HVAC systems, promoting energy-efficient and cost-effective operation.
  arXiv  Detail & Related papers  (2023-08-10T17:20:02Z)
• Data-driven HVAC Control Using Symbolic Regression: Design and Implementation [0.0]
  This study proposes a design and implementation methodology of data-driven heating, ventilation, and air conditioning () control. Building thermodynamics is modeled using a symbolic regression model (SRM) built from the collected data. The proposed framework reduces the peak power by 16.1% compared to the widely used thermostat controller.
  arXiv  Detail & Related papers  (2023-04-06T13:57:50Z)
• Data-Driven Chance Constrained AC-OPF using Hybrid Sparse Gaussian Processes [57.70237375696411]
  The paper proposes a fast data-driven setup that uses the sparse and hybrid Gaussian processes (GP) framework to model the power flow equations with input uncertainty. We advocate the efficiency of the proposed approach by a numerical study over multiple IEEE test cases showing up to two times faster and more accurate solutions.
  arXiv  Detail & Related papers  (2022-08-30T09:27:59Z)
• Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer [128.19212716007794]
  We propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency.
  arXiv  Detail & Related papers  (2022-06-08T07:48:42Z)
• Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings [38.326533494236266]
  In commercial buildings, about 40%-50% of the total electricity is attributed to Heating, Ventilation, and Air Conditioning systems. In this paper, we intend to minimize the energy cost of which an HVAC system in a multi-zone building under dynamic pricing with occupancy of zone, thermal comfort, and indoor air quality comfort.
  arXiv  Detail & Related papers  (2020-06-25T03:41:42Z)
• Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
  We study a risk-aware energy scheduling problem for a microgrid-powered MEC network. We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
  arXiv  Detail & Related papers  (2020-02-21T02:14:38Z)
• Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems [87.4519172058185]
  An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
  arXiv  Detail & Related papers  (2020-02-20T04:58:07Z)
• NeurOpt: Neural network based optimization for building energy management and climate control [58.06411999767069]
  We propose a data-driven control algorithm based on neural networks to reduce this cost of model identification. We validate our learning and control algorithms on a two-story building with ten independently controlled zones, located in Italy.
  arXiv  Detail & Related papers  (2020-01-22T00:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.