Data-driven control of micro-climate in buildings: an event-triggered reinforcement learning approach

• URL: http://arxiv.org/abs/2001.10505v2
• Date: Thu, 2 Jul 2020 19:11:14 GMT
• Title: Data-driven control of micro-climate in buildings: an event-triggered reinforcement learning approach
• Authors: Ashkan Haji Hosseinloo, Alexander Ryzhov, Aldo Bischi, Henni Ouerdane, Konstantin Turitsyn, Munther A. Dahleh
• Abstract summary: We formulate the micro-climate control problem based on semi-Markov decision processes. We propose two learning algorithms for event-triggered control of micro-climate in buildings. We show the efficacy of our proposed approach via designing a smart learning thermostat.
• Score: 56.22460188003505
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Smart buildings have great potential for shaping an energy-efficient, sustainable, and more economic future for our planet as buildings account for approximately 40% of the global energy consumption. Future of the smart buildings lies in using sensory data for adaptive decision making and control that is currently gloomed by the key challenge of learning a good control policy in a short period of time in an online and continuing fashion. To tackle this challenge, an event-triggered -- as opposed to classic time-triggered -- paradigm, is proposed in which learning and control decisions are made when events occur and enough information is collected. Events are characterized by certain design conditions and they occur when the conditions are met, for instance, when a certain state threshold is reached. By systematically adjusting the time of learning and control decisions, the proposed framework can potentially reduce the variance in learning, and consequently, improve the control process. We formulate the micro-climate control problem based on semi-Markov decision processes that allow for variable-time state transitions and decision making. Using extended policy gradient theorems and temporal difference methods in a reinforcement learning set-up, we propose two learning algorithms for event-triggered control of micro-climate in buildings. We show the efficacy of our proposed approach via designing a smart learning thermostat that simultaneously optimizes energy consumption and occupants' comfort in a test building.

Related papers

• Pausing Policy Learning in Non-stationary Reinforcement Learning [23.147618992106867]
  We tackle a common belief that continually updating the decision is optimal to minimize the temporal gap. We propose forecasting an online reinforcement learning framework and show that strategically pausing decision updates yields better overall performance.
  arXiv  Detail & Related papers  (2024-05-25T04:38:09Z)
• Room Occupancy Prediction: Exploring the Power of Machine Learning and Temporal Insights [0.0]
  Energy conservation in buildings is a paramount concern to combat greenhouse gas emissions and combat climate change. In this study, we present a predictive framework for room occupancy that leverages a diverse set of machine learning models. We highlight the promise of machine learning in shaping energy-efficient practices and room occupancy management.
  arXiv  Detail & Related papers  (2023-12-22T04:16:34Z)
• A Stochastic Online Forecast-and-Optimize Framework for Real-Time Energy Dispatch in Virtual Power Plants under Uncertainty [18.485617498705736]
  We propose a real-time uncertainty-aware energy dispatch framework, which is composed of two key elements. The proposed framework is capable to rapidly adapt to the real-time data distribution, as well as to target on uncertainties caused by data drift, model discrepancy and environment perturbations in the control process. The framework won the championship in CityLearn Challenge 2022, which provided an influential opportunity to investigate the potential of AI application in the energy domain.
  arXiv  Detail & Related papers  (2023-09-15T00:04:00Z)
• Dichotomy of Control: Separating What You Can Control from What You Cannot [129.62135987416164]
  We propose a future-conditioned supervised learning framework that separates mechanisms within a policy's control (actions) from those beyond a policy's control (environmentity) We show that DoC yields policies that are consistent with their conditioning inputs, ensuring that conditioning a learned policy on a desired high-return future outcome will correctly induce high-return behavior.
  arXiv  Detail & Related papers  (2022-10-24T17:49:56Z)
• Improving the Performance of Robust Control through Event-Triggered Learning [74.57758188038375]
  We propose an event-triggered learning algorithm that decides when to learn in the face of uncertainty in the LQR problem. We demonstrate improved performance over a robust controller baseline in a numerical example.
  arXiv  Detail & Related papers  (2022-07-28T17:36:37Z)
• Imitating, Fast and Slow: Robust learning from demonstrations via decision-time planning [96.72185761508668]
  Planning at Test-time (IMPLANT) is a new meta-algorithm for imitation learning. We demonstrate that IMPLANT significantly outperforms benchmark imitation learning approaches on standard control environments.
  arXiv  Detail & Related papers  (2022-04-07T17:16:52Z)
• Temporal-Difference Value Estimation via Uncertainty-Guided Soft Updates [110.92598350897192]
  Q-Learning has proven effective at learning a policy to perform control tasks. estimation noise becomes a bias after the max operator in the policy improvement step. We present Unbiased Soft Q-Learning (UQL), which extends the work of EQL from two action, finite state spaces to multi-action, infinite state Markov Decision Processes.
  arXiv  Detail & Related papers  (2021-10-28T00:07:19Z)
• Development of a Soft Actor Critic Deep Reinforcement Learning Approach for Harnessing Energy Flexibility in a Large Office Building [0.0]
  This research is concerned with the novel application and investigation of Soft Actor Critic' (SAC) based Deep Reinforcement Learning (DRL) SAC is a model-free DRL technique that is able to handle continuous action spaces.
  arXiv  Detail & Related papers  (2021-04-25T10:33:35Z)
• A Relearning Approach to Reinforcement Learning for Control of Smart Buildings [1.8799681615947088]
  This paper demonstrates that continual relearning of control policies using incremental deep reinforcement learning (RL) can improve policy learning for non-stationary processes. We develop an incremental RL technique that simultaneously reduces building energy consumption without sacrificing overall comfort.
  arXiv  Detail & Related papers  (2020-08-04T23:31:05Z)
• Anticipating the Long-Term Effect of Online Learning in Control [75.6527644813815]
  AntLer is a design algorithm for learning-based control laws that anticipates learning. We show that AntLer approximates an optimal solution arbitrarily accurately with probability one.
  arXiv  Detail & Related papers  (2020-07-24T07:00:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.