A Relearning Approach to Reinforcement Learning for Control of Smart Buildings

• URL: http://arxiv.org/abs/2008.01879v1
• Date: Tue, 4 Aug 2020 23:31:05 GMT
• Title: A Relearning Approach to Reinforcement Learning for Control of Smart Buildings
• Authors: Avisek Naug and Marcos Qui\~nones-Grueiro and Gautam Biswas
• Abstract summary: This paper demonstrates that continual relearning of control policies using incremental deep reinforcement learning (RL) can improve policy learning for non-stationary processes. We develop an incremental RL technique that simultaneously reduces building energy consumption without sacrificing overall comfort.
• Score: 1.8799681615947088
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper demonstrates that continual relearning of control policies using incremental deep reinforcement learning (RL) can improve policy learning for non-stationary processes. We demonstrate this approach for a data-driven 'smart building environment' that we use as a test-bed for developing HVAC controllers for reducing energy consumption of large buildings on our university campus. The non-stationarity in building operations and weather patterns makes it imperative to develop control strategies that are adaptive to changing conditions. On-policy RL algorithms, such as Proximal Policy Optimization (PPO) represent an approach for addressing this non-stationarity, but exploration on the actual system is not an option for safety-critical systems. As an alternative, we develop an incremental RL technique that simultaneously reduces building energy consumption without sacrificing overall comfort. We compare the performance of our incremental RL controller to that of a static RL controller that does not implement the relearning function. The performance of the static controller diminishes significantly over time, but the relearning controller adjusts to changing conditions while ensuring comfort and optimal energy performance.

Related papers

• Deployable Reinforcement Learning with Variable Control Rate [14.838483990647697]
  We propose a variant of Reinforcement Learning (RL) with variable control rate. In this approach, the policy decides the action the agent should take as well as the duration of the time step associated with that action. We show the efficacy of SEAC through a proof-of-concept simulation driving an agent with Newtonian kinematics.
  arXiv  Detail & Related papers  (2024-01-17T15:40:11Z)
• Action-Quantized Offline Reinforcement Learning for Robotic Skill Learning [68.16998247593209]
  offline reinforcement learning (RL) paradigm provides recipe to convert static behavior datasets into policies that can perform better than the policy that collected the data. In this paper, we propose an adaptive scheme for action quantization. We show that several state-of-the-art offline RL methods such as IQL, CQL, and BRAC improve in performance on benchmarks when combined with our proposed discretization scheme.
  arXiv  Detail & Related papers  (2023-10-18T06:07:10Z)
• Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks [55.591662978280894]
  This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs) Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees.
  arXiv  Detail & Related papers  (2023-10-13T21:26:16Z)
• Efficient Learning of Voltage Control Strategies via Model-based Deep Reinforcement Learning [9.936452412191326]
  This article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results in model-free DRL-based methods for power systems, but model-free methods suffer from poor sample efficiency and training time. We propose a novel model-based-DRL framework where a deep neural network (DNN)-based dynamic surrogate model is utilized with the policy learning framework.
  arXiv  Detail & Related papers  (2022-12-06T02:50:53Z)
• Steady-State Error Compensation in Reference Tracking and Disturbance Rejection Problems for Reinforcement Learning-Based Control [0.9023847175654602]
  Reinforcement learning (RL) is a promising, upcoming topic in automatic control applications. Initiative action state augmentation (IASA) for actor-critic-based RL controllers is introduced. This augmentation does not require any expert knowledge, leaving the approach model free.
  arXiv  Detail & Related papers  (2022-01-31T16:29:19Z)
• Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training [71.7750435554693]
  We show that several state-of-the-art RL agents proposed for power system control are vulnerable to adversarial attacks. Specifically, we use an adversary Markov Decision Process to learn an attack policy, and demonstrate the potency of our attack. We propose to use adversarial training to increase the robustness of RL agent against attacks and avoid infeasible operational decisions.
  arXiv  Detail & Related papers  (2021-10-18T00:50:34Z)
• Development of a Soft Actor Critic Deep Reinforcement Learning Approach for Harnessing Energy Flexibility in a Large Office Building [0.0]
  This research is concerned with the novel application and investigation of Soft Actor Critic' (SAC) based Deep Reinforcement Learning (DRL) SAC is a model-free DRL technique that is able to handle continuous action spaces.
  arXiv  Detail & Related papers  (2021-04-25T10:33:35Z)
• Regularizing Action Policies for Smooth Control with Reinforcement Learning [47.312768123967025]
  Conditioning for Action Policy Smoothness (CAPS) is an effective yet intuitive regularization on action policies. CAPS offers consistent improvement in the smoothness of the learned state-to-action mappings of neural network controllers. Tested on a real system, improvements in controller smoothness on a quadrotor drone resulted in an almost 80% reduction in power consumption.
  arXiv  Detail & Related papers  (2020-12-11T21:35:24Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)
• Data-driven control of micro-climate in buildings: an event-triggered reinforcement learning approach [56.22460188003505]
  We formulate the micro-climate control problem based on semi-Markov decision processes. We propose two learning algorithms for event-triggered control of micro-climate in buildings. We show the efficacy of our proposed approach via designing a smart learning thermostat.
  arXiv  Detail & Related papers  (2020-01-28T18:20:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.