A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings

• URL: http://arxiv.org/abs/2310.08569v1
• Date: Thu, 12 Oct 2023 17:56:23 GMT
• Title: A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings
• Authors: Judah Goldfeder, John Sipple
• Abstract summary: We propose a novel simulation-based approach to train a Reinforcement Learning model. Our open-source simulator is lightweight and calibrated via telemetry from the building to reach a higher level of fidelity. This approach is an important step toward having a real-world RL control system that can be scaled to many buildings.
• Score: 3.2634122554914002
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern commercial Heating, Ventilation, and Air Conditioning (HVAC) devices form a complex and interconnected thermodynamic system with the building and outside weather conditions, and current setpoint control policies are not fully optimized for minimizing energy use and carbon emission. Given a suitable training environment, a Reinforcement Learning (RL) model is able to improve upon these policies, but training such a model, especially in a way that scales to thousands of buildings, presents many real world challenges. We propose a novel simulation-based approach, where a customized simulator is used to train the agent for each building. Our open-source simulator (available online: https://github.com/google/sbsim) is lightweight and calibrated via telemetry from the building to reach a higher level of fidelity. On a two-story, 68,000 square foot building, with 127 devices, we were able to calibrate our simulator to have just over half a degree of drift from the real world over a six-hour interval. This approach is an important step toward having a real-world RL control system that can be scaled to many buildings, allowing for greater efficiency and resulting in reduced energy consumption and carbon emissions.

Related papers

• Real-World Data and Calibrated Simulation Suite for Offline Training of Reinforcement Learning Agents to Optimize Energy and Emission in Buildings for Environmental Sustainability [2.7624021966289605]
  We present the first open source interactive HVAC control dataset extracted from live sensor measurements of devices in real office buildings. For ease of use, our RL environments are all compatible with the OpenAI gym environment standard.
  arXiv  Detail & Related papers  (2024-10-02T06:30:07Z)
• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [60.05963742334746]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers. Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy. We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [76.93956925360638]
  Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training. We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
  arXiv  Detail & Related papers  (2023-10-12T20:49:15Z)
• Exploring Deep Reinforcement Learning for Holistic Smart Building Control [3.463438487417909]
  We develop a system called OCTOPUS that uses a data-driven approach to find the optimal control sequences of all building's subsystems. OCTOPUS can achieve 14.26% and 8.1% energy savings compared with the state-of-the-art rule-based method in a LEED Gold Certified building.
  arXiv  Detail & Related papers  (2023-01-27T03:03:21Z)
• Parallel Reinforcement Learning Simulation for Visual Quadrotor Navigation [4.597465975849579]
  Reinforcement learning (RL) is an agent-based approach for teaching robots to navigate within the physical world. We present a simulation framework, built on AirSim, which provides efficient parallel training. Building on this framework, Ape-X is modified to incorporate decentralised training of AirSim environments.
  arXiv  Detail & Related papers  (2022-09-22T15:27:42Z)
• Low Emission Building Control with Zero-Shot Reinforcement Learning [70.70479436076238]
  Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency. We show it is possible to obtain emission-reducing policies without a priori--a paradigm we call zero-shot building control.
  arXiv  Detail & Related papers  (2022-08-12T17:13:25Z)
• Accelerated Policy Learning with Parallel Differentiable Simulation [59.665651562534755]
  We present a differentiable simulator and a new policy learning algorithm (SHAC) Our algorithm alleviates problems with local minima through a smooth critic function. We show substantial improvements in sample efficiency and wall-clock time over state-of-the-art RL and differentiable simulation-based algorithms.
  arXiv  Detail & Related papers  (2022-04-14T17:46:26Z)
• One for Many: Transfer Learning for Building HVAC Control [24.78264822089494]
  We present a novel transfer learning based approach to overcome this challenge. Our approach can effectively transfer a DRL-based HVAC controller trained for the source building to a controller for the target building with minimal effort and improved performance.
  arXiv  Detail & Related papers  (2020-08-09T01:32:37Z)
• NeurOpt: Neural network based optimization for building energy management and climate control [58.06411999767069]
  We propose a data-driven control algorithm based on neural networks to reduce this cost of model identification. We validate our learning and control algorithms on a two-story building with ten independently controlled zones, located in Italy.
  arXiv  Detail & Related papers  (2020-01-22T00:51:03Z)
• Information Theoretic Model Predictive Q-Learning [64.74041985237105]
  We present a novel theoretical connection between information theoretic MPC and entropy regularized RL. We develop a Q-learning algorithm that can leverage biased models.
  arXiv  Detail & Related papers  (2019-12-31T00:29:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.