Real-World Data and Calibrated Simulation Suite for Offline Training of Reinforcement Learning Agents to Optimize Energy and Emission in Buildings for Environmental Sustainability

• URL: http://arxiv.org/abs/2410.03756v1
• Date: Wed, 2 Oct 2024 06:30:07 GMT
• Title: Real-World Data and Calibrated Simulation Suite for Offline Training of Reinforcement Learning Agents to Optimize Energy and Emission in Buildings for Environmental Sustainability
• Authors: Judah Goldfeder, John Sipple,
• Abstract summary: We present the first open source interactive HVAC control dataset extracted from live sensor measurements of devices in real office buildings. For ease of use, our RL environments are all compatible with the OpenAI gym environment standard.
• Score: 2.7624021966289605
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Commercial office buildings contribute 17 percent of Carbon Emissions in the US, according to the US Energy Information Administration (EIA), and improving their efficiency will reduce their environmental burden and operating cost. A major contributor of energy consumption in these buildings are the Heating, Ventilation, and Air Conditioning (HVAC) devices. HVAC devices form a complex and interconnected thermodynamic system with the building and outside weather conditions, and current setpoint control policies are not fully optimized for minimizing energy use and carbon emission. Given a suitable training environment, a Reinforcement Learning (RL) agent is able to improve upon these policies, but training such a model, especially in a way that scales to thousands of buildings, presents many practical challenges. Most existing work on applying RL to this important task either makes use of proprietary data, or focuses on expensive and proprietary simulations that may not be grounded in the real world. We present the Smart Buildings Control Suite, the first open source interactive HVAC control dataset extracted from live sensor measurements of devices in real office buildings. The dataset consists of two components: six years of real-world historical data from three buildings, for offline RL, and a lightweight interactive simulator for each of these buildings, calibrated using the historical data, for online and model-based RL. For ease of use, our RL environments are all compatible with the OpenAI gym environment standard. We also demonstrate a novel method of calibrating the simulator, as well as baseline results on training an RL agent on the simulator, predicting real-world data, and training an RL agent directly from data. We believe this benchmark will accelerate progress and collaboration on building optimization and environmental sustainability research.

Related papers

• D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning [99.33607114541861]
  We propose a new benchmark for offline RL that focuses on realistic simulations of robotic manipulation and locomotion environments. Our proposed benchmark covers state-based and image-based domains, and supports both offline RL and online fine-tuning evaluation.
  arXiv  Detail & Related papers  (2024-08-15T22:27:00Z)
• Global Transformer Architecture for Indoor Room Temperature Forecasting [49.32130498861987]
  This work presents a global Transformer architecture for indoor temperature forecasting in multi-room buildings. It aims at optimizing energy consumption and reducing greenhouse gas emissions associated with HVAC systems. Notably, this study is the first to apply a Transformer architecture for indoor temperature forecasting in multi-room buildings.
  arXiv  Detail & Related papers  (2023-10-31T14:09:32Z)
• Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks [55.591662978280894]
  This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs) Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees.
  arXiv  Detail & Related papers  (2023-10-13T21:26:16Z)
• A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings [3.2634122554914002]
  We propose a novel simulation-based approach to train a Reinforcement Learning model. Our open-source simulator is lightweight and calibrated via telemetry from the building to reach a higher level of fidelity. This approach is an important step toward having a real-world RL control system that can be scaled to many buildings.
  arXiv  Detail & Related papers  (2023-10-12T17:56:23Z)
• Exploring Deep Reinforcement Learning for Holistic Smart Building Control [3.463438487417909]
  We develop a system called OCTOPUS that uses a data-driven approach to find the optimal control sequences of all building's subsystems. OCTOPUS can achieve 14.26% and 8.1% energy savings compared with the state-of-the-art rule-based method in a LEED Gold Certified building.
  arXiv  Detail & Related papers  (2023-01-27T03:03:21Z)
• MERLIN: Multi-agent offline and transfer learning for occupant-centric energy flexible operation of grid-interactive communities using smart meter data and CityLearn [0.0]
  Decarbonization of buildings presents new challenges for the reliability of the electrical grid. We propose the MERLIN framework and use a digital twin of a real-world grid-interactive residential community in CityLearn. We show that independent RL-controllers for batteries improve building and district level compared to a reference by tailoring their policies to individual buildings.
  arXiv  Detail & Related papers  (2022-12-31T21:37:14Z)
• Low Emission Building Control with Zero-Shot Reinforcement Learning [70.70479436076238]
  Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency. We show it is possible to obtain emission-reducing policies without a priori--a paradigm we call zero-shot building control.
  arXiv  Detail & Related papers  (2022-08-12T17:13:25Z)
• A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learning [48.63610479916003]
  This article breaks down and analyzes the main factors that influence the environmental footprint of distributed learning policies. It models both vanilla and decentralized FL policies driven by consensus. Results show that FL allows remarkable end-to-end energy savings (30%-40%) for wireless systems characterized by low bit/Joule efficiency.
  arXiv  Detail & Related papers  (2021-03-18T16:04:42Z)
• AI Chiller: An Open IoT Cloud Based Machine Learning Framework for the Energy Saving of Building HVAC System via Big Data Analytics on the Fusion of BMS and Environmental Data [12.681421165031576]
  Energy saving and carbon emission reduction in buildings is one of the key measures in combating climate change. The optimization of chiller system power consumption had been extensively studied in the mechanical engineering and building service domains. With the advance of big data and AI, the adoption of machine learning into the optimization problems becomes popular.
  arXiv  Detail & Related papers  (2020-10-09T09:51:03Z)
• NeurOpt: Neural network based optimization for building energy management and climate control [58.06411999767069]
  We propose a data-driven control algorithm based on neural networks to reduce this cost of model identification. We validate our learning and control algorithms on a two-story building with ten independently controlled zones, located in Italy.
  arXiv  Detail & Related papers  (2020-01-22T00:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.