MERLIN: Multi-agent offline and transfer learning for occupant-centric energy flexible operation of grid-interactive communities using smart meter data and CityLearn

• URL: http://arxiv.org/abs/2301.01148v1
• Date: Sat, 31 Dec 2022 21:37:14 GMT
• Title: MERLIN: Multi-agent offline and transfer learning for occupant-centric energy flexible operation of grid-interactive communities using smart meter data and CityLearn
• Authors: Kingsley Nweye and Siva Sankaranarayanan and Zoltan Nagy
• Abstract summary: Decarbonization of buildings presents new challenges for the reliability of the electrical grid. We propose the MERLIN framework and use a digital twin of a real-world grid-interactive residential community in CityLearn. We show that independent RL-controllers for batteries improve building and district level compared to a reference by tailoring their policies to individual buildings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The decarbonization of buildings presents new challenges for the reliability of the electrical grid as a result of the intermittency of renewable energy sources and increase in grid load brought about by end-use electrification. To restore reliability, grid-interactive efficient buildings can provide flexibility services to the grid through demand response. Residential demand response programs are hindered by the need for manual intervention by customers. To maximize the energy flexibility potential of residential buildings, an advanced control architecture is needed. Reinforcement learning is well-suited for the control of flexible resources as it is able to adapt to unique building characteristics compared to expert systems. Yet, factors hindering the adoption of RL in real-world applications include its large data requirements for training, control security and generalizability. Here we address these challenges by proposing the MERLIN framework and using a digital twin of a real-world 17-building grid-interactive residential community in CityLearn. We show that 1) independent RL-controllers for batteries improve building and district level KPIs compared to a reference RBC by tailoring their policies to individual buildings, 2) despite unique occupant behaviours, transferring the RL policy of any one of the buildings to other buildings provides comparable performance while reducing the cost of training, 3) training RL-controllers on limited temporal data that does not capture full seasonality in occupant behaviour has little effect on performance. Although, the zero-net-energy (ZNE) condition of the buildings could be maintained or worsened as a result of controlled batteries, KPIs that are typically improved by ZNE condition (electricity price and carbon emissions) are further improved when the batteries are managed by an advanced controller.

Related papers

• CityLearn v2: Energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive communities [8.658740257657564]
  CityLearn provides an environment for benchmarking simple and advanced distributed energy resource control algorithms. This work details the v2 environment design and provides application examples that utilize reinforcement learning to manage battery energy storage system charging/discharging cycles, vehicle-to-grid control, and thermal comfort during heat pump power modulation.
  arXiv  Detail & Related papers  (2024-05-02T16:31:09Z)
• Explainable Reinforcement Learning-based Home Energy Management Systems using Differentiable Decision Trees [4.573008040057806]
  The residential sector is another major and largely untapped source of flexibility, driven by the increased adoption of solar PV, home batteries, and EVs. We introduce a reinforcement learning-based approach using differentiable decision trees. This approach integrates the scalability of data-driven reinforcement learning with the explainability of (differentiable) decision trees. As a proof-of-concept, we analyze our method using a home energy management problem, comparing its performance with commercially available rule-based baseline and standard neural network-based RL controllers.
  arXiv  Detail & Related papers  (2024-03-18T16:40:41Z)
• Global Transformer Architecture for Indoor Room Temperature Forecasting [49.32130498861987]
  This work presents a global Transformer architecture for indoor temperature forecasting in multi-room buildings. It aims at optimizing energy consumption and reducing greenhouse gas emissions associated with HVAC systems. Notably, this study is the first to apply a Transformer architecture for indoor temperature forecasting in multi-room buildings.
  arXiv  Detail & Related papers  (2023-10-31T14:09:32Z)
• Real-World Implementation of Reinforcement Learning Based Energy Coordination for a Cluster of Households [3.901860248668672]
  We present a real-life pilot study that studies the effectiveness of reinforcement-learning (RL) in coordinating the power consumption of 8 residential buildings to jointly track a target power signal. Our results demonstrate satisfactory power tracking, and the effectiveness of the RL-based ranks which are learnt in a purely data-driven manner.
  arXiv  Detail & Related papers  (2023-10-29T21:10:38Z)
• Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks [55.591662978280894]
  This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs) Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees.
  arXiv  Detail & Related papers  (2023-10-13T21:26:16Z)
• Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
  This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems. In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
  arXiv  Detail & Related papers  (2022-11-29T01:18:58Z)
• Low Emission Building Control with Zero-Shot Reinforcement Learning [70.70479436076238]
  Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency. We show it is possible to obtain emission-reducing policies without a priori--a paradigm we call zero-shot building control.
  arXiv  Detail & Related papers  (2022-08-12T17:13:25Z)
• Zero-Shot Building Control [0.0]
  Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency. Existing solutions require pre-training in simulators that are prohibitively expensive to obtain for every building in the world. We show it is possible to perform safe, zero-shot control of buildings by combining ideas from system identification and model-based RL.
  arXiv  Detail & Related papers  (2022-06-28T17:56:40Z)
• Investigating Underlying Drivers of Variability in Residential Energy Usage Patterns with Daily Load Shape Clustering of Smart Meter Data [53.51471969978107]
  Large-scale deployment of smart meters has motivated increasing studies to explore disaggregated daily load patterns. This paper aims to shed light on the mechanisms by which electricity consumption patterns exhibit variability.
  arXiv  Detail & Related papers  (2021-02-16T16:56:27Z)
• NeurOpt: Neural network based optimization for building energy management and climate control [58.06411999767069]
  We propose a data-driven control algorithm based on neural networks to reduce this cost of model identification. We validate our learning and control algorithms on a two-story building with ten independently controlled zones, located in Italy.
  arXiv  Detail & Related papers  (2020-01-22T00:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.