Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings

• URL: http://arxiv.org/abs/2606.02049v1
• Date: Mon, 01 Jun 2026 10:39:34 GMT
• Title: Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings
• Abstract summary: This paper presents a framework for explainable deep reinforcement learning (XRL) applied to energy management in residential buildings.<n>We train and compare both on-policy and off-policy DRL agents on an expanded state space that incorporates real-time measurements.<n>Our results indicate that on-policy algorithms, particularly Advantage Actor Critic (A2C) and Proximal Policy Optimization (PPO), outperform off-policy methods in terms of cumulative rewards and policy stability.
• Score: 3.4756623477788593
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increasing integration of renewable energy sources into power systems, particularly in buildings equipped with photovoltaic (PV) panels and energy storage systems, introduces significant complexity in energy systems. Volatile power generation, varying electricity tariffs, and increased entities, e.g., PV systems, and heat pumps, have increased the complexity and made the system harder to operate. This leads to the demand for additional control and optimization routes including data-based controls, such as reinforcement learning. While deep reinforcement learning (DRL) has emerged as a promising solution to optimize building operations in dynamic and ever more complex environments, its black-box nature impedes user trust and practical adoption. This paper presents a framework for explainable deep reinforcement learning (XRL) applied to energy management in residential buildings. We demonstrate its usage on both synthetic data but also on real-world data from the Living Lab Energy Campus (LLEC) at KIT. We train and compare both on-policy and off-policy DRL agents on an expanded state space that incorporates real-time measurements (demand, PV generation, battery power, state of charge), external signals (dynamic electricity price, local weather data), calendrical and holiday indicators, and forecasts for demand and price. Our experimental results indicate that on-policy algorithms, particularly Advantage Actor Critic (A2C) and Proximal Policy Optimization (PPO), outperform off-policy methods in terms of cumulative rewards and policy stability. To explain these models, we employ post-hoc interpretation techniques to elaborate the learned control policies. Our findings demonstrate that the XRL framework not only reduces electricity costs through optimal battery management, but also provides transparent, actionable insights into the agent's decision-making process.

Related papers

• Shielded Controller Units for RL with Operational Constraints Applied to Remote Microgrids [50.64533198075622]
  Reinforcement learning (RL) is a powerful framework for optimizing decision-making in complex systems under uncertainty.<n>In this paper, we introduce Shielded Controller Units (SCUs), a systematic and interpretable approach that leverages prior knowledge of system dynamics.<n>We demonstrate the effectiveness of SCUs on a remote microgrid optimization task with strict operational requirements.
  arXiv  Detail & Related papers  (2025-11-30T19:28:34Z)
• Control of Renewable Energy Communities using AI and Real-World Data [0.0]
  This paper introduces a framework designed explicitly to handle these complexities and bridge the simulation to-reality gap.<n>It incorporates EnergAIze, a MADD-based multi-agent control strategy, and specifically addresses challenges related to real-world data collection, system integration, and user behavior modeling.
  arXiv  Detail & Related papers  (2025-05-22T22:20:09Z)
• Joint Resource Management for Energy-efficient UAV-assisted SWIPT-MEC: A Deep Reinforcement Learning Approach [50.52139512096988]
  6G Internet of Things (IoT) networks face challenges in remote areas and disaster scenarios where ground infrastructure is unavailable.<n>This paper proposes a novel aerial unmanned vehicle (UAV)-assisted computing (MEC) system enhanced by directional antennas to provide both computational and energy support for ground edge terminals.
  arXiv  Detail & Related papers  (2025-05-06T06:46:19Z)
• Deep Reinforcement Learning for Community Battery Scheduling under Uncertainties of Load, PV Generation, and Energy Prices [5.694872363688119]
  This paper presents a deep reinforcement learning (RL) strategy to schedule a community battery system in the presence of uncertainties. We position the community battery to play a versatile role, in integrating local PV energy, reducing peak load, and exploiting energy price fluctuations for arbitrage.
  arXiv  Detail & Related papers  (2023-12-04T13:45:17Z)
• Multi-market Energy Optimization with Renewables via Reinforcement Learning [1.0878040851638]
  This paper introduces a deep reinforcement learning framework for optimizing the operations of power plants pairing renewable energy with storage. The framework handles complexities such as time coupling by storage devices, uncertainty in renewable generation and energy prices, and non-linear storage models. It utilizes RL to incorporate complex storage models, overcoming restrictions of optimization-based methods that require convex and differentiable component models.
  arXiv  Detail & Related papers  (2023-06-13T21:35:24Z)
• Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework [53.97223237572147]
  This paper presents a multi-agent Deep Reinforcement Learning (DRL) framework for autonomous control and integration of renewable energy resources into smart power grid systems. In particular, the proposed framework jointly considers demand response (DR) and distributed energy management (DEM) for residential end-users.
  arXiv  Detail & Related papers  (2022-11-29T01:18:58Z)
• Low Emission Building Control with Zero-Shot Reinforcement Learning [70.70479436076238]
  Control via Reinforcement Learning (RL) has been shown to significantly improve building energy efficiency. We show it is possible to obtain emission-reducing policies without a priori--a paradigm we call zero-shot building control.
  arXiv  Detail & Related papers  (2022-08-12T17:13:25Z)
• Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning [1.8175650854482457]
  The model-free deep reinforcement learning algorithm Rainbow Deep Q-Networks is used to control a battery in a small microgrid. The grid operates with its own demand and renewable generation based on a dataset collected at Keele University.
  arXiv  Detail & Related papers  (2021-06-10T21:27:35Z)
• A Relearning Approach to Reinforcement Learning for Control of Smart Buildings [1.8799681615947088]
  This paper demonstrates that continual relearning of control policies using incremental deep reinforcement learning (RL) can improve policy learning for non-stationary processes. We develop an incremental RL technique that simultaneously reduces building energy consumption without sacrificing overall comfort.
  arXiv  Detail & Related papers  (2020-08-04T23:31:05Z)
• Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems [87.4519172058185]
  An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
  arXiv  Detail & Related papers  (2020-02-20T04:58:07Z)
• NeurOpt: Neural network based optimization for building energy management and climate control [58.06411999767069]
  We propose a data-driven control algorithm based on neural networks to reduce this cost of model identification. We validate our learning and control algorithms on a two-story building with ten independently controlled zones, located in Italy.
  arXiv  Detail & Related papers  (2020-01-22T00:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.