Interpretable reinforcement learning for heat pump control through asymmetric differentiable decision trees

• URL: http://arxiv.org/abs/2506.01641v1
• Date: Mon, 02 Jun 2025 13:16:00 GMT
• Title: Interpretable reinforcement learning for heat pump control through asymmetric differentiable decision trees
• Authors: Toon Van Puyvelde, Mehran Zareh, Chris Develder,
• Abstract summary: We propose a novel asymmetric soft DDT construction method.<n>Unlike traditional soft DDTs, our approach adaptively constructs trees by expanding nodes only when necessary.<n>We demonstrate the potential of asymmetric DDTs to provide transparent, efficient, and high-performing decision-making in home energy management systems.
• Score: 4.573008040057806
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, deep reinforcement learning (DRL) algorithms have gained traction in home energy management systems. However, their adoption by energy management companies remains limited due to the black-box nature of DRL, which fails to provide transparent decision-making feedback. To address this, explainable reinforcement learning (XRL) techniques have emerged, aiming to make DRL decisions more transparent. Among these, soft differential decision tree (DDT) distillation provides a promising approach due to the clear decision rules they are based on, which can be efficiently computed. However, achieving high performance often requires deep, and completely full, trees, which reduces interpretability. To overcome this, we propose a novel asymmetric soft DDT construction method. Unlike traditional soft DDTs, our approach adaptively constructs trees by expanding nodes only when necessary. This improves the efficient use of decision nodes, which require a predetermined depth to construct full symmetric trees, enhancing both interpretability and performance. We demonstrate the potential of asymmetric DDTs to provide transparent, efficient, and high-performing decision-making in home energy management systems.

Related papers

• TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree [52.44403214958304]
  In this paper, we introduce TreeLoRA, a novel approach that constructs layer-wise adapters by leveraging hierarchical gradient similarity.<n>To reduce the computational burden of task similarity estimation, we employ bandit techniques to develop an algorithm based on lower confidence bounds.<n> experiments on both vision transformers (ViTs) and large language models (LLMs) demonstrate the effectiveness and efficiency of our approach.
  arXiv  Detail & Related papers  (2025-06-12T05:25:35Z)
• Monte Carlo Tree Diffusion for System 2 Planning [57.50512800900167]
  We introduce Monte Carlo Tree Diffusion (MCTD), a novel framework that integrates the generative strength of diffusion models with the adaptive search capabilities of Monte Carlo Tree Search (MCTS)<n>MCTD achieves the benefits of MCTS such as controlling exploration-exploitation trade-offs within the diffusion framework.
  arXiv  Detail & Related papers  (2025-02-11T02:51:42Z)
• Enhance Learning Efficiency of Oblique Decision Tree via Feature Concatenation [16.81813720905545]
  We propose an enhanced ODT method with Feature Concatenation (textttFC-ODT)<n>textttFC-ODT enables in-model feature transformation to transmit the projections along the decision paths.<n>Experiments show that textttFC-ODT can outperform the other state-of-the-art decision trees with a limited tree depth.
  arXiv  Detail & Related papers  (2025-02-01T15:49:18Z)
• RL-LLM-DT: An Automatic Decision Tree Generation Method Based on RL Evaluation and LLM Enhancement [82.02155942106877]
  We propose RL-LLM-DT, an automatic decision tree generation method based on RL Evaluation and LLM Enhancement.<n>To evaluate the effectiveness of this integrated approach, we conducted experiments in a curling game.
  arXiv  Detail & Related papers  (2024-12-16T03:33:49Z)
• RGMDT: Return-Gap-Minimizing Decision Tree Extraction in Non-Euclidean Metric Space [28.273737052758907]
  We introduce an upper bound on the return gap between the oracle expert policy and an optimal decision tree policy. This enables us to recast the DT extraction problem into a novel non-euclidean clustering problem over the local observation and action values space of each agent. We also propose the Return-Gap-Minimization Decision Tree (RGMDT) algorithm, which is a surprisingly simple design and is integrated with reinforcement learning.
  arXiv  Detail & Related papers  (2024-10-21T21:19:49Z)
• Learning Deep Tree-based Retriever for Efficient Recommendation: Theory and Method [76.31185707649227]
  We propose a Deep Tree-based Retriever (DTR) for efficient recommendation. DTR frames the training task as a softmax-based multi-class classification over tree nodes at the same level. To mitigate the suboptimality induced by the labeling of non-leaf nodes, we propose a rectification method for the loss function.
  arXiv  Detail & Related papers  (2024-08-21T05:09:53Z)
• Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks [55.591662978280894]
  This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs) Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees.
  arXiv  Detail & Related papers  (2023-10-13T21:26:16Z)
• Limits of Actor-Critic Algorithms for Decision Tree Policies Learning in IBMDPs [9.587070290189507]
  Interpretability of AI models allows for user safety checks to build trust in such AIs. Decision Trees (DTs) provide a global look at the learned model and transparently reveal which features of the input are critical for making a decision. Recent Reinforcement Learning framework has been proposed to explore the space of DTs using deep RL.
  arXiv  Detail & Related papers  (2023-09-23T13:06:20Z)
• Breiman meets Bellman: Non-Greedy Decision Trees with MDPs [8.530182510074983]
  We present Dynamic Programming Decision Trees (DPDT), a framework that bridges the gap between greedy and optimal approaches.<n>DPDT relies on a Markov Decision Process formulation combined with split generation to construct near-optimal decision trees.<n>Our empirical study shows that DPDT achieves near-optimal loss with orders of magnitude fewer operations than existing optimal solvers.
  arXiv  Detail & Related papers  (2023-09-22T08:18:08Z)
• Optimal Planning of Hybrid Energy Storage Systems using Curtailed Renewable Energy through Deep Reinforcement Learning [0.0]
  We propose a sophisticated deep reinforcement learning (DRL) methodology with a policy-based algorithm to plan energy storage systems (ESS) A quantitative performance comparison proved that the DRL agent outperforms the scenario-based optimization (SO) algorithm. The corresponding results confirmed that the DRL agent learns the way like what a human expert would do, suggesting reliable application of the proposed methodology.
  arXiv  Detail & Related papers  (2022-12-12T02:24:50Z)
• Knowledge Distillation Decision Tree for Unravelling Black-box Machine Learning Models [0.0]
  We introduce the method of knowledge distillation decision tree (KDDT)<n>KDDT enables the distillation of knowledge about the data from a black-box model into a decision tree.<n>An efficient algorithm is provided for constructing the hybrid KDDT.
  arXiv  Detail & Related papers  (2022-06-09T17:57:37Z)
• Rectified Decision Trees: Exploring the Landscape of Interpretable and Effective Machine Learning [66.01622034708319]
  We propose a knowledge distillation based decision trees extension, dubbed rectified decision trees (ReDT) We extend the splitting criteria and the ending condition of the standard decision trees, which allows training with soft labels. We then train the ReDT based on the soft label distilled from a well-trained teacher model through a novel jackknife-based method.
  arXiv  Detail & Related papers  (2020-08-21T10:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.