Interpretable Modeling of Deep Reinforcement Learning Driven Scheduling

• URL: http://arxiv.org/abs/2403.16293v1
• Date: Sun, 24 Mar 2024 20:56:16 GMT
• Title: Interpretable Modeling of Deep Reinforcement Learning Driven Scheduling
• Authors: Boyang Li, Zhiling Lan, Michael E. Papka,
• Abstract summary: We present a framework called IRL (Interpretable Reinforcement Learning) to address the issue of interpretability of DRL scheduling. ILR is capable of converting a black-box DNN policy into an interpretable rulebased decision tree while maintaining comparable scheduling performance.
• Score: 3.890533943135602
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the field of high-performance computing (HPC), there has been recent exploration into the use of deep reinforcement learning for cluster scheduling (DRL scheduling), which has demonstrated promising outcomes. However, a significant challenge arises from the lack of interpretability in deep neural networks (DNN), rendering them as black-box models to system managers. This lack of model interpretability hinders the practical deployment of DRL scheduling. In this work, we present a framework called IRL (Interpretable Reinforcement Learning) to address the issue of interpretability of DRL scheduling. The core idea is to interpret DNN (i.e., the DRL policy) as a decision tree by utilizing imitation learning. Unlike DNN, decision tree models are non-parametric and easily comprehensible to humans. To extract an effective and efficient decision tree, IRL incorporates the Dataset Aggregation (DAgger) algorithm and introduces the notion of critical state to prune the derived decision tree. Through trace-based experiments, we demonstrate that IRL is capable of converting a black-box DNN policy into an interpretable rulebased decision tree while maintaining comparable scheduling performance. Additionally, IRL can contribute to the setting of rewards in DRL scheduling.

Related papers

• Upside-Down Reinforcement Learning for More Interpretable Optimal Control [2.06242362470764]
  We investigate whether function approximation algorithms other than Neural Networks (NNs) can also be used within a Upside-Down Reinforcement Learning framework. Our experiments, performed over several popular optimal control benchmarks, show that tree-based methods like Random Forests and Extremely Randomized Trees can perform just as well as NNs.
  arXiv  Detail & Related papers  (2024-11-18T10:44:20Z)
• Making Large Language Models Better Planners with Reasoning-Decision Alignment [70.5381163219608]
  We motivate an end-to-end decision-making model based on multimodality-augmented LLM. We propose a reasoning-decision alignment constraint between the paired CoTs and planning results. We dub our proposed large language planners with reasoning-decision alignment as RDA-Driver.
  arXiv  Detail & Related papers  (2024-08-25T16:43:47Z)
• Compressing Deep Reinforcement Learning Networks with a Dynamic Structured Pruning Method for Autonomous Driving [63.155562267383864]
  Deep reinforcement learning (DRL) has shown remarkable success in complex autonomous driving scenarios. DRL models inevitably bring high memory consumption and computation, which hinders their wide deployment in resource-limited autonomous driving devices. We introduce a novel dynamic structured pruning approach that gradually removes a DRL model's unimportant neurons during the training stage.
  arXiv  Detail & Related papers  (2024-02-07T09:00:30Z)
• Differentiable Tree Search Network [14.972768001402898]
  Differentiable Tree Search Network (D-TSN) is a novel neural network architecture that significantly strengthens the inductive bias. D-TSN employs a learned world model to conduct a fully differentiable online search. We demonstrate that D-TSN outperforms popular model-free and model-based baselines.
  arXiv  Detail & Related papers  (2024-01-22T02:33:38Z)
• A Neuromorphic Architecture for Reinforcement Learning from Real-Valued Observations [0.34410212782758043]
  Reinforcement Learning (RL) provides a powerful framework for decision-making in complex environments. This paper presents a novel Spiking Neural Network (SNN) architecture for solving RL problems with real-valued observations.
  arXiv  Detail & Related papers  (2023-07-06T12:33:34Z)
• Optimal Interpretability-Performance Trade-off of Classification Trees with Black-Box Reinforcement Learning [0.0]
  Interpretability of AI models allows for user safety checks to build trust in these models. Decision trees (DTs) provide a global view on the learned model and clearly outlines the role of the features that are critical to classify a given data. To learn compact trees, a Reinforcement Learning framework has been recently proposed to explore the space of DTs.
  arXiv  Detail & Related papers  (2023-04-11T09:43:23Z)
• Scaling Laws Beyond Backpropagation [64.0476282000118]
  We study the ability of Direct Feedback Alignment to train causal decoder-only Transformers efficiently. We find that DFA fails to offer more efficient scaling than backpropagation.
  arXiv  Detail & Related papers  (2022-10-26T10:09:14Z)
• Optimal Decision Diagrams for Classification [68.72078059880018]
  We study the training of optimal decision diagrams from a mathematical programming perspective. We introduce a novel mixed-integer linear programming model for training. We show how this model can be easily extended for fairness, parsimony, and stability notions.
  arXiv  Detail & Related papers  (2022-05-28T18:31:23Z)
• Online Limited Memory Neural-Linear Bandits with Likelihood Matching [53.18698496031658]
  We study neural-linear bandits for solving problems where both exploration and representation learning play an important role. We propose a likelihood matching algorithm that is resilient to catastrophic forgetting and is completely online.
  arXiv  Detail & Related papers  (2021-02-07T14:19:07Z)
• Chance-Constrained Control with Lexicographic Deep Reinforcement Learning [77.34726150561087]
  This paper proposes a lexicographic Deep Reinforcement Learning (DeepRL)-based approach to chance-constrained Markov Decision Processes. A lexicographic version of the well-known DeepRL algorithm DQN is also proposed and validated via simulations.
  arXiv  Detail & Related papers  (2020-10-19T13:09:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.