Utilizing Explainability Techniques for Reinforcement Learning Model Assurance

• URL: http://arxiv.org/abs/2311.15838v1
• Date: Mon, 27 Nov 2023 14:02:47 GMT
• Title: Utilizing Explainability Techniques for Reinforcement Learning Model Assurance
• Authors: Alexander Tapley and Kyle Gatesman and Luis Robaina and Brett Bissey and Joseph Weissman
• Abstract summary: Explainable Reinforcement Learning (XRL) can provide transparency into the decision-making process of a Deep Reinforcement Learning (DRL) model. This paper introduces the ARLIN (Assured RL Model Interrogation) Toolkit, an open-source Python library that identifies potential vulnerabilities and critical points within trained DRL models.
• Score: 42.302469854610315
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Explainable Reinforcement Learning (XRL) can provide transparency into the decision-making process of a Deep Reinforcement Learning (DRL) model and increase user trust and adoption in real-world use cases. By utilizing XRL techniques, researchers can identify potential vulnerabilities within a trained DRL model prior to deployment, therefore limiting the potential for mission failure or mistakes by the system. This paper introduces the ARLIN (Assured RL Model Interrogation) Toolkit, an open-source Python library that identifies potential vulnerabilities and critical points within trained DRL models through detailed, human-interpretable explainability outputs. To illustrate ARLIN's effectiveness, we provide explainability visualizations and vulnerability analysis for a publicly available DRL model. The open-source code repository is available for download at https://github.com/mitre/arlin.

Related papers

• A Survey for Deep Reinforcement Learning Based Network Intrusion Detection [3.493620624883548]
  This paper explores the potential and challenges of using deep reinforcement learning (DRL) in network intrusion detection. The performance of DRL models is analyzed, showing that while DRL holds promise, many recent technologies remain underexplored. The paper concludes with recommendations for enhancing DRL deployment and testing in real-world network scenarios.
  arXiv  Detail & Related papers  (2024-09-25T13:39:30Z)
• How Can LLM Guide RL? A Value-Based Approach [68.55316627400683]
  Reinforcement learning (RL) has become the de facto standard practice for sequential decision-making problems by improving future acting policies with feedback. Recent developments in large language models (LLMs) have showcased impressive capabilities in language understanding and generation, yet they fall short in exploration and self-improvement capabilities. We develop an algorithm named LINVIT that incorporates LLM guidance as a regularization factor in value-based RL, leading to significant reductions in the amount of data needed for learning.
  arXiv  Detail & Related papers  (2024-02-25T20:07:13Z)
• Analyzing Adversarial Inputs in Deep Reinforcement Learning [53.3760591018817]
  We present a comprehensive analysis of the characterization of adversarial inputs, through the lens of formal verification. We introduce a novel metric, the Adversarial Rate, to classify models based on their susceptibility to such perturbations. Our analysis empirically demonstrates how adversarial inputs can affect the safety of a given DRL system with respect to such perturbations.
  arXiv  Detail & Related papers  (2024-02-07T21:58:40Z)
• Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint [104.53687944498155]
  Reinforcement learning (RL) has been widely used in training large language models (LLMs) We propose a new RL method named RLMEC that incorporates a generative model as the reward model. Based on the generative reward model, we design the token-level RL objective for training and an imitation-based regularization for stabilizing RL process.
  arXiv  Detail & Related papers  (2024-01-11T17:58:41Z)
• ORL-AUDITOR: Dataset Auditing in Offline Deep Reinforcement Learning [42.87245000172943]
  offline deep reinforcement learning ( offline DRL) is frequently used to train models on pre-collected datasets. We propose ORL-AUDITOR, which is the first trajectory-level dataset auditing mechanism for offline DRL scenarios. Our experiments on multiple offline DRL models and tasks reveal the efficacy of ORL-AUDITOR, with auditing accuracy over 95% and false positive rates less than 2.88%.
  arXiv  Detail & Related papers  (2023-09-06T15:28:43Z)
• A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges [38.70863329476517]
  Reinforcement Learning (RL) is a popular machine learning paradigm where intelligent agents interact with the environment to fulfill a long-term goal. Despite the encouraging results achieved, the deep neural network-based backbone is widely deemed as a black box that impedes practitioners to trust and employ trained agents in realistic scenarios where high security and reliability are essential. To alleviate this issue, a large volume of literature devoted to shedding light on the inner workings of the intelligent agents has been proposed, by constructing intrinsic interpretability or post-hoc explainability.
  arXiv  Detail & Related papers  (2022-11-12T13:52:06Z)
• Implicit Offline Reinforcement Learning via Supervised Learning [83.8241505499762]
  Offline Reinforcement Learning (RL) via Supervised Learning is a simple and effective way to learn robotic skills from a dataset collected by policies of different expertise levels. We show how implicit models can leverage return information and match or outperform explicit algorithms to acquire robotic skills from fixed datasets.
  arXiv  Detail & Related papers  (2022-10-21T21:59:42Z)
• Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approach [13.57291726431012]
  Deep Reinforcement Learning (DRL) is the application of Deep Learning in the domain of Reinforcement Learning (RL) In this paper, we present the first attempt to categorize faults occurring in DRL programs. We have defined a meta-model of DRL programs and developed DRLinter, a model-based fault detection approach.
  arXiv  Detail & Related papers  (2021-01-01T01:49:03Z)
• Explainability in Deep Reinforcement Learning [68.8204255655161]
  We review recent works in the direction to attain Explainable Reinforcement Learning (XRL) In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
  arXiv  Detail & Related papers  (2020-08-15T10:11:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.