Abstracted Trajectory Visualization for Explainability in Reinforcement Learning

• URL: http://arxiv.org/abs/2402.07928v1
• Date: Mon, 5 Feb 2024 21:17:44 GMT
• Title: Abstracted Trajectory Visualization for Explainability in Reinforcement Learning
• Authors: Yoshiki Takagi, Roderick Tabalba, Nurit Kirshenbaum, Jason Leigh
• Abstract summary: Explainable AI (XAI) has demonstrated the potential to help reinforcement learning (RL) practitioners to understand how RL models work. XAI for users who do not have RL expertise (non-RL experts) has not been studied sufficiently. We argue that abstracted trajectories, that depicts transitions between the major states of the RL model, will be useful for non-RL experts to build a mental model of the agents.
• Score: 2.1028463367241033
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Explainable AI (XAI) has demonstrated the potential to help reinforcement learning (RL) practitioners to understand how RL models work. However, XAI for users who do not have RL expertise (non-RL experts), has not been studied sufficiently. This results in a difficulty for the non-RL experts to participate in the fundamental discussion of how RL models should be designed for an incoming society where humans and AI coexist. Solving such a problem would enable RL experts to communicate with the non-RL experts in producing machine learning solutions that better fit our society. We argue that abstracted trajectories, that depicts transitions between the major states of the RL model, will be useful for non-RL experts to build a mental model of the agents. Our early results suggest that by leveraging a visualization of the abstracted trajectories, users without RL expertise are able to infer the behavior patterns of RL.

Related papers

• Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learning [69.19840497497503]
  It is argued that the commonly used action matching principle is more like an explanation of deep neural networks (DNNs) than the interpretation of RL agents. We propose to consider rewards, the essential objective of RL agents, as the essential objective of interpreting RL agents. We verify and evaluate our method on the Atari 2600 games as well as Duckietown, a challenging self-driving car simulator environment.
  arXiv  Detail & Related papers  (2023-09-04T09:09:54Z)
• A User Study on Explainable Online Reinforcement Learning for Adaptive Systems [0.802904964931021]
  Online reinforcement learning (RL) is increasingly used for realizing adaptive systems in the presence of design time uncertainty. Deep RL gaining interest, the learned knowledge is no longer explicitly represented, but is represented as a neural network. XRL-DINE provides visual insights into why certain decisions were made at important time points.
  arXiv  Detail & Related papers  (2023-07-09T05:12:42Z)
• Making Offline RL Online: Collaborative World Models for Offline Visual Reinforcement Learning [93.99377042564919]
  This paper tries to build more flexible constraints for value estimation without impeding the exploration of potential advantages. The key idea is to leverage off-the-shelf RL simulators, which can be easily interacted with in an online manner, as the "test bed" for offline policies. We introduce CoWorld, a model-based RL approach that mitigates cross-domain discrepancies in state and reward spaces.
  arXiv  Detail & Related papers  (2023-05-24T15:45:35Z)
• On Transforming Reinforcement Learning by Transformer: The Development Trajectory [97.79247023389445]
  Transformer, originally devised for natural language processing, has also attested significant success in computer vision. We group existing developments in two categories: architecture enhancement and trajectory optimization. We examine the main applications of TRL in robotic manipulation, text-based games, navigation and autonomous driving.
  arXiv  Detail & Related papers  (2022-12-29T03:15:59Z)
• A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges [38.70863329476517]
  Reinforcement Learning (RL) is a popular machine learning paradigm where intelligent agents interact with the environment to fulfill a long-term goal. Despite the encouraging results achieved, the deep neural network-based backbone is widely deemed as a black box that impedes practitioners to trust and employ trained agents in realistic scenarios where high security and reliability are essential. To alleviate this issue, a large volume of literature devoted to shedding light on the inner workings of the intelligent agents has been proposed, by constructing intrinsic interpretability or post-hoc explainability.
  arXiv  Detail & Related papers  (2022-11-12T13:52:06Z)
• Agent-Controller Representations: Principled Offline RL with Rich Exogenous Information [49.06422815335159]
  Learning to control an agent from data collected offline is vital for real-world applications of reinforcement learning (RL) This paper introduces offline RL benchmarks offering the ability to study this problem. We find that contemporary representation learning techniques can fail on datasets where the noise is a complex and time dependent process.
  arXiv  Detail & Related papers  (2022-10-31T22:12:48Z)
• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems [92.73407630874841]
  Automated Reinforcement Learning (AutoRL) involves not only standard applications of AutoML but also includes additional challenges unique to RL. We provide a common taxonomy, discuss each area in detail and pose open problems which would be of interest to researchers going forward.
  arXiv  Detail & Related papers  (2022-01-11T12:41:43Z)
• Improving Reinforcement Learning with Human Assistance: An Argument for Human Subject Studies with HIPPO Gym [21.4215863934377]
  Reinforcement learning (RL) is a popular machine learning paradigm for game playing, robotics control, and other sequential decision tasks. This article introduces our new open-source RL framework, the Human Input Parsing Platform for Openai Gym (HIPPO Gym)
  arXiv  Detail & Related papers  (2021-02-02T12:56:02Z)
• Explainability in Deep Reinforcement Learning [68.8204255655161]
  We review recent works in the direction to attain Explainable Reinforcement Learning (XRL) In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
  arXiv  Detail & Related papers  (2020-08-15T10:11:42Z)
• EasyRL: A Simple and Extensible Reinforcement Learning Framework [3.2173369911280023]
  EasyRL provides an interactive graphical user interface for users to train and evaluate RL agents. EasyRL does not require programming knowledge for training and testing simple built-in RL agents. EasyRL also supports custom RL agents and environments, which can be highly beneficial for RL researchers in evaluating and comparing their RL models.
  arXiv  Detail & Related papers  (2020-08-04T17:02:56Z)
• Explainable Reinforcement Learning: A Survey [0.0]
  Explainable Artificial Intelligence (XAI) has gained increased traction over the last few years. XAI models exhibit one detrimential characteristic: a performance-transparency trade-off. This survey attempts to address this gap by offering an overview of Explainable Reinforcement Learning (XRL) methods.
  arXiv  Detail & Related papers  (2020-05-13T10:52:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.