Why the Agent Made that Decision: Explaining Deep Reinforcement Learning with Vision Masks

• URL: http://arxiv.org/abs/2411.16120v1
• Date: Mon, 25 Nov 2024 06:11:46 GMT
• Title: Why the Agent Made that Decision: Explaining Deep Reinforcement Learning with Vision Masks
• Authors: Rui Zuo, Zifan Wang, Simon Khan, Garrett Ethan Katz, Qinru Qiu,
• Abstract summary: VisionMask is a standalone explanation model trained end-to-end to identify the most critical regions in the agent's visual input that can explain its actions. It achieves a 14.9% higher insertion accuracy and a 30.08% higher F1-Score in reproducing original actions from selected visual explanations.
• Score: 11.068220265247385
• License:
• Abstract: Due to the inherent lack of transparency in deep neural networks, it is challenging for deep reinforcement learning (DRL) agents to gain trust and acceptance from users, especially in safety-critical applications such as medical diagnosis and military operations. Existing methods for explaining an agent's decision either require to retrain the agent using models that support explanation generation or rely on perturbation-based techniques to reveal the significance of different input features in the decision making process. However, retraining the agent may compromise its integrity and performance, while perturbation-based methods have limited performance and lack knowledge accumulation or learning capabilities. Moreover, since each perturbation is performed independently, the joint state of the perturbed inputs may not be physically meaningful. To address these challenges, we introduce $\textbf{VisionMask}$, a standalone explanation model trained end-to-end to identify the most critical regions in the agent's visual input that can explain its actions. VisionMask is trained in a self-supervised manner without relying on human-generated labels. Importantly, its training does not alter the agent model, hence preserving the agent's performance and integrity. We evaluate VisionMask on Super Mario Bros (SMB) and three Atari games. Compared to existing methods, VisionMask achieves a 14.9% higher insertion accuracy and a 30.08% higher F1-Score in reproducing original actions from the selected visual explanations. We also present examples illustrating how VisionMask can be used for counterfactual analysis.

Related papers

• MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learning [40.7452827298478]
  We introduce MaDi, a novel algorithm that learns to mask distractions by the reward signal only. In MaDi, the conventional actor-critic structure of deep reinforcement learning agents is complemented by a small third sibling, the Masker. Our algorithm improves the agent's focus with useful masks, while its efficient Masker network only adds 0.2% more parameters to the original structure.
  arXiv  Detail & Related papers  (2023-12-23T20:11:05Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• MA2CL:Masked Attentive Contrastive Learning for Multi-Agent Reinforcement Learning [128.19212716007794]
  We propose an effective framework called textbfMulti-textbfAgent textbfMasked textbfAttentive textbfContrastive textbfLearning (MA2CL) MA2CL encourages learning representation to be both temporal and agent-level predictive by reconstructing the masked agent observation in latent space. Our method significantly improves the performance and sample efficiency of different MARL algorithms and outperforms other methods in various vision-based and state-based scenarios.
  arXiv  Detail & Related papers  (2023-06-03T05:32:19Z)
• Hard Patches Mining for Masked Image Modeling [52.46714618641274]
  Masked image modeling (MIM) has attracted much research attention due to its promising potential for learning scalable visual representations. We propose Hard Patches Mining (HPM), a brand-new framework for MIM pre-training.
  arXiv  Detail & Related papers  (2023-04-12T15:38:23Z)
• Masked Autoencoding for Scalable and Generalizable Decision Making [93.84855114717062]
  MaskDP is a simple and scalable self-supervised pretraining method for reinforcement learning and behavioral cloning. We find that a MaskDP model gains the capability of zero-shot transfer to new BC tasks, such as single and multiple goal reaching.
  arXiv  Detail & Related papers  (2022-11-23T07:04:41Z)
• Exploring Target Representations for Masked Autoencoders [78.57196600585462]
  We show that a careful choice of the target representation is unnecessary for learning good representations. We propose a multi-stage masked distillation pipeline and use a randomly model as the teacher. A proposed method to perform masked knowledge distillation with bootstrapped teachers (dBOT) outperforms previous self-supervised methods by nontrivial margins.
  arXiv  Detail & Related papers  (2022-09-08T16:55:19Z)
• Self-supervised Transformer for Deepfake Detection [112.81127845409002]
  Deepfake techniques in real-world scenarios require stronger generalization abilities of face forgery detectors. Inspired by transfer learning, neural networks pre-trained on other large-scale face-related tasks may provide useful features for deepfake detection. In this paper, we propose a self-supervised transformer based audio-visual contrastive learning method.
  arXiv  Detail & Related papers  (2022-03-02T17:44:40Z)
• Mask or Non-Mask? Robust Face Mask Detector via Triplet-Consistency Representation Learning [23.062034116854875]
  In the absence of vaccines or medicines to stop COVID-19, one of the effective methods to slow the spread of the coronavirus is to wear a face mask. To mandate the use of face masks or coverings in public areas, additional human resources are required, which is tedious and attention-intensive. We propose a face mask detection framework that uses the context attention module to enable the effective attention of the feed-forward convolution neural network.
  arXiv  Detail & Related papers  (2021-10-01T16:44:06Z)
• Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning [9.49864824780503]
  We propose Mask-Attention A3C (Mask A3C), which introduces an attention mechanism into Asynchronous Advantage Actor-Critic (A3C) A3C consists of a feature extractor that extracts features from an image, a policy branch that outputs the policy, and a value branch that outputs the state value. We visualized mask-attention maps for games on the Atari 2600 and found we could easily analyze the reasons behind an agent's decision-making.
  arXiv  Detail & Related papers  (2021-03-06T08:38:12Z)
• Self-Supervised Discovering of Interpretable Features for Reinforcement Learning [40.52278913726904]
  We propose a self-supervised interpretable framework for deep reinforcement learning. A self-supervised interpretable network (SSINet) is employed to produce fine-grained attention masks for highlighting task-relevant information. We verify and evaluate our method on several Atari 2600 games as well as Duckietown, which is a challenging self-driving car simulator environment.
  arXiv  Detail & Related papers  (2020-03-16T08:26:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.