Towards Explainable Goal Recognition Using Weight of Evidence (WoE): A Human-Centered Approach

• URL: http://arxiv.org/abs/2409.11675v1
• Date: Wed, 18 Sep 2024 03:30:01 GMT
• Title: Towards Explainable Goal Recognition Using Weight of Evidence (WoE): A Human-Centered Approach
• Authors: Abeer Alshehri, Amal Abdulrahman, Hajar Alamri, Tim Miller, Mor Vered,
• Abstract summary: Goal recognition (GR) involves inferring an agent's unobserved goal from a sequence of observations. Traditionally, GR has been addressed using 'inference to the best explanation' or abduction. We introduce and evaluate an explainable model for goal recognition (GR) agents, grounded in the theoretical framework and cognitive processes underlying human behavior explanation.
• Score: 5.174712539403376
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Goal recognition (GR) involves inferring an agent's unobserved goal from a sequence of observations. This is a critical problem in AI with diverse applications. Traditionally, GR has been addressed using 'inference to the best explanation' or abduction, where hypotheses about the agent's goals are generated as the most plausible explanations for observed behavior. Alternatively, some approaches enhance interpretability by ensuring that an agent's behavior aligns with an observer's expectations or by making the reasoning behind decisions more transparent. In this work, we tackle a different challenge: explaining the GR process in a way that is comprehensible to humans. We introduce and evaluate an explainable model for goal recognition (GR) agents, grounded in the theoretical framework and cognitive processes underlying human behavior explanation. Drawing on insights from two human-agent studies, we propose a conceptual framework for human-centered explanations of GR. Using this framework, we develop the eXplainable Goal Recognition (XGR) model, which generates explanations for both why and why not questions. We evaluate the model computationally across eight GR benchmarks and through three user studies. The first study assesses the efficiency of generating human-like explanations within the Sokoban game domain, the second examines perceived explainability in the same domain, and the third evaluates the model's effectiveness in aiding decision-making in illegal fishing detection. Results demonstrate that the XGR model significantly enhances user understanding, trust, and decision-making compared to baseline models, underscoring its potential to improve human-agent collaboration.

Related papers

• Model-Agnostic Policy Explanations with Large Language Models [6.405870799906393]
  We propose a method for generating natural language explanations of agent behavior based only on observed states and actions. Our approach learns a locally interpretable surrogate model of the agent's behavior from observations. We find that participants in a user study more accurately predicted the agent's future actions when given our explanations.
  arXiv  Detail & Related papers  (2025-04-08T02:56:02Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Understanding Your Agent: Leveraging Large Language Models for Behavior Explanation [7.647395374489533]
  We propose an approach to generate natural language explanations for an agent's behavior based only on observations of states and actions. We show that our approach generates explanations as helpful as those produced by a human domain expert.
  arXiv  Detail & Related papers  (2023-11-29T20:16:23Z)
• Interpretable Imitation Learning with Dynamic Causal Relations [65.18456572421702]
  We propose to expose captured knowledge in the form of a directed acyclic causal graph. We also design this causal discovery process to be state-dependent, enabling it to model the dynamics in latent causal graphs. The proposed framework is composed of three parts: a dynamic causal discovery module, a causality encoding module, and a prediction module, and is trained in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-09-30T20:59:42Z)
• Explaining Explainability: Towards Deeper Actionable Insights into Deep Learning through Second-order Explainability [70.60433013657693]
  Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level. We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
  arXiv  Detail & Related papers  (2023-06-14T23:24:01Z)
• Explainable Goal Recognition: A Framework Based on Weight of Evidence [9.356870107137093]
  We introduce and evaluate an eXplainable Goal Recognition (XGR) model that uses the Weight of Evidence (WoE) framework to explain goal recognition problems. Our model provides human-centered explanations that answer why? and why not? questions. Using a human behavioral study to obtain the ground truth from human annotators, we show that the XGR model can successfully generate human-like explanations.
  arXiv  Detail & Related papers  (2023-03-09T23:27:08Z)
• Machine Explanations and Human Understanding [31.047297225560566]
  Explanations are hypothesized to improve human understanding of machine learning models. empirical studies have found mixed and even negative results. We show how human intuitions play a central role in enabling human understanding.
  arXiv  Detail & Related papers  (2022-02-08T19:00:38Z)
• Local Explanation of Dialogue Response Generation [77.68077106724522]
  Local explanation of response generation (LERG) is proposed to gain insights into the reasoning process of a generation model. LERG views the sequence prediction as uncertainty estimation of a human response and then creates explanations by perturbing the input and calculating the certainty change over the human response. Our results show that our method consistently improves other widely used methods on proposed automatic- and human- evaluation metrics for this new task by 4.4-12.8%.
  arXiv  Detail & Related papers  (2021-06-11T17:58:36Z)
• Learning Domain Invariant Representations for Generalizable Person Re-Identification [71.35292121563491]
  Generalizable person Re-Identification (ReID) has attracted growing attention in recent computer vision community. We introduce causality into person ReID and propose a novel generalizable framework, named Domain Invariant Representations for generalizable person Re-Identification (DIR-ReID)
  arXiv  Detail & Related papers  (2021-03-29T18:59:48Z)
• Explainability in Deep Reinforcement Learning [68.8204255655161]
  We review recent works in the direction to attain Explainable Reinforcement Learning (XRL) In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
  arXiv  Detail & Related papers  (2020-08-15T10:11:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.