Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representations

• URL: http://arxiv.org/abs/2002.01080v4
• Date: Sat, 19 Mar 2022 22:47:40 GMT
• Title: Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representations
• Authors: Sarath Sreedharan, Utkarsh Soni, Mudit Verma, Siddharth Srivastava, Subbarao Kambhampati
• Abstract summary: This paper introduces methods for providing contrastive explanations in terms of user-specified concepts for sequential decision-making settings. We do this by building partial symbolic models of a local approximation of the task that can be leveraged to answer the user queries.
• Score: 44.016120003139264
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As increasingly complex AI systems are introduced into our daily lives, it becomes important for such systems to be capable of explaining the rationale for their decisions and allowing users to contest these decisions. A significant hurdle to allowing for such explanatory dialogue could be the vocabulary mismatch between the user and the AI system. This paper introduces methods for providing contrastive explanations in terms of user-specified concepts for sequential decision-making settings where the system's model of the task may be best represented as an inscrutable model. We do this by building partial symbolic models of a local approximation of the task that can be leveraged to answer the user queries. We test these methods on a popular Atari game (Montezuma's Revenge) and variants of Sokoban (a well-known planning benchmark) and report the results of user studies to evaluate whether people find explanations generated in this form useful.

Related papers

• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Explainable Data-Driven Optimization: From Context to Decision and Back Again [76.84947521482631]
  Data-driven optimization uses contextual information and machine learning algorithms to find solutions to decision problems with uncertain parameters. We introduce a counterfactual explanation methodology tailored to explain solutions to data-driven problems. We demonstrate our approach by explaining key problems in operations management such as inventory management and routing.
  arXiv  Detail & Related papers  (2023-01-24T15:25:16Z)
• Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems [109.16553492049441]
  We propose a novel method to incorporate the knowledge reasoning capability into dialogue systems in a more scalable and generalizable manner. To the best of our knowledge, this is the first work to have transformer models generate responses by reasoning over differentiable knowledge graphs.
  arXiv  Detail & Related papers  (2022-03-20T17:51:49Z)
• SMACE: A New Method for the Interpretability of Composite Decision Systems [3.1542695050861544]
  We present SMACE, Semi-Model-Agnostic Contextual Explainer, a novel interpretability method for decision systems. We show that established model-agnostic approaches produce poor results in this framework.
  arXiv  Detail & Related papers  (2021-11-16T19:37:35Z)
• Learning User-Interpretable Descriptions of Black-Box AI System Capabilities [9.608555640607731]
  This paper presents an approach for learning user-interpretable symbolic descriptions of the limits and capabilities of a black-box AI system. It uses a hierarchical active querying paradigm to generate questions and to learn a user-interpretable model of the AI system based on its responses.
  arXiv  Detail & Related papers  (2021-07-28T23:33:31Z)
• Dissecting Generation Modes for Abstractive Summarization Models via Ablation and Attribution [34.2658286826597]
  We propose a two-step method to interpret summarization model decisions. We first analyze the model's behavior by ablating the full model to categorize each decoder decision into one of several generation modes. After isolating decisions that do depend on the input, we explore interpreting these decisions using several different attribution methods.
  arXiv  Detail & Related papers  (2021-06-03T00:54:16Z)
• This is not the Texture you are looking for! Introducing Novel Counterfactual Explanations for Non-Experts using Generative Adversarial Learning [59.17685450892182]
  counterfactual explanation systems try to enable a counterfactual reasoning by modifying the input image. We present a novel approach to generate such counterfactual image explanations based on adversarial image-to-image translation techniques. Our results show that our approach leads to significantly better results regarding mental models, explanation satisfaction, trust, emotions, and self-efficacy than two state-of-the art systems.
  arXiv  Detail & Related papers  (2020-12-22T10:08:05Z)
• Altruist: Argumentative Explanations through Local Interpretations of Predictive Models [10.342433824178825]
  Existing explanation techniques are often not comprehensible to the end user. We introduce a preliminary meta-explanation methodology that identifies the truthful parts of feature importance oriented interpretations. Experimentation strongly indicates that an ensemble of multiple interpretation techniques yields considerably more truthful explanations.
  arXiv  Detail & Related papers  (2020-10-15T10:36:48Z)
• Explainable Recommender Systems via Resolving Learning Representations [57.24565012731325]
  Explanations could help improve user experience and discover system defects. We propose a novel explainable recommendation model through improving the transparency of the representation learning process.
  arXiv  Detail & Related papers  (2020-08-21T05:30:48Z)
• DECE: Decision Explorer with Counterfactual Explanations for Machine Learning Models [36.50754934147469]
  We exploit the potential of counterfactual explanations to understand and explore the behavior of machine learning models. We design DECE, an interactive visualization system that helps understand and explore a model's decisions on individual instances and data subsets.
  arXiv  Detail & Related papers  (2020-08-19T09:44:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.