From Black Boxes to Conversations: Incorporating XAI in a Conversational Agent

• URL: http://arxiv.org/abs/2209.02552v3
• Date: Mon, 22 Jul 2024 09:10:34 GMT
• Title: From Black Boxes to Conversations: Incorporating XAI in a Conversational Agent
• Authors: Van Bach Nguyen, Jörg Schlötterer, Christin Seifert,
• Abstract summary: Social science research states that explanations should be conversational, similar to human-to-human explanations. We show how to incorporate XAI in a conversational agent, using a standard design for the agent. We build upon an XAI question bank, which we extend by quality-controlled paraphrases, to understand the user's information needs.
• Score: 2.899704155417792
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The goal of Explainable AI (XAI) is to design methods to provide insights into the reasoning process of black-box models, such as deep neural networks, in order to explain them to humans. Social science research states that such explanations should be conversational, similar to human-to-human explanations. In this work, we show how to incorporate XAI in a conversational agent, using a standard design for the agent comprising natural language understanding and generation components. We build upon an XAI question bank, which we extend by quality-controlled paraphrases, to understand the user's information needs. We further systematically survey the literature for suitable explanation methods that provide the information to answer those questions, and present a comprehensive list of suggestions. Our work is the first step towards truly natural conversations about machine learning models with an explanation agent. The comprehensive list of XAI questions and the corresponding explanation methods may support other researchers in providing the necessary information to address users' demands. To facilitate future work, we release our source code and data.

Related papers

• May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainability [17.052366688978935]
  We investigate if free-form conversations can enhance users' comprehension of static explanations. We measure the effect of the conversation on participants' ability to choose from three machine learning models. Our findings highlight the importance of customized model explanations in the format of free-form conversations.
  arXiv  Detail & Related papers  (2023-09-25T09:00:38Z)
• Towards Reconciling Usability and Usefulness of Explainable AI Methodologies [2.715884199292287]
  Black-box AI systems can lead to liability and accountability issues when they produce an incorrect decision. Explainable AI (XAI) seeks to bridge the knowledge gap, between developers and end-users.
  arXiv  Detail & Related papers  (2023-01-13T01:08:49Z)
• Behaviour Trees for Conversational Explanation Experiences [1.5257668132713955]
  This paper focuses on how users interact with an XAI system to fulfil multiple explanation needs satisfied by an explanation strategy. We model the interactive explanation experience as a dialogue model. An evaluation with a real-world use case shows that BTs have a number of properties that lend naturally to modelling and capturing explanation experiences.
  arXiv  Detail & Related papers  (2022-11-11T18:39:38Z)
• Towards Human Cognition Level-based Experiment Design for Counterfactual Explanations (XAI) [68.8204255655161]
  The emphasis of XAI research appears to have turned to a more pragmatic explanation approach for better understanding. An extensive area where cognitive science research may substantially influence XAI advancements is evaluating user knowledge and feedback. We propose a framework to experiment with generating and evaluating the explanations on the grounds of different cognitive levels of understanding.
  arXiv  Detail & Related papers  (2022-10-31T19:20:22Z)
• "Explanation" is Not a Technical Term: The Problem of Ambiguity in XAI [2.5899040911480173]
  We explore the features of explanations and how to use those features in evaluating their utility. We focus on the requirements for explanations defined by their functional role, the knowledge states of users who are trying to understand them, and the availability of the information needed to generate them.
  arXiv  Detail & Related papers  (2022-06-27T21:42:53Z)
• Rethinking Explainability as a Dialogue: A Practitioner's Perspective [57.87089539718344]
  We ask doctors, healthcare professionals, and policymakers about their needs and desires for explanations. Our study indicates that decision-makers would strongly prefer interactive explanations in the form of natural language dialogues. Considering these needs, we outline a set of five principles researchers should follow when designing interactive explanations.
  arXiv  Detail & Related papers  (2022-02-03T22:17:21Z)
• Diagnosing AI Explanation Methods with Folk Concepts of Behavior [70.10183435379162]
  We consider "success" to depend not only on what information the explanation contains, but also on what information the human explainee understands from it. We use folk concepts of behavior as a framework of social attribution by the human explainee.
  arXiv  Detail & Related papers  (2022-01-27T00:19:41Z)
• CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models [84.32751938563426]
  We propose a new explainable AI (XAI) framework for explaining decisions made by a deep convolutional neural network (CNN) In contrast to the current methods in XAI that generate explanations as a single shot response, we pose explanation as an iterative communication process. Our framework generates sequence of explanations in a dialog by mediating the differences between the minds of machine and human user.
  arXiv  Detail & Related papers  (2021-09-03T09:46:20Z)
• Explainability in Deep Reinforcement Learning [68.8204255655161]
  We review recent works in the direction to attain Explainable Reinforcement Learning (XRL) In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
  arXiv  Detail & Related papers  (2020-08-15T10:11:42Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)
• Questioning the AI: Informing Design Practices for Explainable AI User Experiences [33.81809180549226]
  A surge of interest in explainable AI (XAI) has led to a vast collection of algorithmic work on the topic. We seek to identify gaps between the current XAI algorithmic work and practices to create explainable AI products. We develop an algorithm-informed XAI question bank in which user needs for explainability are represented.
  arXiv  Detail & Related papers  (2020-01-08T12:34:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.