May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainability

• URL: http://arxiv.org/abs/2309.13965v2
• Date: Mon, 12 Aug 2024 08:34:15 GMT
• Title: May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainability
• Authors: Tong Zhang, X. Jessie Yang, Boyang Li,
• Abstract summary: We investigate if free-form conversations can enhance users' comprehension of static explanations. We measure the effect of the conversation on participants' ability to choose from three machine learning models. Our findings highlight the importance of customized model explanations in the format of free-form conversations.
• Score: 17.052366688978935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Research in explainable AI (XAI) aims to provide insights into the decision-making process of opaque AI models. To date, most XAI methods offer one-off and static explanations, which cannot cater to the diverse backgrounds and understanding levels of users. With this paper, we investigate if free-form conversations can enhance users' comprehension of static explanations, improve acceptance and trust in the explanation methods, and facilitate human-AI collaboration. Participants are presented with static explanations, followed by a conversation with a human expert regarding the explanations. We measure the effect of the conversation on participants' ability to choose, from three machine learning models, the most accurate one based on explanations and their self-reported comprehension, acceptance, and trust. Empirical results show that conversations significantly improve comprehension, acceptance, trust, and collaboration. Our findings highlight the importance of customized model explanations in the format of free-form conversations and provide insights for the future design of conversational explanations.

Related papers

• Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting [43.110187812734864]
  We evaluate three types of explanations: visual explanations (saliency maps), natural language explanations, and a combination of both modalities. We find that text-based explanations lead to significant over-reliance, which is alleviated by combining them with saliency maps. We also observe that the quality of explanations, that is, how much factually correct information they entail, and how much this aligns with AI correctness, significantly impacts the usefulness of the different explanation types.
  arXiv  Detail & Related papers  (2024-10-16T06:43:02Z)
• Tell me more: Intent Fulfilment Framework for Enhancing User Experiences in Conversational XAI [0.6333053895057925]
  This paper explores how different types of explanations collaboratively meet users' XAI needs. We introduce the Intent Fulfilment Framework (IFF) for creating explanation experiences. The Explanation Experience Dialogue Model integrates the IFF and "Explanation Followups" to provide users with a conversational interface.
  arXiv  Detail & Related papers  (2024-05-16T21:13:43Z)
• Explaining Explainability: Towards Deeper Actionable Insights into Deep Learning through Second-order Explainability [70.60433013657693]
  Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level. We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
  arXiv  Detail & Related papers  (2023-06-14T23:24:01Z)
• Behaviour Trees for Conversational Explanation Experiences [1.5257668132713955]
  This paper focuses on how users interact with an XAI system to fulfil multiple explanation needs satisfied by an explanation strategy. We model the interactive explanation experience as a dialogue model. An evaluation with a real-world use case shows that BTs have a number of properties that lend naturally to modelling and capturing explanation experiences.
  arXiv  Detail & Related papers  (2022-11-11T18:39:38Z)
• Towards Human Cognition Level-based Experiment Design for Counterfactual Explanations (XAI) [68.8204255655161]
  The emphasis of XAI research appears to have turned to a more pragmatic explanation approach for better understanding. An extensive area where cognitive science research may substantially influence XAI advancements is evaluating user knowledge and feedback. We propose a framework to experiment with generating and evaluating the explanations on the grounds of different cognitive levels of understanding.
  arXiv  Detail & Related papers  (2022-10-31T19:20:22Z)
• From Black Boxes to Conversations: Incorporating XAI in a Conversational Agent [2.899704155417792]
  Social science research states that explanations should be conversational, similar to human-to-human explanations. We show how to incorporate XAI in a conversational agent, using a standard design for the agent. We build upon an XAI question bank, which we extend by quality-controlled paraphrases, to understand the user's information needs.
  arXiv  Detail & Related papers  (2022-09-06T15:01:06Z)
• Rethinking Explainability as a Dialogue: A Practitioner's Perspective [57.87089539718344]
  We ask doctors, healthcare professionals, and policymakers about their needs and desires for explanations. Our study indicates that decision-makers would strongly prefer interactive explanations in the form of natural language dialogues. Considering these needs, we outline a set of five principles researchers should follow when designing interactive explanations.
  arXiv  Detail & Related papers  (2022-02-03T22:17:21Z)
• Human Interpretation of Saliency-based Explanation Over Text [65.29015910991261]
  We study saliency-based explanations over textual data. We find that people often mis-interpret the explanations. We propose a method to adjust saliencies based on model estimates of over- and under-perception.
  arXiv  Detail & Related papers  (2022-01-27T15:20:32Z)
• Explanation as a process: user-centric construction of multi-level and multi-modal explanations [0.34410212782758043]
  We present a process-based approach that combines multi-level and multi-modal explanations. We use Inductive Logic Programming, an interpretable machine learning approach, to learn a comprehensible model.
  arXiv  Detail & Related papers  (2021-10-07T19:26:21Z)
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
  We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
  arXiv  Detail & Related papers  (2021-07-28T17:32:04Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.