Seamful XAI: Operationalizing Seamful Design in Explainable AI
- URL: http://arxiv.org/abs/2211.06753v2
- Date: Tue, 5 Mar 2024 19:55:14 GMT
- Title: Seamful XAI: Operationalizing Seamful Design in Explainable AI
- Authors: Upol Ehsan, Q. Vera Liao, Samir Passi, Mark O. Riedl, Hal Daume III
- Abstract summary: Mistakes in AI systems are inevitable, arising from both technical limitations and sociotechnical gaps.
We propose that seamful design can foster AI explainability by revealing sociotechnical and infrastructural mismatches.
We explore this process with 43 AI practitioners and real end-users.
- Score: 59.89011292395202
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Mistakes in AI systems are inevitable, arising from both technical
limitations and sociotechnical gaps. While black-boxing AI systems can make the
user experience seamless, hiding the seams risks disempowering users to
mitigate fallouts from AI mistakes. Instead of hiding these AI imperfections,
can we leverage them to help the user? While Explainable AI (XAI) has
predominantly tackled algorithmic opaqueness, we propose that seamful design
can foster AI explainability by revealing and leveraging sociotechnical and
infrastructural mismatches. We introduce the concept of Seamful XAI by (1)
conceptually transferring "seams" to the AI context and (2) developing a design
process that helps stakeholders anticipate and design with seams. We explore
this process with 43 AI practitioners and real end-users, using a
scenario-based co-design activity informed by real-world use cases. We found
that the Seamful XAI design process helped users foresee AI harms, identify
underlying reasons (seams), locate them in the AI's lifecycle, learn how to
leverage seamful information to improve XAI and user agency. We share empirical
insights, implications, and reflections on how this process can help
practitioners anticipate and craft seams in AI, how seamfulness can improve
explainability, empower end-users, and facilitate Responsible AI.
Related papers
- Investigating the Role of Explainability and AI Literacy in User Compliance [2.8623940003518156]
We find that users' compliance increases with the introduction of XAI but is also affected by AI literacy.
We also find that the relationships between AI literacy XAI and users' compliance are mediated by the users' mental model of AI.
arXiv Detail & Related papers (2024-06-18T14:28:12Z) - Towards Reconciling Usability and Usefulness of Explainable AI
Methodologies [2.715884199292287]
Black-box AI systems can lead to liability and accountability issues when they produce an incorrect decision.
Explainable AI (XAI) seeks to bridge the knowledge gap, between developers and end-users.
arXiv Detail & Related papers (2023-01-13T01:08:49Z) - What Do End-Users Really Want? Investigation of Human-Centered XAI for
Mobile Health Apps [69.53730499849023]
We present a user-centered persona concept to evaluate explainable AI (XAI)
Results show that users' demographics and personality, as well as the type of explanation, impact explanation preferences.
Our insights bring an interactive, human-centered XAI closer to practical application.
arXiv Detail & Related papers (2022-10-07T12:51:27Z) - Transcending XAI Algorithm Boundaries through End-User-Inspired Design [27.864338632191608]
Lacking explainability-focused functional support for end users may hinder the safe and responsible use of AI in high-stakes domains.
Our work shows that grounding the technical problem in end users' use of XAI can inspire new research questions.
Such end-user-inspired research questions have the potential to promote social good by democratizing AI and ensuring the responsible use of AI in critical domains.
arXiv Detail & Related papers (2022-08-18T09:44:51Z) - Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations.
It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
arXiv Detail & Related papers (2022-01-26T18:53:09Z) - On Two XAI Cultures: A Case Study of Non-technical Explanations in
Deployed AI System [3.4918511133757977]
Not much of XAI is comprehensible to non-AI experts, who nonetheless are the primary audience and major stakeholders of deployed AI systems in practice.
We advocate that it is critical to develop XAI methods for non-technical audiences.
We then present a real-life case study, where AI experts provided non-technical explanations of AI decisions to non-technical stakeholders.
arXiv Detail & Related papers (2021-12-02T07:02:27Z) - A User-Centred Framework for Explainable Artificial Intelligence in
Human-Robot Interaction [70.11080854486953]
We propose a user-centred framework for XAI that focuses on its social-interactive aspect.
The framework aims to provide a structure for interactive XAI solutions thought for non-expert users.
arXiv Detail & Related papers (2021-09-27T09:56:23Z) - The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations.
We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
arXiv Detail & Related papers (2021-07-28T17:32:04Z) - Building Bridges: Generative Artworks to Explore AI Ethics [56.058588908294446]
In recent years, there has been an increased emphasis on understanding and mitigating adverse impacts of artificial intelligence (AI) technologies on society.
A significant challenge in the design of ethical AI systems is that there are multiple stakeholders in the AI pipeline, each with their own set of constraints and interests.
This position paper outlines some potential ways in which generative artworks can play this role by serving as accessible and powerful educational tools.
arXiv Detail & Related papers (2021-06-25T22:31:55Z) - Questioning the AI: Informing Design Practices for Explainable AI User
Experiences [33.81809180549226]
A surge of interest in explainable AI (XAI) has led to a vast collection of algorithmic work on the topic.
We seek to identify gaps between the current XAI algorithmic work and practices to create explainable AI products.
We develop an algorithm-informed XAI question bank in which user needs for explainability are represented.
arXiv Detail & Related papers (2020-01-08T12:34:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.