Information That Matters: Exploring Information Needs of People Affected
by Algorithmic Decisions
- URL: http://arxiv.org/abs/2401.13324v4
- Date: Mon, 29 Jan 2024 08:52:18 GMT
- Title: Information That Matters: Exploring Information Needs of People Affected
by Algorithmic Decisions
- Authors: Timoth\'ee Schmude, Laura Koesten, Torsten M\"oller, Sebastian
Tschiatschek
- Abstract summary: Explanations of AI systems rarely address the information needs of people affected by algorithmic decision-making (ADM)
We present the "XAI Novice Question Bank": A catalog of affected stakeholders' information needs in two ADM use cases.
- Score: 9.15830544182374
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Explanations of AI systems rarely address the information needs of people
affected by algorithmic decision-making (ADM). This gap between conveyed
information and information that matters to affected stakeholders can impede
understanding and adherence to regulatory frameworks such as the AI Act. To
address this gap, we present the "XAI Novice Question Bank": A catalog of
affected stakeholders' information needs in two ADM use cases (employment
prediction and health monitoring), covering the categories data, system
context, system usage, and system specifications. Information needs were
gathered in an interview study where participants received explanations in
response to their inquiries. Participants further reported their understanding
and decision confidence, showing that while confidence tended to increase after
receiving explanations, participants also met understanding challenges, such as
being unable to tell why their understanding felt incomplete. Explanations
further influenced participants' perceptions of the systems' risks and
benefits, which they confirmed or changed depending on the use case. When risks
were perceived as high, participants expressed particular interest in
explanations about intention, such as why and to what end a system was put in
place. With this work, we aim to support the inclusion of affected stakeholders
into explainability by contributing an overview of information and challenges
relevant to them when deciding on the adoption of ADM systems. We close by
summarizing our findings in a list of six key implications that inform the
design of future explanations for affected stakeholder audiences.
Related papers
- ABI Approach: Automatic Bias Identification in Decision-Making Under Risk based in an Ontology of Behavioral Economics [46.57327530703435]
Risk seeking preferences for losses, driven by biases such as loss aversion, pose challenges and can result in severe negative consequences.
This research introduces the ABI approach, a novel solution designed to support organizational decision-makers by automatically identifying and explaining risk seeking preferences.
arXiv Detail & Related papers (2024-05-22T23:53:46Z) - Sociotechnical Implications of Generative Artificial Intelligence for Information Access [4.3867169221012645]
Generative AI technologies may enable new ways to access information and improve effectiveness of existing information retrieval systems.
We present an overview of some of the systemic consequences and risks of employing generative AI in the context of information access.
arXiv Detail & Related papers (2024-05-19T17:04:39Z) - PAQA: Toward ProActive Open-Retrieval Question Answering [34.883834970415734]
This work aims to tackle the challenge of generating relevant clarifying questions by taking into account the inherent ambiguities present in both user queries and documents.
We propose PAQA, an extension to the existing AmbiNQ dataset, incorporating clarifying questions.
We then evaluate various models and assess how passage retrieval impacts ambiguity detection and the generation of clarifying questions.
arXiv Detail & Related papers (2024-02-26T14:40:34Z) - Notion of Explainable Artificial Intelligence -- An Empirical
Investigation from A Users Perspective [0.3069335774032178]
This study aims to investigate usercentric explainable AI and considered recommendation systems as the study context.
We conducted focus group interviews to collect qualitative data on the recommendation system.
Our findings reveal that end users want a non-technical and tailor-made explanation with on-demand supplementary information.
arXiv Detail & Related papers (2023-11-01T22:20:14Z) - ChoiceMates: Supporting Unfamiliar Online Decision-Making with
Multi-Agent Conversational Interactions [58.71970923420007]
We present ChoiceMates, a system that enables conversations with a dynamic set of LLM-powered agents.
Agents, as opinionated personas, flexibly join the conversation, not only providing responses but also conversing among themselves to elicit each agent's preferences.
Our study (n=36) comparing ChoiceMates to conventional web search and single-agent showed that ChoiceMates was more helpful in discovering, diving deeper, and managing information compared to Web with higher confidence.
arXiv Detail & Related papers (2023-10-02T16:49:39Z) - Requirements for Explainability and Acceptance of Artificial
Intelligence in Collaborative Work [0.0]
The present structured literature analysis examines the requirements for the explainability and acceptance of AI.
Results indicate that the two main groups of users are developers who require information about the internal operations of the model.
The acceptance of AI systems depends on information about the system's functions and performance, privacy and ethical considerations.
arXiv Detail & Related papers (2023-06-27T11:36:07Z) - "There Is Not Enough Information": On the Effects of Explanations on
Perceptions of Informational Fairness and Trustworthiness in Automated
Decision-Making [0.0]
Automated decision systems (ADS) are increasingly used for consequential decision-making.
We conduct a human subject study to assess people's perceptions of informational fairness.
A comprehensive analysis of qualitative feedback sheds light on people's desiderata for explanations.
arXiv Detail & Related papers (2022-05-11T20:06:03Z) - Adaptive cognitive fit: Artificial intelligence augmented management of
information facets and representations [62.997667081978825]
Explosive growth in big data technologies and artificial intelligence [AI] applications have led to increasing pervasiveness of information facets.
Information facets, such as equivocality and veracity, can dominate and significantly influence human perceptions of information.
We suggest that artificially intelligent technologies that can adapt information representations to overcome cognitive limitations are necessary.
arXiv Detail & Related papers (2022-04-25T02:47:25Z) - Inverse Online Learning: Understanding Non-Stationary and Reactionary
Policies [79.60322329952453]
We show how to develop interpretable representations of how agents make decisions.
By understanding the decision-making processes underlying a set of observed trajectories, we cast the policy inference problem as the inverse to this online learning problem.
We introduce a practical algorithm for retrospectively estimating such perceived effects, alongside the process through which agents update them.
Through application to the analysis of UNOS organ donation acceptance decisions, we demonstrate that our approach can bring valuable insights into the factors that govern decision processes and how they change over time.
arXiv Detail & Related papers (2022-03-14T17:40:42Z) - Overcoming Failures of Imagination in AI Infused System Development and
Deployment [71.9309995623067]
NeurIPS 2020 requested that research paper submissions include impact statements on "potential nefarious uses and the consequences of failure"
We argue that frameworks of harms must be context-aware and consider a wider range of potential stakeholders, system affordances, as well as viable proxies for assessing harms in the widest sense.
arXiv Detail & Related papers (2020-11-26T18:09:52Z) - Uncertainty as a Form of Transparency: Measuring, Communicating, and
Using Uncertainty [66.17147341354577]
We argue for considering a complementary form of transparency by estimating and communicating the uncertainty associated with model predictions.
We describe how uncertainty can be used to mitigate model unfairness, augment decision-making, and build trustworthy systems.
This work constitutes an interdisciplinary review drawn from literature spanning machine learning, visualization/HCI, design, decision-making, and fairness.
arXiv Detail & Related papers (2020-11-15T17:26:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.