Human-centered explanation does not fit all: The interplay of sociotechnical, cognitive, and individual factors in the effect AI explanations in algorithmic decision-making

• URL: http://arxiv.org/abs/2502.12354v1
• Date: Mon, 17 Feb 2025 22:42:53 GMT
• Title: Human-centered explanation does not fit all: The interplay of sociotechnical, cognitive, and individual factors in the effect AI explanations in algorithmic decision-making
• Authors: Yongsu Ahn, Yu-Run Lin, Malihe Alikhani, Eunjeong Cheon,
• Abstract summary: This study focuses on the cognitive dimensions of explanation evaluation. We evaluate six explanations with different contrastive strategies and information selectivity. We call for a nuanced view of explanation strategies, with implications for designing AI interfaces.
• Score: 21.12778902817385
• License:
• Abstract: Recent XAI studies have investigated what constitutes a \textit{good} explanation in AI-assisted decision-making. Despite the widely accepted human-friendly properties of explanations, such as contrastive and selective, existing studies have yielded inconsistent findings. To address these gaps, our study focuses on the cognitive dimensions of explanation evaluation, by evaluating six explanations with different contrastive strategies and information selectivity and scrutinizing factors behind their valuation process. Our analysis results find that contrastive explanations are not the most preferable or understandable in general; Rather, different contrastive and selective explanations were appreciated to a different extent based on who they are, when, how, and what to explain -- with different level of cognitive load and engagement and sociotechnical contexts. Given these findings, we call for a nuanced view of explanation strategies, with implications for designing AI interfaces to accommodate individual and contextual differences in AI-assisted decision-making.

Related papers

• An Ontology-Enabled Approach For User-Centered and Knowledge-Enabled Explanations of AI Systems [0.3480973072524161]
  Recent research in explainability has focused on explaining the workings of AI models or model explainability. This thesis seeks to bridge some gaps between model and user-centered explainability.
  arXiv  Detail & Related papers  (2024-10-23T02:03:49Z)
• Contrastive Explanations That Anticipate Human Misconceptions Can Improve Human Decision-Making Skills [24.04643864795939]
  People's decision-making abilities often fail to improve when they rely on AI for decision-support. Most AI systems offer "unilateral" explanations that justify the AI's decision but do not account for users' thinking. We introduce a framework for generating human-centered contrastive explanations that explain the difference between AI's choice and a predicted, likely human choice.
  arXiv  Detail & Related papers  (2024-10-05T18:21:04Z)
• Combining AI Control Systems and Human Decision Support via Robustness and Criticality [53.10194953873209]
  We extend a methodology for adversarial explanations (AE) to state-of-the-art reinforcement learning frameworks. We show that the learned AI control system demonstrates robustness against adversarial tampering. In a training / learning framework, this technology can improve both the AI's decisions and explanations through human interaction.
  arXiv  Detail & Related papers  (2024-07-03T15:38:57Z)
• In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Making [25.18203172421461]
  We argue explanations are only useful to the extent that they allow a human decision maker to verify the correctness of an AI's prediction. We also compare the objective of complementary performance with that of appropriate reliance, decomposing the latter into the notions of outcome-graded and strategy-graded reliance.
  arXiv  Detail & Related papers  (2023-05-12T18:28:04Z)
• Selective Explanations: Leveraging Human Input to Align Explainable AI [40.33998268146951]
  We propose a general framework for generating selective explanations by leveraging human input on a small sample. As a showcase, we use a decision-support task to explore selective explanations based on what the decision-maker would consider relevant to the decision task. Our experiments demonstrate the promise of selective explanations in reducing over-reliance on AI.
  arXiv  Detail & Related papers  (2023-01-23T19:00:02Z)
• Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations [44.01143305912054]
  We study how decision-makers' intuition affects their use of AI predictions and explanations. Our results identify three types of intuition involved in reasoning about AI predictions and explanations. We use these pathways to explain why feature-based explanations did not improve participants' decision outcomes and increased their overreliance on AI.
  arXiv  Detail & Related papers  (2023-01-18T01:33:50Z)
• Towards Human Cognition Level-based Experiment Design for Counterfactual Explanations (XAI) [68.8204255655161]
  The emphasis of XAI research appears to have turned to a more pragmatic explanation approach for better understanding. An extensive area where cognitive science research may substantially influence XAI advancements is evaluating user knowledge and feedback. We propose a framework to experiment with generating and evaluating the explanations on the grounds of different cognitive levels of understanding.
  arXiv  Detail & Related papers  (2022-10-31T19:20:22Z)
• Human Interpretation of Saliency-based Explanation Over Text [65.29015910991261]
  We study saliency-based explanations over textual data. We find that people often mis-interpret the explanations. We propose a method to adjust saliencies based on model estimates of over- and under-perception.
  arXiv  Detail & Related papers  (2022-01-27T15:20:32Z)
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
  We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
  arXiv  Detail & Related papers  (2021-07-28T17:32:04Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.