Can Conversational XAI Improve User Performance? An Experimental Study

• URL: http://arxiv.org/abs/2605.20439v1
• Date: Tue, 19 May 2026 19:47:17 GMT
• Title: Can Conversational XAI Improve User Performance? An Experimental Study
• Abstract summary: We propose an experimental design for evaluating explanation assistance through prediction accuracy, model understanding, and error identification.<n>Using an explainable-by-design prediction model, we create conditions where users can outperform the model by identifying and compensating for systematic errors.<n>Preliminary results from testing our experimental design show that participants significantly outperformed the model but reveal no performance differences between assistance types and modest engagement overall.
• Score: 3.9766661294245593
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Explainable AI (XAI) techniques aim to provide insights into predictive models and enhance user performance, yet they often fall short of these expectations. Conversational XAI assistants promise to overcome such limitations, but empirical evidence on their impact on objective performance measures remains limited. We propose an experimental design for evaluating explanation assistance through prediction accuracy, model understanding, and error identification. Using an explainable-by-design prediction model, we create conditions where users can outperform the model by identifying and compensating for systematic errors. We compare conversational assistance against Q&A-based assistance to assess which better supports users in working with model explanations. Preliminary results from testing our experimental design show that participants (N=42) in both treatments significantly outperformed the model but reveal no performance differences between assistance types and modest engagement overall. These findings inform refinements for our planned full study, including enhanced engagement interventions and investigation of the mechanisms driving improved predictions.

Related papers

• Are Concept Bottleneck Models Effective as Decision-Support Systems? [48.16053753967562]
  Concept Bottleneck Models (CBMs) are interpretable-by-design neural networks that detect human-understandable concepts from the input and use them to generate predictions.<n>We present two user studies evaluating how concept-based explanations and user interventions on the model's concepts affect the performance of the human-AI team.<n>Our results show that CBMs can improve human-AI team accuracy relative to both unaided human performance and performance with non-interpretable AI support.
  arXiv  Detail & Related papers  (2026-08-26T09:42:44Z)
• Using LLMs to Directly Guess Conditional Expectations Can Improve Efficiency in Causal Estimation [0.3222802562733787]
  We show that predictions made by generative models trained on historical data can be used to improve the performance of these estimators.<n>We consider a case study using a small dataset of online jewelry auctions, and demonstrate that inclusion of LLM-generated guesses as predictors can improve efficiency in estimation.
  arXiv  Detail & Related papers  (2025-10-09T03:34:06Z)
• Causality can systematically address the monsters under the bench(marks) [64.36592889550431]
  Benchmarks are plagued by various biases, artifacts, or leakage.<n>Models may behave unreliably due to poorly explored failure modes.<n> causality offers an ideal framework to systematically address these challenges.
  arXiv  Detail & Related papers  (2025-02-07T17:01:37Z)
• Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance [61.06245197347139]
  We propose a novel approach to explain the behavior of a black-box model under feature shifts. We refer to our method that combines concepts from Optimal Transport and Shapley Values as Explanatory Performance Estimation.
  arXiv  Detail & Related papers  (2024-08-24T18:28:19Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis [6.606409729669314]
  Post-hoc explainability methods aim to clarify predictions of black-box machine learning models. We conduct a user study to evaluate comprehensibility and predictability in two widely used tools: LIME and SHAP. We find that the comprehensibility of SHAP is significantly reduced when explanations are provided for samples near a model's decision boundary.
  arXiv  Detail & Related papers  (2023-09-21T11:54:20Z)
• A Comprehensive Evaluation and Analysis Study for Chinese Spelling Check [53.152011258252315]
  We show that using phonetic and graphic information reasonably is effective for Chinese Spelling Check. Models are sensitive to the error distribution of the test set, which reflects the shortcomings of models. The commonly used benchmark, SIGHAN, can not reliably evaluate models' performance.
  arXiv  Detail & Related papers  (2023-07-25T17:02:38Z)
• Counterfactuals of Counterfactuals: a back-translation-inspired approach to analyse counterfactual editors [3.4253416336476246]
  We focus on the analysis of counterfactual, contrastive explanations. We propose a new back translation-inspired evaluation methodology. We show that by iteratively feeding the counterfactual to the explainer we can obtain valuable insights into the behaviour of both the predictor and the explainer models.
  arXiv  Detail & Related papers  (2023-05-26T16:04:28Z)
• Clinical outcome prediction under hypothetical interventions -- a representation learning framework for counterfactual reasoning [31.97813934144506]
  We introduce a new representation learning framework, which considers the provision of counterfactual explanations as an embedded property of the risk model. Our results suggest that our proposed framework has the potential to help researchers and clinicians improve personalised care.
  arXiv  Detail & Related papers  (2022-05-15T09:41:16Z)
• Pathologies of Pre-trained Language Models in Few-shot Fine-tuning [50.3686606679048]
  We show that pre-trained language models with few examples show strong prediction bias across labels. Although few-shot fine-tuning can mitigate the prediction bias, our analysis shows models gain performance improvement by capturing non-task-related features. These observations alert that pursuing model performance with fewer examples may incur pathological prediction behavior.
  arXiv  Detail & Related papers  (2022-04-17T15:55:18Z)
• Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations [97.91630330328815]
  We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
  arXiv  Detail & Related papers  (2021-12-17T18:29:56Z)
• A Study on Multimodal and Interactive Explanations for Visual Question Answering [3.086885687016963]
  We evaluate multimodal explanations in the setting of a Visual Question Answering (VQA) task. Results indicate that the explanations help improve human prediction accuracy, especially in trials when the VQA system's answer is inaccurate. We introduce active attention, a novel method for evaluating causal attentional effects through intervention by editing attention maps.
  arXiv  Detail & Related papers  (2020-03-01T07:54:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.