A User-Centric Analysis of Explainability in AI-Based Medical Image Diagnosis

• URL: http://arxiv.org/abs/2605.02903v1
• Date: Tue, 31 Mar 2026 09:42:38 GMT
• Title: A User-Centric Analysis of Explainability in AI-Based Medical Image Diagnosis
• Abstract summary: Our survey of 33 physicians showed that 88% agree that it is important that AI explains the diagnosis.<n>A combination of bounding box and report is rated better than the other tested XAI methods in the evaluated aspects understandability, completeness, speed, and applicability.
• Score: 0.042970700836450486
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, AI systems in the medical domain have advanced significantly. However, despite outperforming humans, they are rarely used in practice since it is often not clear how they make their decisions. Optimal explanation and visualization of the decision process are often lacking. Therefore, we conducted a comparative user-centric analysis of the latest state-of-the-art textual, visual and multimodal explainable artificial intelligence (XAI) methods for medical image diagnosis. Our survey of 33 physicians showed that 88% agree that it is important that AI explains the diagnosis -- 64% even strongly agree. A combination of bounding box and report is rated better than the other tested XAI methods in the evaluated aspects understandability, completeness, speed, and applicability. We even tested the potential negative impact of false AI-based medical image diagnoses and found that 50% of the participants trusted false AI diagnoses over all tested XAI methods.

Related papers

• BenchX: Benchmarking AI Models for Cancer Detection and Localization with Demographic and Protocol Biases [59.19934491064188]
  Artificial intelligence (AI) has achieved remarkable success in medical imaging.<n>These models often perform inconsistently across real-world clinical settings.<n>We develop a benchmark of 85,355 CT scans that systematically evaluates 12 tumor-detection AI models.
  arXiv  Detail & Related papers  (2026-06-23T17:58:59Z)
• Explainable AI as a Double-Edged Sword in Dermatology: The Impact on Clinicians versus The Public [46.86429592892395]
  explainable AI (XAI) addresses this by providing AI decision-making insight.<n>We present results from two large-scale experiments combining a fairness-based diagnosis AI model and different XAI explanations.
  arXiv  Detail & Related papers  (2025-12-14T00:06:06Z)
• How to Evaluate Medical AI [4.23552814358972]
  We introduce Relative Precision and Recall of Algorithmic Diagnostics (RPAD and RRAD)<n>RPAD and RRAD compare AI outputs against multiple expert opinions rather than a single reference.<n>Large-scale study shows that top-performing models, such as DeepSeek-V3, achieve consistency on par with or exceeding expert consensus.
  arXiv  Detail & Related papers  (2025-09-15T14:01:22Z)
• Reverse Physician-AI Relationship: Full-process Clinical Diagnosis Driven by a Large Language Model [71.40113970879219]
  We propose a paradigm shift that reverses the relationship between physicians and AI.<n>We present DxDirector-7B, an LLM endowed with advanced deep thinking capabilities, enabling it to drive the full-process diagnosis with minimal physician involvement.<n>In evaluations across rare, complex, and real-world cases under full-process diagnosis setting, DxDirector-7B not only achieves significant superior diagnostic accuracy but also substantially reduces physician workload.
  arXiv  Detail & Related papers  (2025-08-14T09:51:20Z)
• Dermatologist-like explainable AI enhances melanoma diagnosis accuracy: eye-tracking study [1.1876787296873537]
  Artificial intelligence (AI) systems have substantially improved dermatologists' diagnostic accuracy for melanoma. Despite these advancements, there remains a critical need for objective evaluation of how dermatologists engage with both AI and XAI tools. In this study, 76 dermatologists participated in a reader study, diagnosing 16 dermoscopic images of melanomas and nevi using an XAI system that provides detailed, domain-specific explanations.
  arXiv  Detail & Related papers  (2024-09-20T13:08:33Z)
• The Limits of Perception: Analyzing Inconsistencies in Saliency Maps in XAI [0.0]
  Explainable artificial intelligence (XAI) plays an indispensable role in demystifying the decision-making processes of AI. As they operate as "black boxes," with their reasoning obscured and inaccessible, there's an increased risk of misdiagnosis. This shift towards transparency is not just beneficial -- it's a critical step towards responsible AI integration in healthcare.
  arXiv  Detail & Related papers  (2024-03-23T02:15:23Z)
• BMAD: Benchmarks for Medical Anomaly Detection [51.22159321912891]
  Anomaly detection (AD) is a fundamental research problem in machine learning and computer vision. In medical imaging, AD is especially vital for detecting and diagnosing anomalies that may indicate rare diseases or conditions. We introduce a comprehensive evaluation benchmark for assessing anomaly detection methods on medical images.
  arXiv  Detail & Related papers  (2023-06-20T20:23:46Z)
• What Do End-Users Really Want? Investigation of Human-Centered XAI for Mobile Health Apps [69.53730499849023]
  We present a user-centered persona concept to evaluate explainable AI (XAI) Results show that users' demographics and personality, as well as the type of explanation, impact explanation preferences. Our insights bring an interactive, human-centered XAI closer to practical application.
  arXiv  Detail & Related papers  (2022-10-07T12:51:27Z)
• Towards the Use of Saliency Maps for Explaining Low-Quality Electrocardiograms to End Users [51.644376281196394]
  When using medical images for diagnosis, it is important that the images are of high quality.<n>In telemedicine, a common problem is that the quality issue is only flagged once the patient has left the clinic, meaning they must return in order to have the exam redone.<n>This paper reports on the development of an AI system for flagging and explaining low-quality medical images in real-time.
  arXiv  Detail & Related papers  (2022-07-06T14:53:26Z)
• Artificial Intelligence in Dry Eye Disease [4.444624718360766]
  Dry eye disease (DED) has a prevalence of between 5 and 50%. Recent success in its applications to medicine is mainly due to advancements in the sub-field of machine learning. This is the first literature review on the use of AI in DED.
  arXiv  Detail & Related papers  (2021-09-02T10:17:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.