Visual Evaluative AI: A Hypothesis-Driven Tool with Concept-Based Explanations and Weight of Evidence

• URL: http://arxiv.org/abs/2407.04710v3
• Date: Mon, 25 Aug 2025 08:21:14 GMT
• Title: Visual Evaluative AI: A Hypothesis-Driven Tool with Concept-Based Explanations and Weight of Evidence
• Authors: Thao Le, Tim Miller, Ruihan Zhang, Liz Sonenberg, Ronal Singh,
• Abstract summary: This paper presents Visual Evaluative AI, a decision aid that provides positive and negative evidence from image data for a given hypothesis.<n>We apply and evaluate this tool in the skin cancer domain by building a web-based application that allows users to upload a dermatoscopic image.
• Score: 6.144558727925986
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents Visual Evaluative AI, a decision aid that provides positive and negative evidence from image data for a given hypothesis. This tool finds high-level human concepts in an image and generates the Weight of Evidence (WoE) for each hypothesis in the decision-making process. We apply and evaluate this tool in the skin cancer domain by building a web-based application that allows users to upload a dermatoscopic image, select a hypothesis and analyse their decisions by evaluating the provided evidence. Further, we demonstrate the effectiveness of Visual Evaluative AI on different concept-based explanation approaches.

Related papers

• GEMeX-ThinkVG: Towards Thinking with Visual Grounding in Medical VQA via Reinforcement Learning [50.94508930739623]
  Medical visual question answering aims to support clinical decision-making by enabling models to answer natural language questions based on medical images.<n>Current methods still suffer from limited answer reliability and poor interpretability, impairing the ability of clinicians and patients to understand and trust model-generated answers.<n>This work first proposes a Thinking with Visual Grounding dataset wherein the answer generation is decomposed into intermediate reasoning steps.<n>We introduce a novel verifiable reward mechanism for reinforcement learning to guide post-training, improving the alignment between the model's reasoning process and its final answer.
  arXiv  Detail & Related papers  (2025-06-22T08:09:58Z)
• PCEvE: Part Contribution Evaluation Based Model Explanation for Human Figure Drawing Assessment and Beyond [5.104096700315427]
  We propose a part contribution evaluation based model explanation (PCEvE) framework. Unlike existing attribution-based XAI approaches, the PCEvE provides a straightforward explanation of a model decision. We rigorously validate the PCEvE via extensive experiments on multiple HFD assessment datasets.
  arXiv  Detail & Related papers  (2024-09-26T20:13:03Z)
• Understanding and Evaluating Human Preferences for AI Generated Images with Instruction Tuning [58.41087653543607]
  We first establish a novel Image Quality Assessment (IQA) database for AIGIs, termed AIGCIQA2023+. This paper presents a MINT-IQA model to evaluate and explain human preferences for AIGIs from Multi-perspectives with INstruction Tuning.
  arXiv  Detail & Related papers  (2024-05-12T17:45:11Z)
• Towards the New XAI: A Hypothesis-Driven Approach to Decision Support Using Evidence [9.916507773707917]
  We describe and evaluate an approach for hypothesis-driven AI based on the Weight of Evidence (WoE) framework. We show that our hypothesis-driven approach increases decision accuracy and reduces reliance compared to a recommendation-driven approach.
  arXiv  Detail & Related papers  (2024-02-02T10:28:24Z)
• MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment [4.861768967055006]
  We propose a multi-modal explainable disease diagnosis framework that meticulously aligns medical images and clinical-related concepts semantically at multiple strata. Our method, while preserving model interpretability, attains high performance and label efficiency for concept detection and disease diagnosis.
  arXiv  Detail & Related papers  (2024-01-16T17:45:01Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Decoding Susceptibility: Modeling Misbelief to Misinformation Through a Computational Approach [61.04606493712002]
  Susceptibility to misinformation describes the degree of belief in unverifiable claims that is not observable. Existing susceptibility studies heavily rely on self-reported beliefs. We propose a computational approach to model users' latent susceptibility levels.
  arXiv  Detail & Related papers  (2023-11-16T07:22:56Z)
• Coherent Concept-based Explanations in Medical Image and Its Application to Skin Lesion Diagnosis [0.0]
  Existing deep learning approaches for melanoma skin lesion diagnosis are deemed black-box models. We propose an inherently interpretable framework to improve the interpretability of concept-based models. Our method outperforms existing black-box and concept-based models for skin lesion classification.
  arXiv  Detail & Related papers  (2023-04-10T13:32:04Z)
• Unveiling the Unseen: A Comprehensive Survey on Explainable Anomaly Detection in Images and Videos [49.07140708026425]
  Anomaly detection and localization in visual data, including images and videos, are crucial in machine learning and real-world applications.<n>This paper provides the first comprehensive survey focused specifically on explainable 2D visual anomaly detection (X-VAD)<n>We present a literature review of explainable methods, categorized by their underlying techniques.<n>We discuss promising future directions and open problems, including quantifying explanation quality.
  arXiv  Detail & Related papers  (2023-02-13T20:17:41Z)
• Exploring CLIP for Assessing the Look and Feel of Images [87.97623543523858]
  We introduce Contrastive Language-Image Pre-training (CLIP) models for assessing both the quality perception (look) and abstract perception (feel) of images in a zero-shot manner. Our results show that CLIP captures meaningful priors that generalize well to different perceptual assessments.
  arXiv  Detail & Related papers  (2022-07-25T17:58:16Z)
• ExAID: A Multimodal Explanation Framework for Computer-Aided Diagnosis of Skin Lesions [4.886872847478552]
  ExAID (Explainable AI for Dermatology) is a novel framework for biomedical image analysis. It provides multi-modal concept-based explanations consisting of easy-to-understand textual explanations. It will be the basis for similar applications in other biomedical imaging fields.
  arXiv  Detail & Related papers  (2022-01-04T17:11:28Z)
• Image Quality Assessment in the Modern Age [53.19271326110551]
  This tutorial provides the audience with the basic theories, methodologies, and current progresses of image quality assessment (IQA) We will first revisit several subjective quality assessment methodologies, with emphasis on how to properly select visual stimuli. Both hand-engineered and (deep) learning-based methods will be covered.
  arXiv  Detail & Related papers  (2021-10-19T02:38:46Z)
• A Survey of Orthogonal Moments for Image Representation: Theory, Implementation, and Evaluation [70.0671278823937]
  Moment-based image representation has been reported to be effective in satisfying the core conditions of semantic description. This paper presents a comprehensive survey of the orthogonal moments for image representation, covering recent advances in fast/accurate calculation, robustness/invariance optimization, and definition extension. The presented theory analysis, software implementation, and evaluation results can support the community, particularly in developing novel techniques and promoting real-world applications.
  arXiv  Detail & Related papers  (2021-03-27T03:41:08Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.