Explaining Chest X-ray Pathology Models using Textual Concepts

• URL: http://arxiv.org/abs/2407.00557v2
• Date: Tue, 22 Oct 2024 15:50:00 GMT
• Title: Explaining Chest X-ray Pathology Models using Textual Concepts
• Authors: Vijay Sadashivaiah, Pingkun Yan, James A. Hendler,
• Abstract summary: We propose Conceptual Counterfactual Explanations for Chest X-ray (CoCoX) We leverage the joint embedding space of an existing vision-language model (VLM) to explain black-box classifier outcomes without the need for annotated datasets. We demonstrate that the explanations generated by our method are semantically meaningful and faithful to underlying pathologies.
• Score: 9.67960010121851
• License:
• Abstract: Deep learning models have revolutionized medical imaging and diagnostics, yet their opaque nature poses challenges for clinical adoption and trust. Amongst approaches to improve model interpretability, concept-based explanations aim to provide concise and human-understandable explanations of any arbitrary classifier. However, such methods usually require a large amount of manually collected data with concept annotation, which is often scarce in the medical domain. In this paper, we propose Conceptual Counterfactual Explanations for Chest X-ray (CoCoX), which leverages the joint embedding space of an existing vision-language model (VLM) to explain black-box classifier outcomes without the need for annotated datasets. Specifically, we utilize textual concepts derived from chest radiography reports and a pre-trained chest radiography-based VLM to explain three common cardiothoracic pathologies. We demonstrate that the explanations generated by our method are semantically meaningful and faithful to underlying pathologies.

Related papers

• Aligning Human Knowledge with Visual Concepts Towards Explainable Medical Image Classification [8.382606243533942]
  We introduce a simple yet effective framework, Explicd, towards Explainable language-informed criteria-based diagnosis. By leveraging a pretrained vision-language model, Explicd injects these criteria into the embedding space as knowledge anchors. The final diagnostic outcome is determined based on the similarity scores between the encoded visual concepts and the textual criteria embeddings.
  arXiv  Detail & Related papers  (2024-06-08T23:23:28Z)
• Towards Concept-based Interpretability of Skin Lesion Diagnosis using Vision-Language Models [0.0]
  We show that vision-language models can be used to alleviate the dependence on a large number of concept-annotated samples. In particular, we propose an embedding learning strategy to adapt CLIP to the downstream task of skin lesion classification using concept-based descriptions as textual embeddings.
  arXiv  Detail & Related papers  (2023-11-24T08:31:34Z)
• Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models [49.95603725998561]
  We propose a new paradigm to build robust and interpretable medical image classifiers with natural language concepts. Specifically, we first query clinical concepts from GPT-4, then transform latent image features into explicit concepts with a vision-language model.
  arXiv  Detail & Related papers  (2023-10-04T21:57:09Z)
• Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
  We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records. The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
  arXiv  Detail & Related papers  (2022-09-28T10:27:10Z)
• CheXplaining in Style: Counterfactual Explanations for Chest X-rays using StyleGAN [42.4322542603446]
  We create counterfactual explanations for chest X-rays by manipulating specific latent directions in their latent space. We clinically evaluate the relevancy of our counterfactual explanations with the help of radiologists.
  arXiv  Detail & Related papers  (2022-07-15T15:51:08Z)
• Explaining Chest X-ray Pathologies in Natural Language [46.11255491490225]
  We introduce the task of generating natural language explanations (NLEs) to justify predictions made on medical images. NLEs are human-friendly and comprehensive, and enable the training of intrinsically explainable models. We introduce MIMIC-NLE, the first, large-scale, medical imaging dataset with NLEs.
  arXiv  Detail & Related papers  (2022-07-09T22:09:37Z)
• Interpretable Vertebral Fracture Diagnosis [69.68641439851777]
  Black-box neural network models learn clinically relevant features for fracture diagnosis. This work identifies the concepts networks use for vertebral fracture diagnosis in CT images.
  arXiv  Detail & Related papers  (2022-03-30T13:07:41Z)
• ExAID: A Multimodal Explanation Framework for Computer-Aided Diagnosis of Skin Lesions [4.886872847478552]
  ExAID (Explainable AI for Dermatology) is a novel framework for biomedical image analysis. It provides multi-modal concept-based explanations consisting of easy-to-understand textual explanations. It will be the basis for similar applications in other biomedical imaging fields.
  arXiv  Detail & Related papers  (2022-01-04T17:11:28Z)
• BI-RADS-Net: An Explainable Multitask Learning Approach for Cancer Diagnosis in Breast Ultrasound Images [69.41441138140895]
  This paper introduces BI-RADS-Net, a novel explainable deep learning approach for cancer detection in breast ultrasound images. The proposed approach incorporates tasks for explaining and classifying breast tumors, by learning feature representations relevant to clinical diagnosis. Explanations of the predictions (benign or malignant) are provided in terms of morphological features that are used by clinicians for diagnosis and reporting in medical practice.
  arXiv  Detail & Related papers  (2021-10-05T19:14:46Z)
• Using Causal Analysis for Conceptual Deep Learning Explanation [11.552000005640203]
  An ideal explanation resembles the decision-making process of a domain expert. We take advantage of radiology reports accompanying chest X-ray images to define concepts. We construct a low-depth decision tree to translate all the discovered concepts into a straightforward decision rule.
  arXiv  Detail & Related papers  (2021-07-10T00:01:45Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.