Towards generating more interpretable counterfactuals via concept vectors: a preliminary study on chest X-rays

• URL: http://arxiv.org/abs/2506.04058v1
• Date: Wed, 04 Jun 2025 15:23:12 GMT
• Title: Towards generating more interpretable counterfactuals via concept vectors: a preliminary study on chest X-rays
• Authors: Bulat Maksudov, Kathleen Curran, Alessandra Mileo,
• Abstract summary: We map clinical concepts into the latent space of generative models to identify Concept Activation Vectors (CAVs)<n>The extracted concepts are stable across datasets, enabling visual explanations that highlight clinically relevant features.<n>Preliminary results on chest X-rays show promise for large pathologies like cardiomegaly, while smaller pathologies remain challenging.
• Score: 46.667021835430155
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An essential step in deploying medical imaging models is ensuring alignment with clinical knowledge and interpretability. We focus on mapping clinical concepts into the latent space of generative models to identify Concept Activation Vectors (CAVs). Using a simple reconstruction autoencoder, we link user-defined concepts to image-level features without explicit label training. The extracted concepts are stable across datasets, enabling visual explanations that highlight clinically relevant features. By traversing latent space along concept directions, we produce counterfactuals that exaggerate or reduce specific clinical features. Preliminary results on chest X-rays show promise for large pathologies like cardiomegaly, while smaller pathologies remain challenging due to reconstruction limits. Although not outperforming baselines, this approach offers a path toward interpretable, concept-based explanations aligned with clinical knowledge.

Related papers

• GEMeX-ThinkVG: Towards Thinking with Visual Grounding in Medical VQA via Reinforcement Learning [50.94508930739623]
  Medical visual question answering aims to support clinical decision-making by enabling models to answer natural language questions based on medical images.<n>Current methods still suffer from limited answer reliability and poor interpretability, impairing the ability of clinicians and patients to understand and trust model-generated answers.<n>This work first proposes a Thinking with Visual Grounding dataset wherein the answer generation is decomposed into intermediate reasoning steps.<n>We introduce a novel verifiable reward mechanism for reinforcement learning to guide post-training, improving the alignment between the model's reasoning process and its final answer.
  arXiv  Detail & Related papers  (2025-06-22T08:09:58Z)
• Explaining Chest X-ray Pathology Models using Textual Concepts [9.67960010121851]
  We propose Conceptual Counterfactual Explanations for Chest X-ray (CoCoX) We leverage the joint embedding space of an existing vision-language model (VLM) to explain black-box classifier outcomes without the need for annotated datasets. We demonstrate that the explanations generated by our method are semantically meaningful and faithful to underlying pathologies.
  arXiv  Detail & Related papers  (2024-06-30T01:31:54Z)
• Concept-Attention Whitening for Interpretable Skin Lesion Diagnosis [7.5422729055429745]
  We propose a novel Concept-Attention Whitening (CAW) framework for interpretable skin lesion diagnosis. In the former branch, we train a convolutional neural network (CNN) with an inserted CAW layer to perform skin lesion diagnosis. In the latter branch, the matrix is calculated under the guidance of the concept attention mask.
  arXiv  Detail & Related papers  (2024-04-09T04:04:50Z)
• MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment [4.861768967055006]
  We propose a multi-modal explainable disease diagnosis framework that meticulously aligns medical images and clinical-related concepts semantically at multiple strata. Our method, while preserving model interpretability, attains high performance and label efficiency for concept detection and disease diagnosis.
  arXiv  Detail & Related papers  (2024-01-16T17:45:01Z)
• Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models [49.95603725998561]
  We propose a new paradigm to build robust and interpretable medical image classifiers with natural language concepts. Specifically, we first query clinical concepts from GPT-4, then transform latent image features into explicit concepts with a vision-language model.
  arXiv  Detail & Related papers  (2023-10-04T21:57:09Z)
• Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation [116.87918100031153]
  We propose a Cross-modal clinical Graph Transformer (CGT) for ophthalmic report generation (ORG) CGT injects clinical relation triples into the visual features as prior knowledge to drive the decoding procedure. Experiments on the large-scale FFA-IR benchmark demonstrate that the proposed CGT is able to outperform previous benchmark methods.
  arXiv  Detail & Related papers  (2022-06-04T13:16:30Z)
• Interpretable Vertebral Fracture Diagnosis [69.68641439851777]
  Black-box neural network models learn clinically relevant features for fracture diagnosis. This work identifies the concepts networks use for vertebral fracture diagnosis in CT images.
  arXiv  Detail & Related papers  (2022-03-30T13:07:41Z)
• BI-RADS-Net: An Explainable Multitask Learning Approach for Cancer Diagnosis in Breast Ultrasound Images [69.41441138140895]
  This paper introduces BI-RADS-Net, a novel explainable deep learning approach for cancer detection in breast ultrasound images. The proposed approach incorporates tasks for explaining and classifying breast tumors, by learning feature representations relevant to clinical diagnosis. Explanations of the predictions (benign or malignant) are provided in terms of morphological features that are used by clinicians for diagnosis and reporting in medical practice.
  arXiv  Detail & Related papers  (2021-10-05T19:14:46Z)
• Explaining Clinical Decision Support Systems in Medical Imaging using Cycle-Consistent Activation Maximization [112.2628296775395]
  Clinical decision support using deep neural networks has become a topic of steadily growing interest. clinicians are often hesitant to adopt the technology because its underlying decision-making process is considered to be intransparent and difficult to comprehend. We propose a novel decision explanation scheme based on CycleGAN activation which generates high-quality visualizations of classifier decisions even in smaller data sets.
  arXiv  Detail & Related papers  (2020-10-09T14:39:27Z)
• Interpretable Deep Models for Cardiac Resynchronisation Therapy Response Prediction [8.152884957975354]
  We propose a novel framework for image-based classification based on a variational autoencoder (VAE) The VAE disentangles the latent space based on explanations' drawn from existing clinical knowledge. We demonstrate our framework on the problem of predicting response of patients with cardiomyopathy to cardiac resynchronization therapy (CRT) from cine cardiac magnetic resonance images.
  arXiv  Detail & Related papers  (2020-06-24T15:35:47Z)
• On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors [6.188009802619095]
  We use a well-trained and high performing neural network for classification of three skin tumours, i.e. Melanocytic Naevi, Melanoma and Seborrheic Keratosis. Human understandable concepts are mapped to RECOD image classification model with the help of Concept Activation Vectors (CAVs)
  arXiv  Detail & Related papers  (2020-05-05T08:27:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.