Seeing What Shouldn't Be There: Counterfactual GANs for Medical Image Attribution

• URL: http://arxiv.org/abs/2605.05283v1
• Date: Wed, 06 May 2026 17:30:49 GMT
• Title: Seeing What Shouldn't Be There: Counterfactual GANs for Medical Image Attribution
• Abstract summary: Ascription of an image gives insights into the objects that influence the classification of the whole image or its pixels towards a specific category.<n>Most of the existing visualization techniques are based on discriminative models and highlight regions of the input image participating in the decision-making of a classifier.<n>To overcome the issue, a counterfactual explanation ( CX) based class-oriented feature attribution method is proposed.
• Score: 1.8149406825355134
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ascription of an image gives insights into the objects that influence the classification of the whole image or its pixels towards a specific category. These insights help radiologists to visualize deformities in medical imaging. Most of the existing visualization techniques are based on discriminative models and highlight regions of the input image participating in the decision-making of a classifier. However, these approaches do not take all noticeable objects into account as their objective is to classify the input by using a minimal set of discriminative features. To overcome the issue, a counterfactual explanation (CX) based class-oriented feature attribution method is proposed. A counterfactual explanation (CX) explicates a causal reasoning process of the form: "if X had not happened, then Y would not have happened". The method is built on generative adversarial networks (GANs) with a cyclical-consistent loss function. We evaluate our method on three datasets: synthetic, tuberculosis and BraTS. All experiments confirm the efficacy of the proposed method. This study also highlighted the limitations of existing counterfactual explanation techniques in producing plausible counterfactual instances (CIs). Accompanying CXs with believable CIs thus provides self-explanatory analogy-based explanations. To this end, a CI generation method is proposed. Also, a novel technique is used to evaluate the quality of CI. The baseline results are produced on the BraTS dataset.

Related papers

• Generating Medical Image Counterfactuals using Causal Explanations [7.128196845242498]
  We use counterfactual images to show how an image would need to change to produce a different prediction.<n>We propose a novel counterfactual-generation framework that requires no generative model.<n>The resulting approach is deterministic, requires no additional model training, and enables controllable edits within user-specified regions of interest.
  arXiv  Detail & Related papers  (2026-09-02T15:03:42Z)
• A Graph-Based Framework for Interpretable Whole Slide Image Analysis [86.37618055724441]
  We develop a framework that transforms whole-slide images into biologically-informed graph representations.<n>Our approach builds graph nodes from tissue regions that respect natural structures, not arbitrary grids.<n>We demonstrate strong performance on challenging cancer staging and survival prediction tasks.
  arXiv  Detail & Related papers  (2025-03-14T20:15:04Z)
• Contrastive Learning with Counterfactual Explanations for Radiology Report Generation [83.30609465252441]
  We propose a textbfCountertextbfFactual textbfExplanations-based framework (CoFE) for radiology report generation. Counterfactual explanations serve as a potent tool for understanding how decisions made by algorithms can be changed by asking what if'' scenarios. Experiments on two benchmarks demonstrate that leveraging the counterfactual explanations enables CoFE to generate semantically coherent and factually complete reports.
  arXiv  Detail & Related papers  (2024-07-19T17:24:25Z)
• When a Relation Tells More Than a Concept: Exploring and Evaluating Classifier Decisions with CoReX [1.8213611231184352]
  Explanations for Convolutional Neural Networks (CNNs) based on relevance of input pixels might be too unspecific to evaluate which and how input features impact model decisions. This work presents a novel method to explain and evaluate CNN models, which uses a concept- and relation-based explainer (CoReX)
  arXiv  Detail & Related papers  (2024-05-02T18:31:47Z)
• Chest X-ray Image Classification: A Causal Perspective [49.87607548975686]
  We propose a causal approach to address the CXR classification problem, which constructs a structural causal model (SCM) and uses the backdoor adjustment to select effective visual information for CXR classification. Experimental results demonstrate that our proposed method outperforms the open-source NIH ChestX-ray14 in terms of classification performance.
  arXiv  Detail & Related papers  (2023-05-20T03:17:44Z)
• Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective [51.70661197256033]
  We propose ARCO, a semi-supervised contrastive learning framework with stratified group theory for medical image segmentation. We first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks. We experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings.
  arXiv  Detail & Related papers  (2023-02-03T13:50:25Z)
• Counterfactual Explanation and Instance-Generation using Cycle-Consistent Generative Adversarial Networks [1.5469452301122175]
  We propose a novel strategy that is not reliant on image classification. We leverage Cycle-Consistency principle (a.k.a CycleGAN) to perform the translation in unsupervised way. We evaluate our method on three datasets including a synthetic, tuberculosis and BraTS dataset.
  arXiv  Detail & Related papers  (2023-01-21T11:14:34Z)
• Evaluation of Saliency-based Explainability Method [2.733700237741334]
  A class of Explainable AI (XAI) methods provide saliency maps to highlight part of the image a CNN model looks at to classify the image as a way to explain its working. These methods provide an intuitive way for users to understand predictions made by CNNs. Other than quantitative computational tests, the vast majority of evidence to highlight that the methods are valuable is anecdotal.
  arXiv  Detail & Related papers  (2021-06-24T05:40:50Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)
• Explaining the Black-box Smoothly- A Counterfactual Approach [12.985758871588454]
  We propose a BlackBox emphCounterfactual Explainer for medical imaging applications. Our framework explains the outcome by gradually emphexaggerating the semantic effect of the given outcome label.
  arXiv  Detail & Related papers  (2021-01-11T23:12:11Z)
• This is not the Texture you are looking for! Introducing Novel Counterfactual Explanations for Non-Experts using Generative Adversarial Learning [59.17685450892182]
  counterfactual explanation systems try to enable a counterfactual reasoning by modifying the input image. We present a novel approach to generate such counterfactual image explanations based on adversarial image-to-image translation techniques. Our results show that our approach leads to significantly better results regarding mental models, explanation satisfaction, trust, emotions, and self-efficacy than two state-of-the art systems.
  arXiv  Detail & Related papers  (2020-12-22T10:08:05Z)
• Combining Similarity and Adversarial Learning to Generate Visual Explanation: Application to Medical Image Classification [0.0]
  We leverage a learning framework to produce our visual explanations method. Using metrics from the literature, our method outperforms state-of-the-art approaches. We validate our approach on a large chest X-ray database.
  arXiv  Detail & Related papers  (2020-12-14T08:34:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.