Generating Counterfactual Trajectories with Latent Diffusion Models for Concept Discovery

• URL: http://arxiv.org/abs/2404.10356v2
• Date: Mon, 06 Jan 2025 14:47:53 GMT
• Title: Generating Counterfactual Trajectories with Latent Diffusion Models for Concept Discovery
• Authors: Payal Varshney, Adriano Lucieri, Christoph Balada, Andreas Dengel, Sheraz Ahmed,
• Abstract summary: This study proposes Concept Discovery through Latent Diffusion-based Counterfactual Trajectories (CDCT)<n>CDCT is a novel three-step framework for concept discovery leveraging the superior image synthesis capabilities of diffusion models.<n>The application of CDCT to a trained on the largest public skin lesion dataset revealed not only the presence of several biases but also meaningful biomarkers.
• Score: 4.891597567642704
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Trustworthiness is a major prerequisite for the safe application of opaque deep learning models in high-stakes domains like medicine. Understanding the decision-making process not only contributes to fostering trust but might also reveal previously unknown decision criteria of complex models that could advance the state of medical research. The discovery of decision-relevant concepts from black box models is a particularly challenging task. This study proposes Concept Discovery through Latent Diffusion-based Counterfactual Trajectories (CDCT), a novel three-step framework for concept discovery leveraging the superior image synthesis capabilities of diffusion models. In the first step, CDCT uses a Latent Diffusion Model (LDM) to generate a counterfactual trajectory dataset. This dataset is used to derive a disentangled representation of classification-relevant concepts using a Variational Autoencoder (VAE). Finally, a search algorithm is applied to identify relevant concepts in the disentangled latent space. The application of CDCT to a classifier trained on the largest public skin lesion dataset revealed not only the presence of several biases but also meaningful biomarkers. Moreover, the counterfactuals generated within CDCT show better FID scores than those produced by a previously established state-of-the-art method, while being 12 times more resource-efficient. Unsupervised concept discovery holds great potential for the application of trustworthy AI and the further development of human knowledge in various domains. CDCT represents a further step in this direction.

Related papers

• Generate, Refine, and Encode: Leveraging Synthesized Novel Samples for On-the-Fly Fine-Grained Category Discovery [64.83837781610907]
  We investigate the online identification of newly arriving stream data that may belong to both known and unknown categories.<n>Existing OCD methods are devoted to fully mining transferable knowledge from only labeled data.<n>We propose a diffusion-based OCD framework, dubbed DiffGRE, which integrates attribute-composition generation, Refinement, and supervised recognition.
  arXiv  Detail & Related papers  (2025-07-05T14:20:49Z)
• Stable Vision Concept Transformers for Medical Diagnosis [14.082818181995776]
  Concept Bottleneck Models (CBMs) aim to restrict the model's latent space to human-understandable high-level concepts.<n>CBMs rely solely on concept features to determine the model's predictions.<n>Existing methods rely solely on concept features to determine the model's predictions.
  arXiv  Detail & Related papers  (2025-06-05T17:43:27Z)
• Discovering Concept Directions from Diffusion-based Counterfactuals via Latent Clustering [4.891597567642704]
  Concept-based explanations have emerged as an effective approach within Explainable Artificial Intelligence.<n>This work introduces Concept Directions via Latent Clustering (CDLC), which extracts global, class-specific concept directions.<n>This approach is validated on a real-world skin lesion dataset.
  arXiv  Detail & Related papers  (2025-05-11T17:53:02Z)
• Efficient Epistemic Uncertainty Estimation in Cerebrovascular Segmentation [1.3980986259786223]
  We introduce an efficient ensemble model combining the advantages of Bayesian Approximation and Deep Ensembles. Areas of high model uncertainty and erroneous predictions are aligned which demonstrates the effectiveness and reliability of the approach.
  arXiv  Detail & Related papers  (2025-03-28T09:39:37Z)
• ConceptVAE: Self-Supervised Fine-Grained Concept Disentanglement from 2D Echocardiographies [0.0]
  ConceptVAE is a novel pre-training framework that detects and disentangles fine-grained concepts from their style characteristics in a self-supervised manner. We present a suite of loss terms and model architecture primitives designed to discretise input data into a preset number of concepts along with their local style. We validate ConceptVAE both qualitatively and quantitatively, demonstrating its ability to detect fine-grained anatomical structures such as blood pools and septum walls from 2D cardiac echocardiographies.
  arXiv  Detail & Related papers  (2025-02-03T13:18:01Z)
• MultiEYE: Dataset and Benchmark for OCT-Enhanced Retinal Disease Recognition from Fundus Images [4.885485496458059]
  We present the first large multi-modal multi-class dataset for eye disease diagnosis, MultiEYE. We propose an OCT-assisted Conceptual Distillation Approach ( OCT-CoDA) to extract disease-related knowledge from OCT images. Our proposed OCT-CoDA demonstrates remarkable results and interpretability, showing great potential for clinical application.
  arXiv  Detail & Related papers  (2024-12-12T16:08:43Z)
• Generative Edge Detection with Stable Diffusion [52.870631376660924]
  Edge detection is typically viewed as a pixel-level classification problem mainly addressed by discriminative methods. We propose a novel approach, named Generative Edge Detector (GED), by fully utilizing the potential of the pre-trained stable diffusion model. We conduct extensive experiments on multiple datasets and achieve competitive performance.
  arXiv  Detail & Related papers  (2024-10-04T01:52:23Z)
• Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation [56.87049651707208]
  Few-shot Semantic has evolved into In-context tasks, morphing into a crucial element in assessing generalist segmentation models. Our initial focus lies in understanding how to facilitate interaction between the query image and the support image, resulting in the proposal of a KV fusion method within the self-attention framework. Based on our analysis, we establish a simple and effective framework named DiffewS, maximally retaining the original Latent Diffusion Model's generative framework.
  arXiv  Detail & Related papers  (2024-10-03T10:33:49Z)
• HistoSPACE: Histology-Inspired Spatial Transcriptome Prediction And Characterization Engine [0.0]
  HistoSPACE model explore the diversity of histological images available with ST data to extract molecular insights from tissue image. Model demonstrates significant efficiency compared to contemporary algorithms, revealing a correlation of 0.56 in leave-one-out cross-validation.
  arXiv  Detail & Related papers  (2024-08-07T07:12:52Z)
• Counterfactual Explanations for Medical Image Classification and Regression using Diffusion Autoencoder [38.81441978142279]
  We propose a novel method that operates directly on the latent space of a generative model, specifically a Diffusion Autoencoder (DAE) This approach offers inherent interpretability by enabling the generation of Counterfactual explanations (CEs) We show that these latent representations are helpful for medical condition classification and the ordinal regression of pathologies, such as vertebral compression fractures (VCF) and diabetic retinopathy (DR)
  arXiv  Detail & Related papers  (2024-08-02T21:01:30Z)
• ETSCL: An Evidence Theory-Based Supervised Contrastive Learning Framework for Multi-modal Glaucoma Grading [7.188153974946432]
  Glaucoma is one of the leading causes of vision impairment. It remains challenging to extract reliable features due to the high similarity of medical images and the unbalanced multi-modal data distribution. We propose a novel framework, namely ETSCL, which consists of a contrastive feature extraction stage and a decision-level fusion stage.
  arXiv  Detail & Related papers  (2024-07-19T11:57:56Z)
• Application of Multimodal Fusion Deep Learning Model in Disease Recognition [14.655086303102575]
  This paper introduces an innovative multi-modal fusion deep learning approach to overcome the drawbacks of traditional single-modal recognition techniques. During the feature extraction stage, cutting-edge deep learning models are applied to distill advanced features from image-based, temporal, and structured data sources. The findings demonstrate significant advantages of the multimodal fusion model across multiple evaluation metrics.
  arXiv  Detail & Related papers  (2024-05-22T23:09:49Z)
• Bridging Generative and Discriminative Models for Unified Visual Perception with Diffusion Priors [56.82596340418697]
  We propose a simple yet effective framework comprising a pre-trained Stable Diffusion (SD) model containing rich generative priors, a unified head (U-head) capable of integrating hierarchical representations, and an adapted expert providing discriminative priors. Comprehensive investigations unveil potential characteristics of Vermouth, such as varying granularity of perception concealed in latent variables at distinct time steps and various U-net stages. The promising results demonstrate the potential of diffusion models as formidable learners, establishing their significance in furnishing informative and robust visual representations.
  arXiv  Detail & Related papers  (2024-01-29T10:36:57Z)
• Ambiguous Medical Image Segmentation using Diffusion Models [60.378180265885945]
  We introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent sampling process of diffusion. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks.
  arXiv  Detail & Related papers  (2023-04-10T17:58:22Z)
• Unsupervised Pathology Detection: A Deep Dive Into the State of the Art [6.667150890634173]
  We evaluate a selection of cutting-edge Unsupervised Anomaly Detection (UAD) methods on multiple medical datasets. Our experiments demonstrate that newly developed feature-modeling methods from the industrial and medical literature achieve increased performance. We show that such methods are capable of benefiting from recently developed self-supervised pre-training algorithms.
  arXiv  Detail & Related papers  (2023-03-01T16:03:25Z)
• A Survey on Generative Diffusion Model [75.93774014861978]
  Diffusion models are an emerging class of deep generative models. They have certain limitations, including a time-consuming iterative generation process and confinement to high-dimensional Euclidean space. This survey presents a plethora of advanced techniques aimed at enhancing diffusion models.
  arXiv  Detail & Related papers  (2022-09-06T16:56:21Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.