CytoSAE: Interpretable Cell Embeddings for Hematology

• URL: http://arxiv.org/abs/2507.12464v1
• Date: Wed, 16 Jul 2025 17:59:32 GMT
• Title: CytoSAE: Interpretable Cell Embeddings for Hematology
• Authors: Muhammed Furkan Dasdelen, Hyesu Lim, Michele Buck, Katharina S. Götze, Carsten Marr, Steffen Schneider,
• Abstract summary: Sparse autoencoders (SAEs) emerged as a promising tool for mechanistic interpretability of transformer-based foundation models.<n>We propose CytoSAE, a sparse autoencoder which is trained on over 40,000 peripheral blood single-cell images.<n>We show that CytoSAE concepts reach performance comparable to the state-of-the-art, while offering explainability on the sub-cellular level.
• Score: 3.4855184894829594
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sparse autoencoders (SAEs) emerged as a promising tool for mechanistic interpretability of transformer-based foundation models. Very recently, SAEs were also adopted for the visual domain, enabling the discovery of visual concepts and their patch-wise attribution to tokens in the transformer model. While a growing number of foundation models emerged for medical imaging, tools for explaining their inferences are still lacking. In this work, we show the applicability of SAEs for hematology. We propose CytoSAE, a sparse autoencoder which is trained on over 40,000 peripheral blood single-cell images. CytoSAE generalizes to diverse and out-of-domain datasets, including bone marrow cytology, where it identifies morphologically relevant concepts which we validated with medical experts. Furthermore, we demonstrate scenarios in which CytoSAE can generate patient-specific and disease-specific concepts, enabling the detection of pathognomonic cells and localized cellular abnormalities at the patch level. We quantified the effect of concepts on a patient-level AML subtype classification task and show that CytoSAE concepts reach performance comparable to the state-of-the-art, while offering explainability on the sub-cellular level. Source code and model weights are available at https://github.com/dynamical-inference/cytosae.

Related papers

• Towards generating more interpretable counterfactuals via concept vectors: a preliminary study on chest X-rays [46.667021835430155]
  We map clinical concepts into the latent space of generative models to identify Concept Activation Vectors (CAVs)<n>The extracted concepts are stable across datasets, enabling visual explanations that highlight clinically relevant features.<n>Preliminary results on chest X-rays show promise for large pathologies like cardiomegaly, while smaller pathologies remain challenging.
  arXiv  Detail & Related papers  (2025-06-04T15:23:12Z)
• CytoFM: The first cytology foundation model [3.591868126270513]
  We introduce CytoFM, the first self-supervised foundation model for digital Cytology.<n>We pretrain CytoFM on a diverse collection of datasets to learn robust, transferable representations.<n>Our results demonstrate that CytoFM performs better on two out of three downstream tasks than existing foundation models pretrained on histopathology.
  arXiv  Detail & Related papers  (2025-04-18T01:37:50Z)
• Fine-tuning Vision Language Models with Graph-based Knowledge for Explainable Medical Image Analysis [44.38638601819933]
  Current staging models for Diabetic Retinopathy (DR) are hardly interpretable.<n>We present a novel method that integrates graph representation learning with vision-language models (VLMs) to deliver explainable DR diagnosis.
  arXiv  Detail & Related papers  (2025-03-12T20:19:07Z)
• PathVG: A New Benchmark and Dataset for Pathology Visual Grounding [45.21597220882424]
  We propose a new benchmark called Pathology Visual Grounding (PathVG), which aims to detect regions based on expressions with different attributes.<n>In the experimental study, we found that the biggest challenge was the implicit information underlying the pathological expressions.<n>The proposed method achieves state-of-the-art performance on the PathVG benchmark.
  arXiv  Detail & Related papers  (2025-02-28T09:13:01Z)
• Unified Representation of Genomic and Biomedical Concepts through Multi-Task, Multi-Source Contrastive Learning [45.6771125432388]
  We introduce GENomic REpresentation with Language Model (GENEREL) GENEREL is a framework designed to bridge genetic and biomedical knowledge bases. Our experiments demonstrate GENEREL's ability to effectively capture the nuanced relationships between SNPs and clinical concepts.
  arXiv  Detail & Related papers  (2024-10-14T04:19:52Z)
• Counterfactual Explanations for Medical Image Classification and Regression using Diffusion Autoencoder [38.81441978142279]
  We propose a novel method that operates directly on the latent space of a generative model, specifically a Diffusion Autoencoder (DAE) This approach offers inherent interpretability by enabling the generation of Counterfactual explanations (CEs) We show that these latent representations are helpful for medical condition classification and the ordinal regression of pathologies, such as vertebral compression fractures (VCF) and diabetic retinopathy (DR)
  arXiv  Detail & Related papers  (2024-08-02T21:01:30Z)
• Learning biologically relevant features in a pathology foundation model using sparse autoencoders [2.5919097694815365]
  We trained a Sparse Autoencoder on the embeddings of a pathology pretrained foundation model.<n>We found that Sparse Autoencoder features represent interpretable and monosemantic biological concepts.
  arXiv  Detail & Related papers  (2024-07-15T15:03:01Z)
• Tertiary Lymphoid Structures Generation through Graph-based Diffusion [54.37503714313661]
  In this work, we leverage state-of-the-art graph-based diffusion models to generate biologically meaningful cell-graphs. We show that the adopted graph diffusion model is able to accurately learn the distribution of cells in terms of their tertiary lymphoid structures (TLS) content.
  arXiv  Detail & Related papers  (2023-10-10T14:37:17Z)
• Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models [49.95603725998561]
  We propose a new paradigm to build robust and interpretable medical image classifiers with natural language concepts. Specifically, we first query clinical concepts from GPT-4, then transform latent image features into explicit concepts with a vision-language model.
  arXiv  Detail & Related papers  (2023-10-04T21:57:09Z)
• Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Prediction [3.2274401541163322]
  We propose a memory-efficient multimodal Transformer that can model interactions between pathway and histology patch tokens. Our proposed model, SURVPATH, achieves state-of-the-art performance when evaluated against both unimodal and multimodal baselines.
  arXiv  Detail & Related papers  (2023-04-13T21:02:32Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)
• On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors [6.188009802619095]
  We use a well-trained and high performing neural network for classification of three skin tumours, i.e. Melanocytic Naevi, Melanoma and Seborrheic Keratosis. Human understandable concepts are mapped to RECOD image classification model with the help of Concept Activation Vectors (CAVs)
  arXiv  Detail & Related papers  (2020-05-05T08:27:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.