SAGE: Semantic Explainability of Attention-Based Survival Models in Computational Pathology

• URL: http://arxiv.org/abs/2608.02803v1
• Date: Mon, 03 Aug 2026 18:58:39 GMT
• Title: SAGE: Semantic Explainability of Attention-Based Survival Models in Computational Pathology
• Abstract summary: We present Semantic Attention Global Explanations (SAGE), a post-hoc framework that extracts global, language-grounded explanations from a frozen ABMIL model.<n>Using a pathology vision-language model, SAGE scores image patches against a dictionary of 25 histological concepts, aggregates these scores according to the model's learned attention, and quantifies how each concept relates to prediction risk across a cohort.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Attention-based multiple instance learning (ABMIL) is the predominant approach for slide-level prediction in computational pathology, yet its attention maps provide only local explanations: they indicate where a model focuses but not which histological features drive its predictions or how the model behaves across a patient cohort. We present Semantic Attention Global Explanations (SAGE), a post-hoc framework that extracts global, language-grounded explanations from a frozen ABMIL model. Using a pathology vision-language model, SAGE scores image patches against a dictionary of 25 histological concepts, aggregates these scores according to the model's learned attention, and quantifies how each concept relates to prediction risk across a cohort. Applied to survival prediction using seven TCGA cancer cohorts and three foundation models, SAGE recovered established prognostic features, such as the adverse association of necrosis, while revealing cancer-specific biology, including a favorable angiogenic signature in renal cell carcinoma consistent with known molecular subtypes. Ablation studies demonstrated that these associations depend on the model's learned attention rather than concept prevalence alone, and that the concept dictionary captures much of the prognostic information encoded by the foundation model features. Through semantically-grounded explanations, SAGE provides a scalable, model-agnostic framework for understanding what ABMIL survival models learn, enabling pathologists to interpret model behavior at the cohort level and offering the potential for biomarker identification.

Related papers

• Multimodal Graph-based Classification of Esophageal Motility Disorders [73.90451172929117]
  Diagnosing esophageal motility disorders pose significant challenges due to the complexity of high-resolution impedance manometry data and variability in clinical interpretation.<n>This work explores the feasibility of a multimodal Machine Learning-based classification approach that combines HRIM recordings with patient-specific information and incorporates a graph-based modeling of esophageal physiology.
  arXiv  Detail & Related papers  (2026-05-13T14:52:12Z)
• Investigating the Impact of Histopathological Foundation Models on Regressive Prediction of Homologous Recombination Deficiency [52.50039435394964]
  We systematically evaluate foundation models for regression-based tasks.<n>We extract patch-level features from whole slide images (WSI) using five state-of-the-art foundation models.<n>Models are trained to predict continuous HRD scores based on these extracted features across breast, endometrial, and lung cancer cohorts.
  arXiv  Detail & Related papers  (2026-01-29T14:06:50Z)
• CytoSAE: Interpretable Cell Embeddings for Hematology [3.4855184894829594]
  Sparse autoencoders (SAEs) emerged as a promising tool for mechanistic interpretability of transformer-based foundation models.<n>We propose CytoSAE, a sparse autoencoder which is trained on over 40,000 peripheral blood single-cell images.<n>We show that CytoSAE concepts reach performance comparable to the state-of-the-art, while offering explainability on the sub-cellular level.
  arXiv  Detail & Related papers  (2025-07-16T17:59:32Z)
• MIRROR: Multi-Modal Pathological Self-Supervised Representation Learning via Modality Alignment and Retention [57.044719143401664]
  Histopathology and transcriptomics are fundamental modalities in oncology, encapsulating the morphological and molecular aspects of the disease.<n>We present MIRROR, a novel multi-modal representation learning method designed to foster both modality alignment and retention.<n>Extensive evaluations on TCGA cohorts for cancer subtyping and survival analysis highlight MIRROR's superior performance.
  arXiv  Detail & Related papers  (2025-03-01T07:02:30Z)
• Label-free Concept Based Multiple Instance Learning for Gigapixel Histopathology [3.3353153750791718]
  Multiple Instance Learning (MIL) methods allow for gigapixel Whole-Slide Image (WSI) analysis with only slide-level annotations.<n>We propose a novel inherently interpretable WSI-classification approach that uses human-understandable pathology concepts to generate explanations.
  arXiv  Detail & Related papers  (2025-01-06T11:03:04Z)
• Learning biologically relevant features in a pathology foundation model using sparse autoencoders [2.5919097694815365]
  We trained a Sparse Autoencoder on the embeddings of a pathology pretrained foundation model.<n>We found that Sparse Autoencoder features represent interpretable and monosemantic biological concepts.
  arXiv  Detail & Related papers  (2024-07-15T15:03:01Z)
• Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models [49.95603725998561]
  We propose a new paradigm to build robust and interpretable medical image classifiers with natural language concepts. Specifically, we first query clinical concepts from GPT-4, then transform latent image features into explicit concepts with a vision-language model.
  arXiv  Detail & Related papers  (2023-10-04T21:57:09Z)
• Beyond attention: deriving biologically interpretable insights from weakly-supervised multiple-instance learning models [2.639541396835675]
  We introduce prediction-attention-weighted (PAW) maps by combining tile-level attention and prediction scores produced by a refined encoder. We also introduce a biological feature instantiation technique by integrating PAW maps with nuclei segmentation masks. Our approach reveals that regions that are predictive of adverse prognosis do not tend to co-locate with the tumour regions.
  arXiv  Detail & Related papers  (2023-09-07T09:44:35Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)
• ICAM-reg: Interpretable Classification and Regression with Feature Attribution for Mapping Neurological Phenotypes in Individual Scans [3.589107822343127]
  We take advantage of recent developments in generative deep learning to develop a method for simultaneous classification, or regression, and feature attribution. We validate our method on the tasks of Mini-Mental State Examination (MMSE) cognitive test score prediction for the Alzheimer's Disease Neuroimaging Initiative cohort. We show that the generated FA maps can be used to explain outlier predictions and demonstrate that the inclusion of a regression module improves the disentanglement of the latent space.
  arXiv  Detail & Related papers  (2021-03-03T17:55:14Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.