Beyond Attention Heatmaps: How to Get Better Explanations for Multiple Instance Learning Models in Histopathology

• URL: http://arxiv.org/abs/2603.08328v1
• Date: Mon, 09 Mar 2026 12:45:48 GMT
• Title: Beyond Attention Heatmaps: How to Get Better Explanations for Multiple Instance Learning Models in Histopathology
• Abstract summary: Multiple instance learning (MIL) has enabled substantial progress in computational histopathology.<n>Heatmaps are widely used to validate MIL models and to discover tissue biomarkers.<n>We introduce a general framework for evaluating the quality of MIL heatmaps without requiring additional labels.
• Score: 14.800356715603087
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Multiple instance learning (MIL) has enabled substantial progress in computational histopathology, where a large amount of patches from gigapixel whole slide images are aggregated into slide-level predictions. Heatmaps are widely used to validate MIL models and to discover tissue biomarkers. Yet, the validity of these heatmaps has barely been investigated. In this work, we introduce a general framework for evaluating the quality of MIL heatmaps without requiring additional labels. We conduct a large-scale benchmark experiment to assess six explanation methods across histopathology task types (classification, regression, survival), MIL model architectures (Attention-, Transformer-, Mamba-based), and patch encoder backbones (UNI2, Virchow2). Our results show that explanation quality mostly depends on MIL model architecture and task type, with perturbation ("Single"), layer-wise relevance propagation (LRP), and integrated gradients (IG) consistently outperforming attention-based and gradient-based saliency heatmaps, which often fail to reflect model decision mechanisms. We further demonstrate the advanced capabilities of the best-performing explanation methods: (i) We provide a proof-of-concept that MIL heatmaps of a bulk gene expression prediction model can be correlated with spatial transcriptomics for biological validation, and (ii) showcase the discovery of distinct model strategies for predicting human papillomavirus (HPV) infection from head and neck cancer slides. Our work highlights the importance of validating MIL heatmaps and establishes that improved explainability can enable more reliable model validation and yield biological insights, making a case for a broader adoption of explainable AI in digital pathology. Our code is provided in a public GitHub repository: https://github.com/bifold-pathomics/xMIL/tree/xmil-journal

Related papers

• MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models [83.50100003741628]
  Vision and language models (VLMs) hold immense promise to transform biomedical imaging.<n>We introduce the Massive Multimodal Biomedical Understanding (MMBU) benchmark.<n>It is the largest vision and language benchmark to date, covering 35 submodalities with rich structured metadata.
  arXiv  Detail & Related papers  (2026-06-04T20:24:47Z)
• Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital Pathology [12.138272834650502]
  Explanations of multiple instance learning (MIL) models are widely used for validation and discovery in digital histopathology.<n>We introduce explainable MIL (Symb-xMIL), a post-hoc explanation framework that quantifies how a MIL model's behavior aligns with human-readable decision rules.<n>We evaluate Symb-xMIL on synthetic and real-world histopathology datasets.
  arXiv  Detail & Related papers  (2026-06-04T14:35:22Z)
• Multimodal Graph-based Classification of Esophageal Motility Disorders [73.90451172929117]
  Diagnosing esophageal motility disorders pose significant challenges due to the complexity of high-resolution impedance manometry data and variability in clinical interpretation.<n>This work explores the feasibility of a multimodal Machine Learning-based classification approach that combines HRIM recordings with patient-specific information and incorporates a graph-based modeling of esophageal physiology.
  arXiv  Detail & Related papers  (2026-05-13T14:52:12Z)
• DCMIL: A Progressive Representation Learning of Whole Slide Images for Cancer Prognosis Analysis [20.81711023855778]
  We propose an easy-to-hard progressive representation learning, termed dual-curriculum contrastive multi-instance learning (DCMIL)<n>Experiments on twelve cancer types (5,954 patients, 12.54 million tiles) demonstrate that DCMIL outperforms standard WSI-based prognostic models.<n>DCMIL identifies fine-grained prognosis-salient regions, provides robust instance uncertainty estimation, and captures morphological differences between normal and tumor tissues.
  arXiv  Detail & Related papers  (2025-10-16T08:03:54Z)
• MMAP: A Multi-Magnification and Prototype-Aware Architecture for Predicting Spatial Gene Expression [1.083137038945176]
  Spatial Transcriptomics (ST) enables the measurement of gene expression while preserving spatial information.<n>Recent developments have explored the use of hematoxylin and eosin (H&E)-stained whole-slide images (WSIs) to predict transcriptome-wide gene expression profiles through deep neural networks.<n>However, predicting spatial gene expression from histological images remains a challenging problem due to the significant modality gap between visual features and molecular signals.<n>In this work, we propose a novel framework, MMAP (Multi-MAgnification and Prototype-enhanced architecture), that addresses both challenges simultaneously.
  arXiv  Detail & Related papers  (2025-10-13T12:41:09Z)
• Pathological Prior-Guided Multiple Instance Learning For Mitigating Catastrophic Forgetting in Breast Cancer Whole Slide Image Classification [50.899861205016265]
  We propose a new framework PaGMIL to mitigate catastrophic forgetting in breast cancer WSI classification.<n>Our framework introduces two key components into the common MIL model architecture.<n>We evaluate the continual learning performance of PaGMIL across several public breast cancer datasets.
  arXiv  Detail & Related papers  (2025-03-08T04:51:58Z)
• xMIL: Insightful Explanations for Multiple Instance Learning in Histopathology [13.939494815120666]
  Multiple instance learning (MIL) is an effective and widely used approach for weakly supervised machine learning.<n>We revisit MIL through the lens of explainable AI (XAI) and introduce xMIL, a refined framework with more general assumptions.<n>Our approach consistently outperforms previous explanation attempts with particularly improved faithfulness scores on challenging biomarker prediction tasks.
  arXiv  Detail & Related papers  (2024-06-06T17:26:40Z)
• Introducing instance label correlation in multiple instance learning. Application to cancer detection on histopathological images [5.895585247199983]
  In this work, we extend a state-of-the-art GP-based MIL method, which is called VGPMIL-PR, to exploit such correlation. We show that our model achieves better results than other state-of-the-art probabilistic MIL methods.
  arXiv  Detail & Related papers  (2023-10-30T08:57:59Z)
• Genetic InfoMax: Exploring Mutual Information Maximization in High-Dimensional Imaging Genetics Studies [50.11449968854487]
  Genome-wide association studies (GWAS) are used to identify relationships between genetic variations and specific traits. Representation learning for imaging genetics is largely under-explored due to the unique challenges posed by GWAS. We introduce a trans-modal learning framework Genetic InfoMax (GIM) to address the specific challenges of GWAS.
  arXiv  Detail & Related papers  (2023-09-26T03:59:21Z)
• Hierarchical Transformer for Survival Prediction Using Multimodality Whole Slide Images and Genomics [63.76637479503006]
  Learning good representation of giga-pixel level whole slide pathology images (WSI) for downstream tasks is critical. This paper proposes a hierarchical-based multimodal transformer framework that learns a hierarchical mapping between pathology images and corresponding genes. Our architecture requires fewer GPU resources compared with benchmark methods while maintaining better WSI representation ability.
  arXiv  Detail & Related papers  (2022-11-29T23:47:56Z)
• Differentiable Agent-based Epidemiology [71.81552021144589]
  We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation. GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-07-20T07:32:02Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.