A Spatially-Aware Multiple Instance Learning Framework for Digital Pathology

• URL: http://arxiv.org/abs/2504.17379v2
• Date: Fri, 25 Apr 2025 07:23:36 GMT
• Title: A Spatially-Aware Multiple Instance Learning Framework for Digital Pathology
• Authors: Hassan Keshvarikhojasteh, Mihail Tifrea, Sibylle Hess, Josien P. W. Pluim, Mitko Veta,
• Abstract summary: Multiple instance learning (MIL) is a promising approach for weakly supervised classification in pathology using whole slide images.<n>Recent advancements, such as Transformer based MIL (TransMIL), have incorporated spatial context and inter-patch relationships.<n>In this work, we enhance the ABMIL framework by integrating interaction-aware representations to address this question.
• Score: 4.012490059423154
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Multiple instance learning (MIL) is a promising approach for weakly supervised classification in pathology using whole slide images (WSIs). However, conventional MIL methods such as Attention-Based Deep Multiple Instance Learning (ABMIL) typically disregard spatial interactions among patches that are crucial to pathological diagnosis. Recent advancements, such as Transformer based MIL (TransMIL), have incorporated spatial context and inter-patch relationships. However, it remains unclear whether explicitly modeling patch relationships yields similar performance gains in ABMIL, which relies solely on Multi-Layer Perceptrons (MLPs). In contrast, TransMIL employs Transformer-based layers, introducing a fundamental architectural shift at the cost of substantially increased computational complexity. In this work, we enhance the ABMIL framework by integrating interaction-aware representations to address this question. Our proposed model, Global ABMIL (GABMIL), explicitly captures inter-instance dependencies while preserving computational efficiency. Experimental results on two publicly available datasets for tumor subtyping in breast and lung cancers demonstrate that GABMIL achieves up to a 7 percentage point improvement in AUPRC and a 5 percentage point increase in the Kappa score over ABMIL, with minimal or no additional computational overhead. These findings underscore the importance of incorporating patch interactions within MIL frameworks. Our code is available at \href{https://github.com/tueimage/GABMIL}{\texttt{GABMIL}}.

Related papers

• Pathological Prior-Guided Multiple Instance Learning For Mitigating Catastrophic Forgetting in Breast Cancer Whole Slide Image Classification [50.899861205016265]
  We propose a new framework PaGMIL to mitigate catastrophic forgetting in breast cancer WSI classification. Our framework introduces two key components into the common MIL model architecture. We evaluate the continual learning performance of PaGMIL across several public breast cancer datasets.
  arXiv  Detail & Related papers  (2025-03-08T04:51:58Z)
• Attention Is Not What You Need: Revisiting Multi-Instance Learning for Whole Slide Image Classification [51.95824566163554]
  We argue that synergizing the standard MIL assumption with variational inference encourages the model to focus on tumour morphology instead of spurious correlations. Our method also achieves better classification boundaries for identifying hard instances and mitigates the effect of spurious correlations between bags and labels.
  arXiv  Detail & Related papers  (2024-08-18T12:15:22Z)
• MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models [56.37780601189795]
  We propose a framework named MamMIL for WSI analysis. We represent each WSI as an undirected graph. To address the problem that Mamba can only process 1D sequences, we propose a topology-aware scanning mechanism.
  arXiv  Detail & Related papers  (2024-03-08T09:02:13Z)
• Reproducibility in Multiple Instance Learning: A Case For Algorithmic Unit Tests [59.623267208433255]
  Multiple Instance Learning (MIL) is a sub-domain of classification problems with positive and negative labels and a "bag" of inputs. In this work, we examine five of the most prominent deep-MIL models and find that none of them respects the standard MIL assumption. We identify and demonstrate this problem via a proposed "algorithmic unit test", where we create synthetic datasets that can be solved by a MIL respecting model.
  arXiv  Detail & Related papers  (2023-10-27T03:05:11Z)
• PDL: Regularizing Multiple Instance Learning with Progressive Dropout Layers [2.069061136213899]
  Multiple instance learning (MIL) was a weakly supervised learning approach that sought to assign binary class labels to collections of instances known as bags. We present a novel approach in the form of a Progressive Dropout Layer (PDL) to address overfitting and empower the MIL model in uncovering intricate and impactful feature representations.
  arXiv  Detail & Related papers  (2023-08-19T21:20:30Z)
• Deep Multiple Instance Learning with Distance-Aware Self-Attention [9.361964965928063]
  We introduce a novel multiple instance learning (MIL) model with distance-aware self-attention (DAS-MIL) Unlike existing relative position representations for self-attention which are discrete, our approach introduces continuous distance-dependent terms into the computation of the attention weights. We evaluate our model on a custom MNIST-based MIL dataset and on CAMELYON16, a publicly available cancer metastasis detection dataset.
  arXiv  Detail & Related papers  (2023-05-17T20:11:43Z)
• DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition [62.95223898214866]
  We explore effective Vision Transformers to pursue a preferable trade-off between the computational complexity and size of the attended receptive field. With a pyramid architecture, we construct a Multi-Scale Dilated Transformer (DilateFormer) by stacking MSDA blocks at low-level stages and global multi-head self-attention blocks at high-level stages. Our experiment results show that our DilateFormer achieves state-of-the-art performance on various vision tasks.
  arXiv  Detail & Related papers  (2023-02-03T14:59:31Z)
• Hierarchical Transformer for Survival Prediction Using Multimodality Whole Slide Images and Genomics [63.76637479503006]
  Learning good representation of giga-pixel level whole slide pathology images (WSI) for downstream tasks is critical. This paper proposes a hierarchical-based multimodal transformer framework that learns a hierarchical mapping between pathology images and corresponding genes. Our architecture requires fewer GPU resources compared with benchmark methods while maintaining better WSI representation ability.
  arXiv  Detail & Related papers  (2022-11-29T23:47:56Z)
• DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification [18.11776334311096]
  Multiple instance learning (MIL) has been increasingly used in the classification of histopathology whole slide images (WSIs) We propose to virtually enlarge the number of bags by introducing the concept of pseudo-bags. We also contribute to deriving the instance probability under the framework of attention-based MIL, and utilize the derivation to help construct and analyze the proposed framework.
  arXiv  Detail & Related papers  (2022-03-22T22:33:42Z)
• CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction [52.94486705393062]
  We go beyond typical multi-instance learning (MIL) framework and propose a novel contrastive instance learning (CIL) framework. Specifically, we regard the initial MIL as the relational triple encoder and constraint positive pairs against negative pairs for each instance. Experiments demonstrate the effectiveness of our proposed framework, with significant improvements over the previous methods on NYT10, GDS and KBP.
  arXiv  Detail & Related papers  (2021-06-21T04:51:59Z)
• TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classication [38.58585442160062]
  Multiple instance learning (MIL) is a powerful tool to solve the weakly supervised classification in whole slide image (WSI) based pathology diagnosis. We proposed a new framework, called correlated MIL, and provided a proof for convergence. We conducted various experiments for three different computational pathology problems and achieved better performance and faster convergence compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-02T02:57:54Z)
• Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning [16.84711797934138]
  We address the challenging problem of whole slide image (WSI) classification. WSI classification can be cast as a multiple instance learning (MIL) problem when only slide-level labels are available. We propose a MIL-based method for WSI classification and tumor detection that does not require localized annotations.
  arXiv  Detail & Related papers  (2020-11-17T20:51:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.