Prototype-Based Multiple Instance Learning for Gigapixel Whole Slide Image Classification

• URL: http://arxiv.org/abs/2503.08384v1
• Date: Tue, 11 Mar 2025 12:44:03 GMT
• Title: Prototype-Based Multiple Instance Learning for Gigapixel Whole Slide Image Classification
• Authors: Susu Sun, Dominique van Midden, Geert Litjens, Christian F. Baumgartner,
• Abstract summary: ProtoMIL is an inherently interpretable MIL model for whole slide image (WSI) analysis.<n>Our approach employs a sparse autoencoder to discover human-interpretable concepts from the image feature space.<n>ProtoMIL allows users to perform model interventions by altering the input concepts.
• Score: 3.9817342822800916
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Multiple Instance Learning (MIL) methods have succeeded remarkably in histopathology whole slide image (WSI) analysis. However, most MIL models only offer attention-based explanations that do not faithfully capture the model's decision mechanism and do not allow human-model interaction. To address these limitations, we introduce ProtoMIL, an inherently interpretable MIL model for WSI analysis that offers user-friendly explanations and supports human intervention. Our approach employs a sparse autoencoder to discover human-interpretable concepts from the image feature space, which are then used to train ProtoMIL. The model represents predictions as linear combinations of concepts, making the decision process transparent. Furthermore, ProtoMIL allows users to perform model interventions by altering the input concepts. Experiments on two widely used pathology datasets demonstrate that ProtoMIL achieves a classification performance comparable to state-of-the-art MIL models while offering intuitively understandable explanations. Moreover, we demonstrate that our method can eliminate reliance on diagnostically irrelevant information via human intervention, guiding the model toward being right for the right reason. Code will be publicly available at https://github.com/ss-sun/ProtoMIL.

Related papers

• Label-free Concept Based Multiple Instance Learning for Gigapixel Histopathology [3.3353153750791718]
  Multiple Instance Learning (MIL) methods allow for gigapixel Whole-Slide Image (WSI) analysis with only slide-level annotations.<n>We propose a novel inherently interpretable WSI-classification approach that uses human-understandable pathology concepts to generate explanations.
  arXiv  Detail & Related papers  (2025-01-06T11:03:04Z)
• xMIL: Insightful Explanations for Multiple Instance Learning in Histopathology [13.939494815120666]
  Multiple instance learning (MIL) is an effective and widely used approach for weakly supervised machine learning.<n>We revisit MIL through the lens of explainable AI (XAI) and introduce xMIL, a refined framework with more general assumptions.<n>Our approach consistently outperforms previous explanation attempts with particularly improved faithfulness scores on challenging biomarker prediction tasks.
  arXiv  Detail & Related papers  (2024-06-06T17:26:40Z)
• DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perception [66.88792390480343]
  We propose DEEM, a simple but effective approach that utilizes the generative feedback of diffusion models to align the semantic distributions of the image encoder.<n>DEEM exhibits enhanced robustness and a superior capacity to alleviate model hallucinations while utilizing fewer trainable parameters, less pre-training data, and a smaller base model size.
  arXiv  Detail & Related papers  (2024-05-24T05:46:04Z)
• Prompt-Guided Adaptive Model Transformation for Whole Slide Image Classification [27.21493446754789]
  Multiple instance learning (MIL) has emerged as a popular method for classifying histopathology whole slide images (WSIs) We propose Prompt-guided Adaptive Model Transformation framework that seamlessly adapts pre-trained models to the specific characteristics of histopathology data. We rigorously evaluate our approach on two datasets, Camelyon16 and TCGA-NSCLC, showcasing substantial improvements across various MIL models.
  arXiv  Detail & Related papers  (2024-03-19T08:23:12Z)
• MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models [56.37780601189795]
  We propose a framework named MamMIL for WSI analysis. We represent each WSI as an undirected graph. To address the problem that Mamba can only process 1D sequences, we propose a topology-aware scanning mechanism.
  arXiv  Detail & Related papers  (2024-03-08T09:02:13Z)
• Manipulating Feature Visualizations with Gradient Slingshots [54.31109240020007]
  We introduce a novel method for manipulating Feature Visualization (FV) without significantly impacting the model's decision-making process. We evaluate the effectiveness of our method on several neural network models and demonstrate its capabilities to hide the functionality of arbitrarily chosen neurons.
  arXiv  Detail & Related papers  (2024-01-11T18:57:17Z)
• Diffusion models for probabilistic programming [56.47577824219207]
  Diffusion Model Variational Inference (DMVI) is a novel method for automated approximate inference in probabilistic programming languages (PPLs) DMVI is easy to implement, allows hassle-free inference in PPLs without the drawbacks of, e.g., variational inference using normalizing flows, and does not make any constraints on the underlying neural network model.
  arXiv  Detail & Related papers  (2023-11-01T12:17:05Z)
• Introducing instance label correlation in multiple instance learning. Application to cancer detection on histopathological images [5.895585247199983]
  In this work, we extend a state-of-the-art GP-based MIL method, which is called VGPMIL-PR, to exploit such correlation. We show that our model achieves better results than other state-of-the-art probabilistic MIL methods.
  arXiv  Detail & Related papers  (2023-10-30T08:57:59Z)
• Reproducibility in Multiple Instance Learning: A Case For Algorithmic Unit Tests [59.623267208433255]
  Multiple Instance Learning (MIL) is a sub-domain of classification problems with positive and negative labels and a "bag" of inputs. In this work, we examine five of the most prominent deep-MIL models and find that none of them respects the standard MIL assumption. We identify and demonstrate this problem via a proposed "algorithmic unit test", where we create synthetic datasets that can be solved by a MIL respecting model.
  arXiv  Detail & Related papers  (2023-10-27T03:05:11Z)
• ProtoMIL: Multiple Instance Learning with Prototypical Parts for Fine-Grained Interpretability [2.094672430475796]
  Multiple Instance Learning (MIL) gains popularity in many real-life machine learning applications due to its weakly supervised nature. In this paper, we introduce ProtoMIL, a novel self-explainable MIL method inspired by the case-based reasoning process that operates on visual prototypes. Thanks to incorporating prototypical features into objects description, ProtoMIL unprecedentedly joins the model accuracy and fine-grained interpretability.
  arXiv  Detail & Related papers  (2021-08-24T10:02:31Z)
• Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations [64.85696493596821]
  In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction. We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss. Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-18T12:57:34Z)
• Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resources [78.72922528736011]
  We propose a novel approach, black-box adversarial reprogramming (BAR), that repurposes a well-trained black-box machine learning model. Using zeroth order optimization and multi-label mapping techniques, BAR can reprogram a black-box ML model solely based on its input-output responses. BAR outperforms state-of-the-art methods and yields comparable performance to the vanilla adversarial reprogramming method.
  arXiv  Detail & Related papers  (2020-07-17T01:52:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.