MicroMIL: Graph-based Contextual Multiple Instance Learning for Patient Diagnosis Using Microscopy Images

• URL: http://arxiv.org/abs/2407.21604v1
• Date: Wed, 31 Jul 2024 13:38:47 GMT
• Title: MicroMIL: Graph-based Contextual Multiple Instance Learning for Patient Diagnosis Using Microscopy Images
• Authors: JongWoo Kim, Bryan Wong, YoungSin Ko, MunYong Yi,
• Abstract summary: Whole-slide images (WSIs) produced by scanners with weakly-supervised multiple instance learning (MIL) are costly, memory-intensive, and require extensive analysis time. We introduce MicroMIL, a weakly-supervised MIL framework specifically built to address these challenges. Graph edges are constructed from the upper triangular similarity matrix, with nodes connected to their most similar neighbors, and a graph neural network (GNN) is utilized to capture contextual information.
• Score: 2.324913904215885
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current histopathology research has primarily focused on using whole-slide images (WSIs) produced by scanners with weakly-supervised multiple instance learning (MIL). However, WSIs are costly, memory-intensive, and require extensive analysis time. As an alternative, microscopy-based analysis offers cost and memory efficiency, though microscopy images face issues with unknown absolute positions and redundant images due to multiple captures from the subjective perspectives of pathologists. To this end, we introduce MicroMIL, a weakly-supervised MIL framework specifically built to address these challenges by dynamically clustering images using deep cluster embedding (DCE) and Gumbel Softmax for representative image extraction. Graph edges are then constructed from the upper triangular similarity matrix, with nodes connected to their most similar neighbors, and a graph neural network (GNN) is utilized to capture local and diverse areas of contextual information. Unlike existing graph-based MIL methods designed for WSIs that require absolute positions, MicroMIL efficiently handles the graph edges without this need. Extensive evaluations on real-world colon cancer (Seegene) and public BreakHis datasets demonstrate that MicroMIL outperforms state-of-the-art (SOTA) methods, offering a robust and efficient solution for patient diagnosis using microscopy images. The code is available at https://anonymous.4open.science/r/MicroMIL-6C7C

Related papers

• CARMIL: Context-Aware Regularization on Multiple Instance Learning models for Whole Slide Images [0.41873161228906586]
  Multiple Instance Learning models have proven effective for cancer prognosis from Whole Slide Images. The original MIL formulation incorrectly assumes the patches of the same image to be independent. We propose a versatile regularization scheme designed to seamlessly integrate spatial knowledge into any MIL model.
  arXiv  Detail & Related papers  (2024-08-01T09:59:57Z)
• MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models [56.37780601189795]
  We propose a framework named MamMIL for WSI analysis. We represent each WSI as an undirected graph. To address the problem that Mamba can only process 1D sequences, we propose a topology-aware scanning mechanism.
  arXiv  Detail & Related papers  (2024-03-08T09:02:13Z)
• Style transfer between Microscopy and Magnetic Resonance Imaging via Generative Adversarial Network in small sample size settings [49.84018914962972]
  Cross-modal augmentation of Magnetic Resonance Imaging (MRI) and microscopic imaging based on the same tissue samples is promising. We tested a method for generating microscopic histological images from MRI scans of the corpus callosum using conditional generative adversarial network (cGAN) architecture.
  arXiv  Detail & Related papers  (2023-10-16T13:58:53Z)
• Optimizations of Autoencoders for Analysis and Classification of Microscopic In Situ Hybridization Images [68.8204255655161]
  We propose a deep-learning framework to detect and classify areas of microscopic images with similar levels of gene expression. The data we analyze requires an unsupervised learning model for which we employ a type of Artificial Neural Network - Deep Learning Autoencoders.
  arXiv  Detail & Related papers  (2023-04-19T13:45:28Z)
• Cross-scale Multi-instance Learning for Pathological Image Diagnosis [20.519711186151635]
  Multi-instance learning (MIL) is a common solution for working with high resolution images by classifying bags of objects. We propose a novel cross-scale MIL algorithm to explicitly aggregate inter-scale relationships into a single MIL network for pathological image diagnosis.
  arXiv  Detail & Related papers  (2023-04-01T03:52:52Z)
• Active Learning Enhances Classification of Histopathology Whole Slide Images with Attention-based Multiple Instance Learning [48.02011627390706]
  We train an attention-based MIL and calculate a confidence metric for every image in the dataset to select the most uncertain WSIs for expert annotation. With a novel attention guiding loss, this leads to an accuracy boost of the trained models with few regions annotated for each class. It may in the future serve as an important contribution to train MIL models in the clinically relevant context of cancer classification in histopathology.
  arXiv  Detail & Related papers  (2023-03-02T15:18:58Z)
• AMIGO: Sparse Multi-Modal Graph Transformer with Shared-Context Processing for Representation Learning of Giga-pixel Images [53.29794593104923]
  We present a novel concept of shared-context processing for whole slide histopathology images. AMIGO uses the celluar graph within the tissue to provide a single representation for a patient. We show that our model is strongly robust to missing information to an extent that it can achieve the same performance with as low as 20% of the data.
  arXiv  Detail & Related papers  (2023-03-01T23:37:45Z)
• Hierarchical Transformer for Survival Prediction Using Multimodality Whole Slide Images and Genomics [63.76637479503006]
  Learning good representation of giga-pixel level whole slide pathology images (WSI) for downstream tasks is critical. This paper proposes a hierarchical-based multimodal transformer framework that learns a hierarchical mapping between pathology images and corresponding genes. Our architecture requires fewer GPU resources compared with benchmark methods while maintaining better WSI representation ability.
  arXiv  Detail & Related papers  (2022-11-29T23:47:56Z)
• Histopathological Image Classification based on Self-Supervised Vision Transformer and Weak Labels [16.865729758055448]
  We propose Self-ViT-MIL, a novel approach for classifying and localizing cancerous areas based on slide-level annotations. Self-ViT-MIL surpasses existing state-of-the-art MIL-based approaches in terms of accuracy and area under the curve.
  arXiv  Detail & Related papers  (2022-10-17T12:43:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.