Integrative Graph-Transformer Framework for Histopathology Whole Slide Image Representation and Classification

• URL: http://arxiv.org/abs/2403.18134v1
• Date: Tue, 26 Mar 2024 22:31:05 GMT
• Title: Integrative Graph-Transformer Framework for Histopathology Whole Slide Image Representation and Classification
• Authors: Zhan Shi, Jingwei Zhang, Jun Kong, Fusheng Wang,
• Abstract summary: In digital pathology, the multiple instance learning (MIL) strategy is widely used in the weakly supervised histopathology whole slide image (WSI) classification task. Existing attention-based MIL approaches often overlook contextual information and intrinsic spatial relationships between neighboring tissue tiles. We introduce an integrative graph-transformer framework that simultaneously captures the context-aware relational features and global WSI representations.
• Score: 18.16710321320098
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In digital pathology, the multiple instance learning (MIL) strategy is widely used in the weakly supervised histopathology whole slide image (WSI) classification task where giga-pixel WSIs are only labeled at the slide level. However, existing attention-based MIL approaches often overlook contextual information and intrinsic spatial relationships between neighboring tissue tiles, while graph-based MIL frameworks have limited power to recognize the long-range dependencies. In this paper, we introduce the integrative graph-transformer framework that simultaneously captures the context-aware relational features and global WSI representations through a novel Graph Transformer Integration (GTI) block. Specifically, each GTI block consists of a Graph Convolutional Network (GCN) layer modeling neighboring relations at the local instance level and an efficient global attention model capturing comprehensive global information from extensive feature embeddings. Extensive experiments on three publicly available WSI datasets: TCGA-NSCLC, TCGA-RCC and BRIGHT, demonstrate the superiority of our approach over current state-of-the-art MIL methods, achieving an improvement of 1.0% to 2.6% in accuracy and 0.7%-1.6% in AUROC.

Related papers

• Hierarchical Multi-Scale Graph Learning with Knowledge-Guided Attention for Whole-Slide Image Survival Analysis [7.719549259296449]
  Hierarchical Multi-scale Knowledge-aware Graph Network (HMKGN) models multi-scale interactions and spatially hierarchical relationships within whole-slide images (WSIs) for cancer prognostication.<n>We evaluate HMKGN on four TCGA cohorts (KIRC, LGG, PAAD, and STAD) for survival prediction.
  arXiv  Detail & Related papers  (2026-02-26T23:47:27Z)
• Hierarchical Graph Feature Enhancement with Adaptive Frequency Modulation for Visual Recognition [6.580655899524989]
  Convolutional neural networks (CNNs) have demonstrated strong performance in visual recognition tasks.<n>We propose a novel framework that integrates graph-based rea soning into CNNs to enhance both structural awareness and feature representation.<n>The proposed HGFE module is lightweight, end-to-end trainable, and can be seamlessly integrated into standard CNN backbone networks.
  arXiv  Detail & Related papers  (2025-08-15T14:19:50Z)
• GCRPNet: Graph-Enhanced Contextual and Regional Perception Network for Salient Object Detection in Optical Remote Sensing Images [68.33481681452675]
  We propose a graph-enhanced contextual and regional perception network (GCRPNet)<n>It builds upon the Mamba architecture to simultaneously capture long-range dependencies and enhance regional feature representation.<n>It performs adaptive patch scanning on feature maps processed via multi-scale convolutions, thereby capturing rich local region information.
  arXiv  Detail & Related papers  (2025-08-14T11:31:43Z)
• Deformable Attention Graph Representation Learning for Histopathology Whole Slide Image Analysis [9.724220291296927]
  We propose a novel GNN framework with deformable attention for pathology image analysis.<n>We construct a dynamic weighted directed graph based on patch features, where each node aggregates contextual information from its neighbors via attention-weighted edges.<n>Specifically, we incorporate learnable spatial offsets informed by the real coordinates of each patch, enabling the model to adaptively attend to morphologically relevant regions across the slide.
  arXiv  Detail & Related papers  (2025-08-07T13:30:29Z)
• Structural-Spectral Graph Convolution with Evidential Edge Learning for Hyperspectral Image Clustering [59.24638672786966]
  Hyperspectral image (HSI) clustering assigns similar pixels to the same class without any annotations.<n>Existing graph neural networks (GNNs) cannot fully exploit the spectral information of the input HSI.<n>We propose a structural-spectral graph convolutional operator (SSGCO) tailored for graph-structured HSI superpixels.
  arXiv  Detail & Related papers  (2025-06-11T16:41:34Z)
• G-MSGINet: A Grouped Multi-Scale Graph-Involution Network for Contactless Fingerprint Recognition [20.458766184257147]
  G-MSGINet is a unified framework for robust contactless fingerprint recognition.<n>It jointly performs minutiae localization and identity embedding directly from raw input images.<n>Extensive experiments on three benchmark datasets show G-MSGINet consistently achieves minutiae F1-scores in the range of $0.83pm0.02$ and Rank-1 identification accuracies between 97.0% and 99.1%.
  arXiv  Detail & Related papers  (2025-05-13T05:24:24Z)
• The Role of Graph-based MIL and Interventional Training in the Generalization of WSI Classifiers [8.867734798489037]
  Whole Slide Imaging (WSI), which involves high-resolution digital scans of pathology slides, has become the gold standard for cancer diagnosis. Its gigapixel resolution and the scarcity of annotated datasets present challenges for deep learning models. We introduce a new framework, Graph-based Multiple Instance Learning with Interventional Training (GMIL-IT) for WSI classification.
  arXiv  Detail & Related papers  (2025-01-31T11:21:08Z)
• Combining Graph Neural Network and Mamba to Capture Local and Global Tissue Spatial Relationships in Whole Slide Images [1.1813933389519358]
  In computational pathology, extracting spatial features from gigapixel whole slide images (WSIs) is a fundamental task. We introduce a model that combines a message-passing graph neural network (GNN) with a state space model (Mamba) to capture both local and global spatial relationships. The model's effectiveness was demonstrated in predicting progression-free survival among patients with early-stage lung adenocarcinomas.
  arXiv  Detail & Related papers  (2024-06-05T22:06:57Z)
• Multi-level Graph Subspace Contrastive Learning for Hyperspectral Image Clustering [16.027471624621924]
  This study proposes a multi-level graph subspace contrastive learning (MLGSC) for HSI clustering. The proposed model is evaluated on four popular HSI datasets: Indian Pines, Pavia University, Houston, and Xu Zhou. The overall accuracies are 97.75%, 99.96%, 92.28%, and 95.73%, which significantly outperforms the current state-of-the-art clustering methods.
  arXiv  Detail & Related papers  (2024-04-08T05:50:46Z)
• Dynamic Graph Representation with Knowledge-aware Attention for Histopathology Whole Slide Image Analysis [11.353826466710398]
  We propose a novel dynamic graph representation algorithm that conceptualizes WSIs as a form of the knowledge graph structure. Specifically, we dynamically construct neighbors and directed edge embeddings based on the head and tail relationships between instances. Our end-to-end graph representation learning approach has outperformed the state-of-the-art WSI analysis methods on three TCGA benchmark datasets and in-house test sets.
  arXiv  Detail & Related papers  (2024-03-12T14:58:51Z)
• MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models [56.37780601189795]
  We propose a framework named MamMIL for WSI analysis. We represent each WSI as an undirected graph. To address the problem that Mamba can only process 1D sequences, we propose a topology-aware scanning mechanism.
  arXiv  Detail & Related papers  (2024-03-08T09:02:13Z)
• Efficient Multi-View Graph Clustering with Local and Global Structure Preservation [59.49018175496533]
  We propose a novel anchor-based multi-view graph clustering framework termed Efficient Multi-View Graph Clustering with Local and Global Structure Preservation (EMVGC-LG) Specifically, EMVGC-LG jointly optimize anchor construction and graph learning to enhance the clustering quality. In addition, EMVGC-LG inherits the linear complexity of existing AMVGC methods respecting the sample number.
  arXiv  Detail & Related papers  (2023-08-31T12:12:30Z)
• Histopathology Whole Slide Image Analysis with Heterogeneous Graph Representation Learning [78.49090351193269]
  We propose a novel graph-based framework to leverage the inter-relationships among different types of nuclei for WSI analysis. Specifically, we formulate the WSI as a heterogeneous graph with "nucleus-type" attribute to each node and a semantic attribute similarity to each edge. Our framework outperforms the state-of-the-art methods with considerable margins on various tasks.
  arXiv  Detail & Related papers  (2023-07-09T14:43:40Z)
• Spatial-Spectral Clustering with Anchor Graph for Hyperspectral Image [88.60285937702304]
  This paper proposes a novel unsupervised approach called spatial-spectral clustering with anchor graph (SSCAG) for HSI data clustering. The proposed SSCAG is competitive against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-04-24T08:09:27Z)
• Spatial-spectral Hyperspectral Image Classification via Multiple Random Anchor Graphs Ensemble Learning [88.60285937702304]
  This paper proposes a novel spatial-spectral HSI classification method via multiple random anchor graphs ensemble learning (RAGE) Firstly, the local binary pattern is adopted to extract the more descriptive features on each selected band, which preserves local structures and subtle changes of a region. Secondly, the adaptive neighbors assignment is introduced in the construction of anchor graph, to reduce the computational complexity.
  arXiv  Detail & Related papers  (2021-03-25T09:31:41Z)
• Multi-Level Graph Convolutional Network with Automatic Graph Learning for Hyperspectral Image Classification [63.56018768401328]
  We propose a Multi-level Graph Convolutional Network (GCN) with Automatic Graph Learning method (MGCN-AGL) for HSI classification. By employing attention mechanism to characterize the importance among spatially neighboring regions, the most relevant information can be adaptively incorporated to make decisions. Our MGCN-AGL encodes the long range dependencies among image regions based on the expressive representations that have been produced at local level.
  arXiv  Detail & Related papers  (2020-09-19T09:26:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.