Deformable Attention Graph Representation Learning for Histopathology Whole Slide Image Analysis

• URL: http://arxiv.org/abs/2508.05382v1
• Date: Thu, 07 Aug 2025 13:30:29 GMT
• Title: Deformable Attention Graph Representation Learning for Histopathology Whole Slide Image Analysis
• Authors: Mingxi Fu, Xitong Ling, Yuxuan Chen, Jiawen Li, fanglei fu, Huaitian Yuan, Tian Guan, Yonghong He, Lianghui Zhu,
• Abstract summary: We propose a novel GNN framework with deformable attention for pathology image analysis.<n>We construct a dynamic weighted directed graph based on patch features, where each node aggregates contextual information from its neighbors via attention-weighted edges.<n>Specifically, we incorporate learnable spatial offsets informed by the real coordinates of each patch, enabling the model to adaptively attend to morphologically relevant regions across the slide.
• Score: 9.724220291296927
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate classification of Whole Slide Images (WSIs) and Regions of Interest (ROIs) is a fundamental challenge in computational pathology. While mainstream approaches often adopt Multiple Instance Learning (MIL), they struggle to capture the spatial dependencies among tissue structures. Graph Neural Networks (GNNs) have emerged as a solution to model inter-instance relationships, yet most rely on static graph topologies and overlook the physical spatial positions of tissue patches. Moreover, conventional attention mechanisms lack specificity, limiting their ability to focus on structurally relevant regions. In this work, we propose a novel GNN framework with deformable attention for pathology image analysis. We construct a dynamic weighted directed graph based on patch features, where each node aggregates contextual information from its neighbors via attention-weighted edges. Specifically, we incorporate learnable spatial offsets informed by the real coordinates of each patch, enabling the model to adaptively attend to morphologically relevant regions across the slide. This design significantly enhances the contextual field while preserving spatial specificity. Our framework achieves state-of-the-art performance on four benchmark datasets (TCGA-COAD, BRACS, gastric intestinal metaplasia grading, and intestinal ROI classification), demonstrating the power of deformable attention in capturing complex spatial structures in WSIs and ROIs.

Related papers

• From Pixels to Histopathology: A Graph-Based Framework for Interpretable Whole Slide Image Analysis [81.19923502845441]
  We develop a graph-based framework that constructs WSI graph representations.<n>We build tissue representations (nodes) that follow biological boundaries rather than arbitrary patches.<n>In our method's final step, we solve the diagnostic task through a graph attention network.
  arXiv  Detail & Related papers  (2025-03-14T20:15:04Z)
• Global graph features unveiled by unsupervised geometric deep learning [0.0]
  We introduce GAUDI (Graph Autoencoder Uncovering Descriptive Information), a novel geometric unsupervised deep learning framework.<n>GAUDI employs an innovative hourglass architecture with hierarchical pooling and upsampling layers, linked through skip connections to preserve connectivity information.<n>We demonstrate its power across multiple applications, including modeling small-world networks, characterizing assemblies from super-resolution microscopy, analyzing collective motion in the Vicsek model, and capturing age changes in brain connectivity.
  arXiv  Detail & Related papers  (2025-03-07T16:38:41Z)
• TransGUNet: Transformer Meets Graph-based Skip Connection for Medical Image Segmentation [1.2186950360560143]
  We introduce an attentional cross-scale graph neural network (ACS-GNN) to enhance skip connection framework.<n>ACS-GNN converts cross-scale feature maps into a graph structure and captures complex anatomical structures through node attention.<n>Our framework, TransGUNet, comprises ACS-GNN and EFS-based spatial attentio to enhance domain generalizability across various modalities.
  arXiv  Detail & Related papers  (2025-02-14T05:54:13Z)
• Graph Structure Learning for Spatial-Temporal Imputation: Adapting to Node and Feature Scales [29.499581329290805]
  We introduce the multi-scale Graph Structure Learning framework for spatial-temporal Imputation (GSLI)<n>Our framework encompasses node-scale graph structure learning to cater to the distinct global spatial correlations of different features.<n> integrated with prominence modeling, our framework emphasizes nodes and features with greater significance in the imputation process.
  arXiv  Detail & Related papers  (2024-12-24T16:34:50Z)
• Dynamic Graph Representation with Knowledge-aware Attention for Histopathology Whole Slide Image Analysis [11.353826466710398]
  We propose a novel dynamic graph representation algorithm that conceptualizes WSIs as a form of the knowledge graph structure. Specifically, we dynamically construct neighbors and directed edge embeddings based on the head and tail relationships between instances. Our end-to-end graph representation learning approach has outperformed the state-of-the-art WSI analysis methods on three TCGA benchmark datasets and in-house test sets.
  arXiv  Detail & Related papers  (2024-03-12T14:58:51Z)
• Histopathology Whole Slide Image Analysis with Heterogeneous Graph Representation Learning [78.49090351193269]
  We propose a novel graph-based framework to leverage the inter-relationships among different types of nuclei for WSI analysis. Specifically, we formulate the WSI as a heterogeneous graph with "nucleus-type" attribute to each node and a semantic attribute similarity to each edge. Our framework outperforms the state-of-the-art methods with considerable margins on various tasks.
  arXiv  Detail & Related papers  (2023-07-09T14:43:40Z)
• Distance-aware Molecule Graph Attention Network for Drug-Target Binding Affinity Prediction [54.93890176891602]
  We propose a diStance-aware Molecule graph Attention Network (S-MAN) tailored to drug-target binding affinity prediction. As a dedicated solution, we first propose a position encoding mechanism to integrate the topological structure and spatial position information into the constructed pocket-ligand graph. We also propose a novel edge-node hierarchical attentive aggregation structure which has edge-level aggregation and node-level aggregation.
  arXiv  Detail & Related papers  (2020-12-17T17:44:01Z)
• Multi-Level Graph Convolutional Network with Automatic Graph Learning for Hyperspectral Image Classification [63.56018768401328]
  We propose a Multi-level Graph Convolutional Network (GCN) with Automatic Graph Learning method (MGCN-AGL) for HSI classification. By employing attention mechanism to characterize the importance among spatially neighboring regions, the most relevant information can be adaptively incorporated to make decisions. Our MGCN-AGL encodes the long range dependencies among image regions based on the expressive representations that have been produced at local level.
  arXiv  Detail & Related papers  (2020-09-19T09:26:20Z)
• Structured Landmark Detection via Topology-Adapting Deep Graph Learning [75.20602712947016]
  We present a new topology-adapting deep graph learning approach for accurate anatomical facial and medical landmark detection. The proposed method constructs graph signals leveraging both local image features and global shape features. Experiments are conducted on three public facial image datasets (WFLW, 300W, and COFW-68) as well as three real-world X-ray medical datasets (Cephalometric (public), Hand and Pelvis)
  arXiv  Detail & Related papers  (2020-04-17T11:55:03Z)
• High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification [84.43394420267794]
  We propose a novel framework by learning high-order relation and topology information for discriminative features and robust alignment. Our framework significantly outperforms state-of-the-art by6.5%mAP scores on Occluded-Duke dataset.
  arXiv  Detail & Related papers  (2020-03-18T12:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.