PATHS: A Hierarchical Transformer for Efficient Whole Slide Image Analysis

• URL: http://arxiv.org/abs/2411.18225v1
• Date: Wed, 27 Nov 2024 11:03:38 GMT
• Title: PATHS: A Hierarchical Transformer for Efficient Whole Slide Image Analysis
• Authors: Zak Buzzard, Konstantin Hemker, Nikola Simidjievski, Mateja Jamnik,
• Abstract summary: We propose a novel top-down method for hierarchical weakly supervised representation learning on slide-level tasks in computational pathology. PATHS is inspired by the cross-magnification manner in which a human pathologist examines a slide, filtering patches at each magnification level to a small subset relevant to the diagnosis. We apply PATHS to five datasets of The Cancer Genome Atlas (TCGA), and achieve superior performance on slide-level prediction tasks.
• Score: 9.862551438475666
• License:
• Abstract: Computational analysis of whole slide images (WSIs) has seen significant research progress in recent years, with applications ranging across important diagnostic and prognostic tasks such as survival or cancer subtype prediction. Many state-of-the-art models process the entire slide - which may be as large as $150,000 \times 150,000$ pixels - as a bag of many patches, the size of which necessitates computationally cheap feature aggregation methods. However, a large proportion of these patches are uninformative, such as those containing only healthy or adipose tissue, adding significant noise and size to the bag. We propose Pathology Transformer with Hierarchical Selection (PATHS), a novel top-down method for hierarchical weakly supervised representation learning on slide-level tasks in computational pathology. PATHS is inspired by the cross-magnification manner in which a human pathologist examines a slide, recursively filtering patches at each magnification level to a small subset relevant to the diagnosis. Our method overcomes the complications of processing the entire slide, enabling quadratic self-attention and providing a simple interpretable measure of region importance. We apply PATHS to five datasets of The Cancer Genome Atlas (TCGA), and achieve superior performance on slide-level prediction tasks when compared to previous methods, despite processing only a small proportion of the slide.

Related papers

• Semantic Segmentation Based Quality Control of Histopathology Whole Slide Images [2.953447779233234]
  We developed a software pipeline for quality control (QC) of histopathology whole slide images (WSIs) It segments various regions, such as blurs of different levels, tissue regions, tissue folds, and pen marks. It was evaluated in all TCGAs, which is the largest publicly available WSI dataset containing more than 11,000 histopathology images from 28 organs.
  arXiv  Detail & Related papers  (2024-10-04T10:03:04Z)
• Towards a Benchmark for Colorectal Cancer Segmentation in Endorectal Ultrasound Videos: Dataset and Model Development [59.74920439478643]
  In this paper, we collect and annotated the first benchmark dataset that covers diverse ERUS scenarios. Our ERUS-10K dataset comprises 77 videos and 10,000 high-resolution annotated frames. We introduce a benchmark model for colorectal cancer segmentation, named the Adaptive Sparse-context TRansformer (ASTR)
  arXiv  Detail & Related papers  (2024-08-19T15:04:42Z)
• A self-supervised framework for learning whole slide representations [52.774822784847565]
  We present Slide Pre-trained Transformers (SPT) for gigapixel-scale self-supervision of whole slide images. We benchmark SPT visual representations on five diagnostic tasks across three biomedical microscopy datasets.
  arXiv  Detail & Related papers  (2024-02-09T05:05:28Z)
• Active Learning Enhances Classification of Histopathology Whole Slide Images with Attention-based Multiple Instance Learning [48.02011627390706]
  We train an attention-based MIL and calculate a confidence metric for every image in the dataset to select the most uncertain WSIs for expert annotation. With a novel attention guiding loss, this leads to an accuracy boost of the trained models with few regions annotated for each class. It may in the future serve as an important contribution to train MIL models in the clinically relevant context of cancer classification in histopathology.
  arXiv  Detail & Related papers  (2023-03-02T15:18:58Z)
• AMIGO: Sparse Multi-Modal Graph Transformer with Shared-Context Processing for Representation Learning of Giga-pixel Images [53.29794593104923]
  We present a novel concept of shared-context processing for whole slide histopathology images. AMIGO uses the celluar graph within the tissue to provide a single representation for a patient. We show that our model is strongly robust to missing information to an extent that it can achieve the same performance with as low as 20% of the data.
  arXiv  Detail & Related papers  (2023-03-01T23:37:45Z)
• RLogist: Fast Observation Strategy on Whole-slide Images with Deep Reinforcement Learning [15.955265218706467]
  Whole-slide images (WSI) in computational pathology have high resolution with gigapixel size, but are generally with sparse regions of interest. We develop RLogist, a deep reinforcement learning (DRL) method for fast observation strategy on WSIs.
  arXiv  Detail & Related papers  (2022-12-04T04:03:34Z)
• Hierarchical Transformer for Survival Prediction Using Multimodality Whole Slide Images and Genomics [63.76637479503006]
  Learning good representation of giga-pixel level whole slide pathology images (WSI) for downstream tasks is critical. This paper proposes a hierarchical-based multimodal transformer framework that learns a hierarchical mapping between pathology images and corresponding genes. Our architecture requires fewer GPU resources compared with benchmark methods while maintaining better WSI representation ability.
  arXiv  Detail & Related papers  (2022-11-29T23:47:56Z)
• Fast whole-slide cartography in colon cancer histology using superpixels and CNN classification [0.22312377591335414]
  Whole-slide-images typically have to be divided into smaller patches which are then analyzed individually using machine learning-based approaches. We propose to subdivide the image into coherent regions prior to classification by grouping visually similar adjacent image pixels into larger segments, i.e. superpixels. The algorithm has been developed and validated on a dataset of 159 hand-annotated whole-slide-images of colon resections and its performance has been compared to a standard patch-based approach.
  arXiv  Detail & Related papers  (2021-06-30T08:34:06Z)
• Wide & Deep neural network model for patch aggregation in CNN-based prostate cancer detection systems [51.19354417900591]
  Prostate cancer (PCa) is one of the leading causes of death among men, with almost 1.41 million new cases and around 375,000 deaths in 2020. To perform an automatic diagnosis, prostate tissue samples are first digitized into gigapixel-resolution whole-slide images. Small subimages called patches are extracted and predicted, obtaining a patch-level classification.
  arXiv  Detail & Related papers  (2021-05-20T18:13:58Z)
• Overcoming the limitations of patch-based learning to detect cancer in whole slide images [0.15658704610960567]
  Whole slide images (WSIs) pose unique challenges when training deep learning models. We outline the differences between patch or slide-level classification versus methods that need to localize or segment cancer accurately across the whole slide. We propose a negative data sampling strategy, which drastically reduces the false positive rate and improves each metric pertinent to our problem.
  arXiv  Detail & Related papers  (2020-12-01T16:37:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.