Promptable Representation Distribution Learning and Data Augmentation for Gigapixel Histopathology WSI Analysis

• URL: http://arxiv.org/abs/2412.14473v1
• Date: Thu, 19 Dec 2024 02:47:17 GMT
• Title: Promptable Representation Distribution Learning and Data Augmentation for Gigapixel Histopathology WSI Analysis
• Authors: Kunming Tang, Zhiguo Jiang, Jun Shi, Wei Wang, Haibo Wu, Yushan Zheng,
• Abstract summary: We propose a Promptable Representation Distribution Learning framework (PRDL) for both patch-level representation learning and WSI-level data augmentation. The proposed method stably outperforms state-of-the-art methods.
• Score: 7.823674912857107
• License:
• Abstract: Gigapixel image analysis, particularly for whole slide images (WSIs), often relies on multiple instance learning (MIL). Under the paradigm of MIL, patch image representations are extracted and then fixed during the training of the MIL classifiers for efficiency consideration. However, the invariance of representations makes it difficult to perform data augmentation for WSI-level model training, which significantly limits the performance of the downstream WSI analysis. The current data augmentation methods for gigapixel images either introduce additional computational costs or result in a loss of semantic information, which is hard to meet the requirements for efficiency and stability needed for WSI model training. In this paper, we propose a Promptable Representation Distribution Learning framework (PRDL) for both patch-level representation learning and WSI-level data augmentation. Meanwhile, we explore the use of prompts to guide data augmentation in feature space, which achieves promptable data augmentation for training robust WSI-level models. The experimental results have demonstrated that the proposed method stably outperforms state-of-the-art methods.

Related papers

• Adversarial Semantic Augmentation for Training Generative Adversarial Networks under Limited Data [27.27230943686822]
  We propose an adversarial semantic augmentation (ASA) technique to enlarge the training data at the semantic level instead of the image level. Our method consistently improve the synthesis quality under various data regimes.
  arXiv  Detail & Related papers  (2025-02-02T13:50:38Z)
• Global Semantic-Guided Sub-image Feature Weight Allocation in High-Resolution Large Vision-Language Models [50.98559225639266]
  Sub-images with higher semantic relevance to the entire image encapsulate richer visual information for preserving the model's visual understanding ability. Global Semantic-guided Weight Allocator (GSWA) module allocates weights to sub-images based on their relative information density. SleighVL, a lightweight yet high-performing model, outperforms models with comparable parameters and remains competitive with larger models.
  arXiv  Detail & Related papers  (2025-01-24T06:42:06Z)
• Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration. We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• A self-supervised framework for learning whole slide representations [52.774822784847565]
  We present Slide Pre-trained Transformers (SPT) for gigapixel-scale self-supervision of whole slide images. We benchmark SPT visual representations on five diagnostic tasks across three biomedical microscopy datasets.
  arXiv  Detail & Related papers  (2024-02-09T05:05:28Z)
• Diffusion-based generation of Histopathological Whole Slide Images at a Gigapixel scale [10.481781668319886]
  Synthetic Whole Slide Images (WSIs) can augment training datasets to enhance the performance of many computational applications. No existing deep-learning-based method generates WSIs at their typically high resolutions. We present a novel coarse-to-fine sampling scheme to tackle image generation of high-resolution WSIs.
  arXiv  Detail & Related papers  (2023-11-14T14:33:39Z)
• A Dual-branch Self-supervised Representation Learning Framework for Tumour Segmentation in Whole Slide Images [12.961686610789416]
  Self-supervised learning (SSL) has emerged as an alternative solution to reduce the annotation overheads in whole slide images. These SSL approaches are not designed for handling multi-resolution WSIs, which limits their performance in learning discriminative image features. We propose a Dual-branch SSL Framework for WSI tumour segmentation (DSF-WSI) that can effectively learn image features from multi-resolution WSIs.
  arXiv  Detail & Related papers  (2023-03-20T10:57:28Z)
• Task-specific Fine-tuning via Variational Information Bottleneck for Weakly-supervised Pathology Whole Slide Image Classification [10.243293283318415]
  Multiple Instance Learning (MIL) has shown promising results in digital Pathology Whole Slide Image (WSI) classification. We propose an efficient WSI fine-tuning framework motivated by the Information Bottleneck theory. Our framework is evaluated on five pathology WSI datasets on various WSI heads.
  arXiv  Detail & Related papers  (2023-03-15T08:41:57Z)
• Embedding Space Augmentation for Weakly Supervised Learning in Whole-Slide Images [3.858809922365453]
  Multiple Instance Learning (MIL) is a widely employed framework for learning on gigapixel whole-slide images (WSIs) from WSI-level annotations. We present EmbAugmenter, a data augmentation generative adversarial network (DA-GAN) that can synthesize data augmentations in the embedding space rather than in the pixel space. Our approach outperforms MIL without augmentation and is on par with traditional patch-level augmentation for MIL training while being substantially faster.
  arXiv  Detail & Related papers  (2022-10-31T02:06:39Z)
• Learning Representational Invariances for Data-Efficient Action Recognition [52.23716087656834]
  We show that our data augmentation strategy leads to promising performance on the Kinetics-100, UCF-101, and HMDB-51 datasets. We also validate our data augmentation strategy in the fully supervised setting and demonstrate improved performance.
  arXiv  Detail & Related papers  (2021-03-30T17:59:49Z)
• Data Augmentation for Meta-Learning [58.47185740820304]
  meta-learning algorithms sample data, query data, and tasks on each training step. Data augmentation can be used not only to expand the number of images available per class, but also to generate entirely new classes/tasks. Our proposed meta-specific data augmentation significantly improves the performance of meta-learners on few-shot classification benchmarks.
  arXiv  Detail & Related papers  (2020-10-14T13:48:22Z)
• Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond [62.730497582218284]
  We develop a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data.
  arXiv  Detail & Related papers  (2020-04-30T03:23:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.