Prompt-Guided Adaptive Model Transformation for Whole Slide Image Classification

• URL: http://arxiv.org/abs/2403.12537v1
• Date: Tue, 19 Mar 2024 08:23:12 GMT
• Title: Prompt-Guided Adaptive Model Transformation for Whole Slide Image Classification
• Authors: Yi Lin, Zhengjie Zhu, Kwang-Ting Cheng, Hao Chen,
• Abstract summary: Multiple instance learning (MIL) has emerged as a popular method for classifying histopathology whole slide images (WSIs) We propose Prompt-guided Adaptive Model Transformation framework that seamlessly adapts pre-trained models to the specific characteristics of histopathology data. We rigorously evaluate our approach on two datasets, Camelyon16 and TCGA-NSCLC, showcasing substantial improvements across various MIL models.
• Score: 27.21493446754789
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multiple instance learning (MIL) has emerged as a popular method for classifying histopathology whole slide images (WSIs). Existing approaches typically rely on frozen pre-trained models to extract instance features, neglecting the substantial domain shift between pre-training natural and histopathological images. To address this issue, we propose PAMT, a novel Prompt-guided Adaptive Model Transformation framework that enhances MIL classification performance by seamlessly adapting pre-trained models to the specific characteristics of histopathology data. To capture the intricate histopathology distribution, we introduce Representative Patch Sampling (RPS) and Prototypical Visual Prompt (PVP) to reform the input data, building a compact while informative representation. Furthermore, to narrow the domain gap, we introduce Adaptive Model Transformation (AMT) that integrates adapter blocks within the feature extraction pipeline, enabling the pre-trained models to learn domain-specific features. We rigorously evaluate our approach on two publicly available datasets, Camelyon16 and TCGA-NSCLC, showcasing substantial improvements across various MIL models. Our findings affirm the potential of PAMT to set a new benchmark in WSI classification, underscoring the value of a targeted reprogramming approach.

Related papers

• Foundation Models for Slide-level Cancer Subtyping in Digital Pathology [1.7641392161755438]
  This work aims to compare the performance of various feature extractors developed under different pretraining strategies for cancer subtyping on WSI under a MIL framework. Results demonstrate the ability of foundation models to surpass ImageNet-pretrained models for the prediction of six skin cancer subtypes.
  arXiv  Detail & Related papers  (2024-10-21T11:04:58Z)
• Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixture [58.60915132222421]
  We introduce an approach that is both general and parameter-efficient for face forgery detection. We design a forgery-style mixture formulation that augments the diversity of forgery source domains. We show that the designed model achieves state-of-the-art generalizability with significantly reduced trainable parameters.
  arXiv  Detail & Related papers  (2024-08-23T01:53:36Z)
• Generalized Face Forgery Detection via Adaptive Learning for Pre-trained Vision Transformer [54.32283739486781]
  We present a textbfForgery-aware textbfAdaptive textbfVision textbfTransformer (FA-ViT) under the adaptive learning paradigm. FA-ViT achieves 93.83% and 78.32% AUC scores on Celeb-DF and DFDC datasets in the cross-dataset evaluation.
  arXiv  Detail & Related papers  (2023-09-20T06:51:11Z)
• Consistency Regularization for Generalizable Source-free Domain Adaptation [62.654883736925456]
  Source-free domain adaptation (SFDA) aims to adapt a well-trained source model to an unlabelled target domain without accessing the source dataset. Existing SFDA methods ONLY assess their adapted models on the target training set, neglecting the data from unseen but identically distributed testing sets. We propose a consistency regularization framework to develop a more generalizable SFDA method.
  arXiv  Detail & Related papers  (2023-08-03T07:45:53Z)
• Exploring Visual Prompts for Whole Slide Image Classification with Multiple Instance Learning [25.124855361054763]
  We present a novel, simple yet effective method for learning domain-specific knowledge transformation from pre-trained models to histopathology images. Our approach entails using a prompt component to assist the pre-trained model in discerning differences between the pre-trained dataset and the target histopathology dataset.
  arXiv  Detail & Related papers  (2023-03-23T09:23:52Z)
• Domain-Specific Pre-training Improves Confidence in Whole Slide Image Classification [15.354256205808273]
  Whole Slide Images (WSIs) or histopathology images are used in digital pathology. WSIs pose great challenges to deep learning models for clinical diagnosis.
  arXiv  Detail & Related papers  (2023-02-20T08:42:06Z)
• From Modern CNNs to Vision Transformers: Assessing the Performance, Robustness, and Classification Strategies of Deep Learning Models in Histopathology [1.8947504307591034]
  We develop a new methodology to extensively evaluate a wide range of classification models. We thoroughly tested the models on five widely used histopathology datasets. We extend existing interpretability methods and systematically reveal insights of the models' classifications strategies.
  arXiv  Detail & Related papers  (2022-04-11T12:26:19Z)
• Dynamic multi feature-class Gaussian process models [0.0]
  This study presents a statistical modelling method for automatic learning of shape, pose and intensity features in medical images. A DMFC-GPM is a Gaussian process (GP)-based model with a shared latent space that encodes linear and non-linear variation. The model performance results suggest that this new modelling paradigm is robust, accurate, accessible, and has potential applications.
  arXiv  Detail & Related papers  (2021-12-08T15:12:47Z)
• Semantic Correspondence with Transformers [68.37049687360705]
  We propose Cost Aggregation with Transformers (CATs) to find dense correspondences between semantically similar images. We include appearance affinity modelling to disambiguate the initial correlation maps and multi-level aggregation. We conduct experiments to demonstrate the effectiveness of the proposed model over the latest methods and provide extensive ablation studies.
  arXiv  Detail & Related papers  (2021-06-04T14:39:03Z)
• Shape-aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains [68.73614619875814]
  We present a novel shape-aware meta-learning scheme to improve the model generalization in prostate MRI segmentation. Experimental results show that our approach outperforms many state-of-the-art generalization methods consistently across all six settings of unseen domains.
  arXiv  Detail & Related papers  (2020-07-04T07:56:02Z)
• FLAT: Few-Shot Learning via Autoencoding Transformation Regularizers [67.46036826589467]
  We present a novel regularization mechanism by learning the change of feature representations induced by a distribution of transformations without using the labels of data examples. It could minimize the risk of overfitting into base categories by inspecting the transformation-augmented variations at the encoded feature level. Experiment results show the superior performances to the current state-of-the-art methods in literature.
  arXiv  Detail & Related papers  (2019-12-29T15:26:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.