Memory-Efficient Prompt Tuning for Incremental Histopathology Classification

• URL: http://arxiv.org/abs/2401.11674v1
• Date: Mon, 22 Jan 2024 03:24:45 GMT
• Title: Memory-Efficient Prompt Tuning for Incremental Histopathology Classification
• Authors: Yu Zhu, Kang Li, Lequan Yu, Pheng-Ann Heng
• Abstract summary: We present a memory-efficient prompt tuning framework to cultivate model generalization potential in economical memory cost. We have extensively evaluated our framework with two histopathology tasks, i.e., breast cancer metastasis classification and epithelium-stroma tissue classification.
• Score: 69.46798702300042
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent studies have made remarkable progress in histopathology classification. Based on current successes, contemporary works proposed to further upgrade the model towards a more generalizable and robust direction through incrementally learning from the sequentially delivered domains. Unlike previous parameter isolation based approaches that usually demand massive computation resources during model updating, we present a memory-efficient prompt tuning framework to cultivate model generalization potential in economical memory cost. For each incoming domain, we reuse the existing parameters of the initial classification model and attach lightweight trainable prompts into it for customized tuning. Considering the domain heterogeneity, we perform decoupled prompt tuning, where we adopt a domain-specific prompt for each domain to independently investigate its distinctive characteristics, and one domain-invariant prompt shared across all domains to continually explore the common content embedding throughout time. All domain-specific prompts will be appended to the prompt bank and isolated from further changes to prevent forgetting the distinctive features of early-seen domains. While the domain-invariant prompt will be passed on and iteratively evolve by style-augmented prompt refining to improve model generalization capability over time. In specific, we construct a graph with existing prompts and build a style-augmented graph attention network to guide the domain-invariant prompt exploring the overlapped latent embedding among all delivered domains for more domain generic representations. We have extensively evaluated our framework with two histopathology tasks, i.e., breast cancer metastasis classification and epithelium-stroma tissue classification, where our approach yielded superior performance and memory efficiency over the competing methods.

Related papers

• Domain Expansion and Boundary Growth for Open-Set Single-Source Domain Generalization [70.02187124865627]
  Open-set single-source domain generalization aims to use a single-source domain to learn a robust model that can be generalized to unknown target domains. We propose a novel learning approach based on domain expansion and boundary growth to expand the scarce source samples. Our approach can achieve significant improvements and reach state-of-the-art performance on several cross-domain image classification datasets.
  arXiv  Detail & Related papers  (2024-11-05T09:08:46Z)
• StyDeSty: Min-Max Stylization and Destylization for Single Domain Generalization [85.18995948334592]
  Single domain generalization (single DG) aims at learning a robust model generalizable to unseen domains from only one training domain. State-of-the-art approaches have mostly relied on data augmentations, such as adversarial perturbation and style enhancement, to synthesize new data. We propose emphStyDeSty, which explicitly accounts for the alignment of the source and pseudo domains in the process of data augmentation.
  arXiv  Detail & Related papers  (2024-06-01T02:41:34Z)
• Prompting Diffusion Representations for Cross-Domain Semantic Segmentation [101.04326113360342]
  diffusion-pretraining achieves extraordinary domain generalization results for semantic segmentation. We introduce a scene prompt and a prompt randomization strategy to help further disentangle the domain-invariant information when training the segmentation head.
  arXiv  Detail & Related papers  (2023-07-05T09:28:25Z)
• Learning Domain Invariant Prompt for Vision-Language Models [31.581652862478965]
  We propose a novel prompt learning paradigm that directly generates emphdomain invariant prompt that can be generalized to unseen domains, called MetaPrompt. Our method consistently and significantly outperforms existing methods.
  arXiv  Detail & Related papers  (2022-12-08T11:23:24Z)
• Domain-incremental Cardiac Image Segmentation with Style-oriented Replay and Domain-sensitive Feature Whitening [67.6394526631557]
  M&Ms should incrementally learn from each incoming dataset and progressively update with improved functionality as time goes by. In medical scenarios, this is particularly challenging as accessing or storing past data is commonly not allowed due to data privacy. We propose a novel domain-incremental learning framework to recover past domain inputs first and then regularly replay them during model optimization.
  arXiv  Detail & Related papers  (2022-11-09T13:07:36Z)
• Context-Conditional Adaptation for Recognizing Unseen Classes in Unseen Domains [48.17225008334873]
  We propose a feature generative framework integrated with a COntext COnditional Adaptive (COCOA) Batch-Normalization. The generated visual features better capture the underlying data distribution enabling us to generalize to unseen classes and domains at test-time. We thoroughly evaluate and analyse our approach on established large-scale benchmark - DomainNet.
  arXiv  Detail & Related papers  (2021-07-15T17:51:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.