Prompting Segment Anything Model with Domain-Adaptive Prototype for Generalizable Medical Image Segmentation

• URL: http://arxiv.org/abs/2409.12522v1
• Date: Thu, 19 Sep 2024 07:28:33 GMT
• Title: Prompting Segment Anything Model with Domain-Adaptive Prototype for Generalizable Medical Image Segmentation
• Authors: Zhikai Wei, Wenhui Dong, Peilin Zhou, Yuliang Gu, Zhou Zhao, Yongchao Xu,
• Abstract summary: We propose a novel Domain-Adaptive Prompt framework for fine-tuning the Segment Anything Model (termed as DAPSAM) in segmenting medical images. Our DAPSAM achieves state-of-the-art performance on two medical image segmentation tasks with different modalities.
• Score: 49.5901368256326
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning based methods often suffer from performance degradation caused by domain shift. In recent years, many sophisticated network structures have been designed to tackle this problem. However, the advent of large model trained on massive data, with its exceptional segmentation capability, introduces a new perspective for solving medical segmentation problems. In this paper, we propose a novel Domain-Adaptive Prompt framework for fine-tuning the Segment Anything Model (termed as DAPSAM) to address single-source domain generalization (SDG) in segmenting medical images. DAPSAM not only utilizes a more generalization-friendly adapter to fine-tune the large model, but also introduces a self-learning prototype-based prompt generator to enhance model's generalization ability. Specifically, we first merge the important low-level features into intermediate features before feeding to each adapter, followed by an attention filter to remove redundant information. This yields more robust image embeddings. Then, we propose using a learnable memory bank to construct domain-adaptive prototypes for prompt generation, helping to achieve generalizable medical image segmentation. Extensive experimental results demonstrate that our DAPSAM achieves state-of-the-art performance on two SDG medical image segmentation tasks with different modalities. The code is available at https://github.com/wkklavis/DAPSAM.

Related papers

• ASPS: Augmented Segment Anything Model for Polyp Segmentation [77.25557224490075]
  The Segment Anything Model (SAM) has introduced unprecedented potential for polyp segmentation. SAM's Transformer-based structure prioritizes global and low-frequency information. CFA integrates a trainable CNN encoder branch with a frozen ViT encoder, enabling the integration of domain-specific knowledge.
  arXiv  Detail & Related papers  (2024-06-30T14:55:32Z)
• Dual-scale Enhanced and Cross-generative Consistency Learning for Semi-supervised Medical Image Segmentation [49.57907601086494]
  Medical image segmentation plays a crucial role in computer-aided diagnosis. We propose a novel Dual-scale Enhanced and Cross-generative consistency learning framework for semi-supervised medical image (DEC-Seg)
  arXiv  Detail & Related papers  (2023-12-26T12:56:31Z)
• DG-TTA: Out-of-domain medical image segmentation through Domain Generalization and Test-Time Adaptation [43.842694540544194]
  We propose to combine domain generalization and test-time adaptation to create a highly effective approach for reusing pre-trained models in unseen target domains. We demonstrate that our method, combined with pre-trained whole-body CT models, can effectively segment MR images with high accuracy.
  arXiv  Detail & Related papers  (2023-12-11T10:26:21Z)
• Frequency-mixed Single-source Domain Generalization for Medical Image Segmentation [29.566769388674473]
  The scarcity of medical image segmentation poses challenges in collecting sufficient training data for deep learning models. We propose a novel approach called the Frequency-mixed Single-source Domain Generalization method (FreeSDG) Experimental results on five datasets of three modalities demonstrate the effectiveness of the proposed algorithm.
  arXiv  Detail & Related papers  (2023-07-18T06:44:45Z)
• Learning with Explicit Shape Priors for Medical Image Segmentation [17.110893665132423]
  We propose a novel shape prior module (SPM) to promote the segmentation performance of UNet-based models. Explicit shape priors consist of global and local shape priors. Our proposed model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-03-31T11:12:35Z)
• Self-Supervised Correction Learning for Semi-Supervised Biomedical Image Segmentation [84.58210297703714]
  We propose a self-supervised correction learning paradigm for semi-supervised biomedical image segmentation. We design a dual-task network, including a shared encoder and two independent decoders for segmentation and lesion region inpainting. Experiments on three medical image segmentation datasets for different tasks demonstrate the outstanding performance of our method.
  arXiv  Detail & Related papers  (2023-01-12T08:19:46Z)
• Learning from partially labeled data for multi-organ and tumor segmentation [102.55303521877933]
  We propose a Transformer based dynamic on-demand network (TransDoDNet) that learns to segment organs and tumors on multiple datasets. A dynamic head enables the network to accomplish multiple segmentation tasks flexibly. We create a large-scale partially labeled Multi-Organ and Tumor benchmark, termed MOTS, and demonstrate the superior performance of our TransDoDNet over other competitors.
  arXiv  Detail & Related papers  (2022-11-13T13:03:09Z)
• Generalizable Medical Image Segmentation via Random Amplitude Mixup and Domain-Specific Image Restoration [17.507951655445652]
  We present a novel generalizable medical image segmentation method. To be specific, we design our approach as a multi-task paradigm by combining the segmentation model with a self-supervision domain-specific image restoration module. We demonstrate the performance of our method on two public generalizable segmentation benchmarks in medical images.
  arXiv  Detail & Related papers  (2022-08-08T03:56:20Z)
• Contrastive Domain Disentanglement for Generalizable Medical Image Segmentation [12.863227646939563]
  We propose Contrastive Disentangle Domain (CDD) network for generalizable medical image segmentation. We first introduce a disentangle network to decompose medical images into an anatomical representation factor and a modality representation factor. We then propose a domain augmentation strategy that can randomly generate new domains for model generalization training.
  arXiv  Detail & Related papers  (2022-05-13T10:32:41Z)
• AF$_2$: Adaptive Focus Framework for Aerial Imagery Segmentation [86.44683367028914]
  Aerial imagery segmentation has some unique challenges, the most critical one among which lies in foreground-background imbalance. We propose Adaptive Focus Framework (AF$), which adopts a hierarchical segmentation procedure and focuses on adaptively utilizing multi-scale representations. AF$ has significantly improved the accuracy on three widely used aerial benchmarks, as fast as the mainstream method.
  arXiv  Detail & Related papers  (2022-02-18T10:14:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.