Zero-shot Domain Generalization of Foundational Models for 3D Medical Image Segmentation: An Experimental Study

• URL: http://arxiv.org/abs/2503.22862v1
• Date: Fri, 28 Mar 2025 20:33:41 GMT
• Title: Zero-shot Domain Generalization of Foundational Models for 3D Medical Image Segmentation: An Experimental Study
• Authors: Soumitri Chattopadhyay, Basar Demir, Marc Niethammer,
• Abstract summary: Foundations models (FMs) trained on diverse large-scale data hold promise for zero-shot generalization.<n>In this study, we examine their ability towards domain generalization (DG) by conducting a comprehensive experimental study.<n>Our findings reveal the potential of promptable FMs in bridging the domain gap via smart prompting techniques.
• Score: 15.3909625201792
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Domain shift, caused by variations in imaging modalities and acquisition protocols, limits model generalization in medical image segmentation. While foundation models (FMs) trained on diverse large-scale data hold promise for zero-shot generalization, their application to volumetric medical data remains underexplored. In this study, we examine their ability towards domain generalization (DG), by conducting a comprehensive experimental study encompassing 6 medical segmentation FMs and 12 public datasets spanning multiple modalities and anatomies. Our findings reveal the potential of promptable FMs in bridging the domain gap via smart prompting techniques. Additionally, by probing into multiple facets of zero-shot DG, we offer valuable insights into the viability of FMs for DG and identify promising avenues for future research.

Related papers

• MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applications [10.321593505248341]
  This paper introduces a diffusion-based foundation model to address a diverse range of medical image tasks, namely MedDiff-FM. MedDiff-FM leverages 3D CT images from multiple publicly available datasets, covering anatomical regions from head to abdomen, to pre-train a diffusion foundation model. Experimental results demonstrate the effectiveness of MedDiff-FM in addressing diverse downstream medical image tasks.
  arXiv  Detail & Related papers  (2024-10-20T16:03:55Z)
• Do Vision Foundation Models Enhance Domain Generalization in Medical Image Segmentation? [10.20366295974822]
  We introduce a novel decode head architecture, HQHSAM, which simply integrates elements from two state-of-the-art decoder heads, HSAM and HQSAM, to enhance segmentation performance. Our experiments on multiple datasets, encompassing various anatomies and modalities, reveal that FMs, particularly with the HQHSAM decode head, improve domain generalization for medical image segmentation.
  arXiv  Detail & Related papers  (2024-09-12T11:41:35Z)
• RadGenome-Chest CT: A Grounded Vision-Language Dataset for Chest CT Analysis [56.57177181778517]
  RadGenome-Chest CT is a large-scale, region-guided 3D chest CT interpretation dataset based on CT-RATE. We leverage the latest powerful universal segmentation and large language models to extend the original datasets.
  arXiv  Detail & Related papers  (2024-04-25T17:11:37Z)
• Multi-domain improves out-of-distribution and data-limited scenarios for medical image analysis [2.315156126698557]
  We show that employing models that incorporate multiple domains instead of specialized ones significantly alleviates the limitations observed in specialized models. For organ recognition, multi-domain model can enhance accuracy by up to 8% compared to conventional specialized models.
  arXiv  Detail & Related papers  (2023-10-10T16:07:23Z)
• DGM-DR: Domain Generalization with Mutual Information Regularized Diabetic Retinopathy Classification [40.35834579068518]
  Domain shift between training and testing data presents a significant challenge for training general deep learning models. We introduce a DG method that re-establishes the model objective function as a pretrained model to the medical imaging field. Our proposed method consistently outperforms the previous state-of-the-art by a margin of 5.25% in average accuracy and a lower standard deviation.
  arXiv  Detail & Related papers  (2023-09-18T11:17:13Z)
• Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data [66.9359934608229]
  This study aims to initiate the development of Radiology Foundation Model, termed as RadFM. To the best of our knowledge, this is the first large-scale, high-quality, medical visual-language dataset, with both 2D and 3D scans. We propose a new evaluation benchmark, RadBench, that comprises five tasks, including modality recognition, disease diagnosis, visual question answering, report generation and rationale diagnosis.
  arXiv  Detail & Related papers  (2023-08-04T17:00:38Z)
• Ambiguous Medical Image Segmentation using Diffusion Models [60.378180265885945]
  We introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent sampling process of diffusion. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks.
  arXiv  Detail & Related papers  (2023-04-10T17:58:22Z)
• Domain Generalization with Adversarial Intensity Attack for Medical Image Segmentation [27.49427483473792]
  In real-world scenarios, it is common for models to encounter data from new and different domains to which they were not exposed to during training. domain generalization (DG) is a promising direction as it enables models to handle data from previously unseen domains. We introduce a novel DG method called Adversarial Intensity Attack (AdverIN), which leverages adversarial training to generate training data with an infinite number of styles.
  arXiv  Detail & Related papers  (2023-04-05T19:40:51Z)
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding [60.89561661441736]
  We propose a novel method for few-shot medical image segmentation. We construct our few-shot image segmentor using a deep convolutional network trained episodically. We enhance discriminability of deep embedding to encourage clustering of the feature domains of the same class.
  arXiv  Detail & Related papers  (2020-12-10T04:01:07Z)
• Universal Model for Multi-Domain Medical Image Retrieval [88.67940265012638]
  Medical Image Retrieval (MIR) helps doctors quickly find similar patients' data. MIR is becoming increasingly helpful due to the wide use of digital imaging modalities. However, the popularity of various digital imaging modalities in hospitals also poses several challenges to MIR.
  arXiv  Detail & Related papers  (2020-07-14T23:22:04Z)
• Shape-aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains [68.73614619875814]
  We present a novel shape-aware meta-learning scheme to improve the model generalization in prostate MRI segmentation. Experimental results show that our approach outperforms many state-of-the-art generalization methods consistently across all six settings of unseen domains.
  arXiv  Detail & Related papers  (2020-07-04T07:56:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.