On the Out of Distribution Robustness of Foundation Models in Medical
Image Segmentation
- URL: http://arxiv.org/abs/2311.11096v1
- Date: Sat, 18 Nov 2023 14:52:10 GMT
- Title: On the Out of Distribution Robustness of Foundation Models in Medical
Image Segmentation
- Authors: Duy Minh Ho Nguyen, Tan Ngoc Pham, Nghiem Tuong Diep, Nghi Quoc Phan,
Quang Pham, Vinh Tong, Binh T. Nguyen, Ngan Hoang Le, Nhat Ho, Pengtao Xie,
Daniel Sonntag, Mathias Niepert
- Abstract summary: Foundations for vision and language, pre-trained on extensive sets of natural image and text data, have emerged as a promising approach.
We compare the generalization performance to unseen domains of various pre-trained models after being fine-tuned on the same in-distribution dataset.
We further developed a new Bayesian uncertainty estimation for frozen models and used them as an indicator to characterize the model's performance on out-of-distribution data.
- Score: 47.95611203419802
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Constructing a robust model that can effectively generalize to test samples
under distribution shifts remains a significant challenge in the field of
medical imaging. The foundational models for vision and language, pre-trained
on extensive sets of natural image and text data, have emerged as a promising
approach. It showcases impressive learning abilities across different tasks
with the need for only a limited amount of annotated samples. While numerous
techniques have focused on developing better fine-tuning strategies to adapt
these models for specific domains, we instead examine their robustness to
domain shifts in the medical image segmentation task. To this end, we compare
the generalization performance to unseen domains of various pre-trained models
after being fine-tuned on the same in-distribution dataset and show that
foundation-based models enjoy better robustness than other architectures. From
here, we further developed a new Bayesian uncertainty estimation for frozen
models and used them as an indicator to characterize the model's performance on
out-of-distribution (OOD) data, proving particularly beneficial for real-world
applications. Our experiments not only reveal the limitations of current
indicators like accuracy on the line or agreement on the line commonly used in
natural image applications but also emphasize the promise of the introduced
Bayesian uncertainty. Specifically, lower uncertainty predictions usually tend
to higher out-of-distribution (OOD) performance.
Related papers
- Enhancing Robustness of Foundation Model Representations under
Provenance-related Distribution Shifts [8.298173603769063]
We examine the stability of models based on foundation models under distribution shift.
We focus on confounding by provenance, a form of distribution shift that emerges in the context of multi-institutional datasets.
Results indicate that while foundation models do show some out-of-the-box robustness to confounding-by-provenance related distribution shifts, this can be improved through adjustment.
arXiv Detail & Related papers (2023-12-09T02:02:45Z) - Improving Robustness and Reliability in Medical Image Classification
with Latent-Guided Diffusion and Nested-Ensembles [4.642805070301818]
We introduce a novel three-stage approach based on transformers and conditional diffusion models.
We show that our method improves upon state-of-the-art methods in terms of robustness and confidence calibration.
arXiv Detail & Related papers (2023-10-24T15:53:07Z) - Certification of Deep Learning Models for Medical Image Segmentation [44.177565298565966]
We present for the first time a certified segmentation baseline for medical imaging based on randomized smoothing and diffusion models.
Our results show that leveraging the power of denoising diffusion probabilistic models helps us overcome the limits of randomized smoothing.
arXiv Detail & Related papers (2023-10-05T16:40:33Z) - Consistency Regularization for Generalizable Source-free Domain
Adaptation [62.654883736925456]
Source-free domain adaptation (SFDA) aims to adapt a well-trained source model to an unlabelled target domain without accessing the source dataset.
Existing SFDA methods ONLY assess their adapted models on the target training set, neglecting the data from unseen but identically distributed testing sets.
We propose a consistency regularization framework to develop a more generalizable SFDA method.
arXiv Detail & Related papers (2023-08-03T07:45:53Z) - Realistic Data Enrichment for Robust Image Segmentation in
Histopathology [2.248423960136122]
We propose a new approach, based on diffusion models, which can enrich an imbalanced dataset with plausible examples from underrepresented groups.
Our method can simply expand limited clinical datasets making them suitable to train machine learning pipelines.
arXiv Detail & Related papers (2023-04-19T09:52:50Z) - Ambiguous Medical Image Segmentation using Diffusion Models [60.378180265885945]
We introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights.
Our proposed model generates a distribution of segmentation masks by leveraging the inherent sampling process of diffusion.
Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks.
arXiv Detail & Related papers (2023-04-10T17:58:22Z) - Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness.
We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
arXiv Detail & Related papers (2023-03-06T11:51:28Z) - Adapting Pretrained Vision-Language Foundational Models to Medical
Imaging Domains [3.8137985834223502]
Building generative models for medical images that faithfully depict clinical context may help alleviate the paucity of healthcare datasets.
We explore the sub-components of the Stable Diffusion pipeline to fine-tune the model to generate medical images.
Our best-performing model improves upon the stable diffusion baseline and can be conditioned to insert a realistic-looking abnormality on a synthetic radiology image.
arXiv Detail & Related papers (2022-10-09T01:43:08Z) - General Greedy De-bias Learning [163.65789778416172]
We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space.
GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
arXiv Detail & Related papers (2021-12-20T14:47:32Z) - MEMO: Test Time Robustness via Adaptation and Augmentation [131.28104376280197]
We study the problem of test time robustification, i.e., using the test input to improve model robustness.
Recent prior works have proposed methods for test time adaptation, however, they each introduce additional assumptions.
We propose a simple approach that can be used in any test setting where the model is probabilistic and adaptable.
arXiv Detail & Related papers (2021-10-18T17:55:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.