Towards foundation models and few-shot parameter-efficient fine-tuning
for volumetric organ segmentation
- URL: http://arxiv.org/abs/2303.17051v2
- Date: Fri, 29 Sep 2023 01:16:18 GMT
- Title: Towards foundation models and few-shot parameter-efficient fine-tuning
for volumetric organ segmentation
- Authors: Julio Silva-Rodr\'iguez, Jose Dolz and Ismail Ben Ayed
- Abstract summary: Few-shot efficient fine-tuning (FSEFT) is a novel and realistic setting for medical image segmentation.
We introduce a novel parameter-efficient fine-tuning strategy tailored to medical image segmentation.
Our comprehensive experiments on a collection of public CT datasets for organ segmentation point to the potential of vision adapters and transductive inference.
- Score: 21.588709922418765
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: With the recent raise of foundation models in computer vision and NLP, the
pretrain-and-adapt strategy, where a large-scale model is fine-tuned on
downstream tasks, is gaining popularity. However, traditional fine-tuning
approaches may still require significant resources and yield sub-optimal
results when the labeled data of the target task is scarce. This is especially
the case in clinical settings. To address this challenge, we formalize few-shot
efficient fine-tuning (FSEFT), a novel and realistic setting for medical image
segmentation. Furthermore, we introduce a novel parameter-efficient fine-tuning
strategy tailored to medical image segmentation, with (a) spatial adapter
modules that are more appropriate for dense prediction tasks; and (b) a
constrained transductive inference, which leverages task-specific prior
knowledge. Our comprehensive experiments on a collection of public CT datasets
for organ segmentation reveal the limitations of standard fine-tuning methods
in few-shot scenarios, point to the potential of vision adapters and
transductive inference, and confirm the suitability of foundation models.
Related papers
- Few-Shot Airway-Tree Modeling using Data-Driven Sparse Priors [0.0]
Few-shot learning approaches are cost-effective to transfer pre-trained models using only limited annotated data.
We train a data-driven sparsification module to enhance airways efficiently in lung CT scans.
We then incorporate these sparse representations in a standard supervised segmentation pipeline as a pretraining step to enhance the performance of the DL models.
arXiv Detail & Related papers (2024-07-05T13:46:11Z) - Embedded Prompt Tuning: Towards Enhanced Calibration of Pretrained Models for Medical Images [18.094731760514264]
We study the effectiveness of fine-tuning methods when adapting foundation models to medical image classification tasks.
We propose the Embedded Prompt Tuning (EPT) method by embedding prompt tokens into the expanded channels.
EPT outperforms several state-of-the-art finetuning methods by a significant margin on few-shot medical image classification tasks.
arXiv Detail & Related papers (2024-07-01T06:35:53Z) - Low-rank finetuning for LLMs: A fairness perspective [54.13240282850982]
Low-rank approximation techniques have become the de facto standard for fine-tuning Large Language Models.
This paper investigates the effectiveness of these methods in capturing the shift of fine-tuning datasets from the initial pre-trained data distribution.
We show that low-rank fine-tuning inadvertently preserves undesirable biases and toxic behaviors.
arXiv Detail & Related papers (2024-05-28T20:43:53Z) - Meta Transfer of Self-Supervised Knowledge: Foundation Model in Action
for Post-Traumatic Epilepsy Prediction [0.6291443816903801]
We introduce a novel training strategy for our foundation model.
We demonstrate that the proposed strategy significantly improves task performance on small-scale clinical datasets.
Results further demonstrated the enhanced generalizability of our foundation model.
arXiv Detail & Related papers (2023-12-21T07:42:49Z) - Parameter Efficient Fine-tuning via Cross Block Orchestration for Segment Anything Model [81.55141188169621]
We equip PEFT with a cross-block orchestration mechanism to enable the adaptation of the Segment Anything Model (SAM) to various downstream scenarios.
We propose an intra-block enhancement module, which introduces a linear projection head whose weights are generated from a hyper-complex layer.
Our proposed approach consistently improves the segmentation performance significantly on novel scenarios with only around 1K additional parameters.
arXiv Detail & Related papers (2023-11-28T11:23:34Z) - Learnable Weight Initialization for Volumetric Medical Image Segmentation [66.3030435676252]
We propose a learnable weight-based hybrid medical image segmentation approach.
Our approach is easy to integrate into any hybrid model and requires no external training data.
Experiments on multi-organ and lung cancer segmentation tasks demonstrate the effectiveness of our approach.
arXiv Detail & Related papers (2023-06-15T17:55:05Z) - Prompt-Based Tuning of Transformer Models for Multi-Center Medical Image
Segmentation of Head and Neck Cancer [0.0]
This paper proposes a novel fine-tuning strategy for adapting a pretrained transformer-based segmentation model on data from a new medical center.
Our strategy delivers great accuracy with minimum re-training on new-center data, significantly decreasing the computational and time costs.
arXiv Detail & Related papers (2023-05-30T11:26:52Z) - Studying How to Efficiently and Effectively Guide Models with Explanations [52.498055901649025]
'Model guidance' is the idea of regularizing the models' explanations to ensure that they are "right for the right reasons"
We conduct an in-depth evaluation across various loss functions, attribution methods, models, and 'guidance depths' on the PASCAL VOC 2007 and MS COCO 2014 datasets.
Specifically, we guide the models via bounding box annotations, which are much cheaper to obtain than the commonly used segmentation masks.
arXiv Detail & Related papers (2023-03-21T15:34:50Z) - Fine-grained Retrieval Prompt Tuning [149.9071858259279]
Fine-grained Retrieval Prompt Tuning steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompt and feature adaptation.
Our FRPT with fewer learnable parameters achieves the state-of-the-art performance on three widely-used fine-grained datasets.
arXiv Detail & Related papers (2022-07-29T04:10:04Z) - An Uncertainty-Driven GCN Refinement Strategy for Organ Segmentation [53.425900196763756]
We propose a segmentation refinement method based on uncertainty analysis and graph convolutional networks.
We employ the uncertainty levels of the convolutional network in a particular input volume to formulate a semi-supervised graph learning problem.
We show that our method outperforms the state-of-the-art CRF refinement method by improving the dice score by 1% for the pancreas and 2% for spleen.
arXiv Detail & Related papers (2020-12-06T18:55:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.