Prototype Learning for Out-of-Distribution Polyp Segmentation
- URL: http://arxiv.org/abs/2308.03709v1
- Date: Mon, 7 Aug 2023 16:30:24 GMT
- Title: Prototype Learning for Out-of-Distribution Polyp Segmentation
- Authors: Nikhil Kumar Tomar, Debesh Jha, Ulas Bagci
- Abstract summary: Existing polyp segmentation models from colonoscopy images often fail to provide reliable segmentation results on datasets from different centers.
Our model is designed to perform effectively on out-of-distribution (OOD) datasets from multiple centers.
PrototypeLab offers a promising solution with a dice coefficient of $geq$ 90% and mIoU $geq$ 85% with a near real-time processing speed for polyp segmentation.
- Score: 2.6179759969345002
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Existing polyp segmentation models from colonoscopy images often fail to
provide reliable segmentation results on datasets from different centers,
limiting their applicability. Our objective in this study is to create a robust
and well-generalized segmentation model named PrototypeLab that can assist in
polyp segmentation. To achieve this, we incorporate various lighting modes such
as White light imaging (WLI), Blue light imaging (BLI), Linked color imaging
(LCI), and Flexible spectral imaging color enhancement (FICE) into our new
segmentation model, that learns to create prototypes for each class of object
present in the images. These prototypes represent the characteristic features
of the objects, such as their shape, texture, color. Our model is designed to
perform effectively on out-of-distribution (OOD) datasets from multiple
centers. We first generate a coarse mask that is used to learn prototypes for
the main object class, which are then employed to generate the final
segmentation mask. By using prototypes to represent the main class, our
approach handles the variability present in the medical images and generalize
well to new data since prototype capture the underlying distribution of the
data. PrototypeLab offers a promising solution with a dice coefficient of
$\geq$ 90\% and mIoU $\geq$ 85\% with a near real-time processing speed for
polyp segmentation. It achieved superior performance on OOD datasets compared
to 16 state-of-the-art image segmentation architectures, potentially improving
clinical outcomes. Codes are available at
https://github.com/xxxxx/PrototypeLab.
Related papers
- ProtoSAM: One-Shot Medical Image Segmentation With Foundational Models [29.781228739479893]
ProtoSAM is a new framework for one-shot medical image segmentation.
It combines the use of prototypical networks, known for few-shot segmentation, with SAM - a natural image foundation model.
arXiv Detail & Related papers (2024-07-09T17:04:08Z) - Diffusion-based Data Augmentation for Nuclei Image Segmentation [68.28350341833526]
We introduce the first diffusion-based augmentation method for nuclei segmentation.
The idea is to synthesize a large number of labeled images to facilitate training the segmentation model.
The experimental results show that by augmenting 10% labeled real dataset with synthetic samples, one can achieve comparable segmentation results.
arXiv Detail & Related papers (2023-10-22T06:16:16Z) - Rethinking Polyp Segmentation from an Out-of-Distribution Perspective [37.1338930936671]
We leverage the ability of masked autoencoders -- self-supervised vision transformers trained on a reconstruction task -- to learn in-distribution representations.
We perform out-of-distribution reconstruction and inference, with feature space standardisation to align the latent distribution of the diverse abnormal samples with the statistics of the healthy samples.
Experimental results on six benchmarks show that our model has excellent segmentation performance and generalises across datasets.
arXiv Detail & Related papers (2023-06-13T14:13:16Z) - Distilling Ensemble of Explanations for Weakly-Supervised Pre-Training
of Image Segmentation Models [54.49581189337848]
We propose a method to enable the end-to-end pre-training for image segmentation models based on classification datasets.
The proposed method leverages a weighted segmentation learning procedure to pre-train the segmentation network en masse.
Experiment results show that, with ImageNet accompanied by PSSL as the source dataset, the proposed end-to-end pre-training strategy successfully boosts the performance of various segmentation models.
arXiv Detail & Related papers (2022-07-04T13:02:32Z) - Rethinking Semantic Segmentation: A Prototype View [126.59244185849838]
We present a nonparametric semantic segmentation model based on non-learnable prototypes.
Our framework yields compelling results over several datasets.
We expect this work will provoke a rethink of the current de facto semantic segmentation model design.
arXiv Detail & Related papers (2022-03-28T21:15:32Z) - Multi-dataset Pretraining: A Unified Model for Semantic Segmentation [97.61605021985062]
We propose a unified framework, termed as Multi-Dataset Pretraining, to take full advantage of the fragmented annotations of different datasets.
This is achieved by first pretraining the network via the proposed pixel-to-prototype contrastive loss over multiple datasets.
In order to better model the relationship among images and classes from different datasets, we extend the pixel level embeddings via cross dataset mixing.
arXiv Detail & Related papers (2021-06-08T06:13:11Z) - SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive
Background Prototypes [56.387647750094466]
Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples.
Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype.
This framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only.
arXiv Detail & Related papers (2021-04-19T11:21:47Z) - Deep ensembles based on Stochastic Activation Selection for Polyp
Segmentation [82.61182037130406]
This work deals with medical image segmentation and in particular with accurate polyp detection and segmentation during colonoscopy examinations.
Basic architecture in image segmentation consists of an encoder and a decoder.
We compare some variant of the DeepLab architecture obtained by varying the decoder backbone.
arXiv Detail & Related papers (2021-04-02T02:07:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.