Unsupervised Segmentation of Colonoscopy Images

• URL: http://arxiv.org/abs/2312.12599v1
• Date: Tue, 19 Dec 2023 20:59:19 GMT
• Title: Unsupervised Segmentation of Colonoscopy Images
• Authors: Heming Yao, J\'er\^ome L\"uscher, Benjamin Gutierrez Becker, Josep Ar\'us-Pous, Tommaso Biancalani, Amelie Bigorgne, David Richmond
• Abstract summary: We explore using self-supervised features from vision transformers in three challenging tasks for colonoscopy images. Our results indicate that image-level features learned from DINO models achieve image classification performance comparable to fully supervised models.
• Score: 0.7775266571852477
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Colonoscopy plays a crucial role in the diagnosis and prognosis of various gastrointestinal diseases. Due to the challenges of collecting large-scale high-quality ground truth annotations for colonoscopy images, and more generally medical images, we explore using self-supervised features from vision transformers in three challenging tasks for colonoscopy images. Our results indicate that image-level features learned from DINO models achieve image classification performance comparable to fully supervised models, and patch-level features contain rich semantic information for object detection. Furthermore, we demonstrate that self-supervised features combined with unsupervised segmentation can be used to discover multiple clinically relevant structures in a fully unsupervised manner, demonstrating the tremendous potential of applying these methods in medical image analysis.

Related papers

• Frontiers in Intelligent Colonoscopy [96.57251132744446]
  This study investigates the frontiers of intelligent colonoscopy techniques and their prospective implications for multimodal medical applications. We assess the current data-centric and model-centric landscapes through four tasks for colonoscopic scene perception. To embrace the coming multimodal era, we establish three foundational initiatives: a large-scale multimodal instruction tuning dataset ColonINST, a colonoscopy-designed multimodal language model ColonGPT, and a multimodal benchmark.
  arXiv  Detail & Related papers  (2024-10-22T17:57:12Z)
• Advancing Medical Image Segmentation: Morphology-Driven Learning with Diffusion Transformer [4.672688418357066]
  We propose a novel Transformer Diffusion (DTS) model for robust segmentation in the presence of noise. Our model, which analyzes the morphological representation of images, shows better results than the previous models in various medical imaging modalities.
  arXiv  Detail & Related papers  (2024-08-01T07:35:54Z)
• Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning [65.54680361074882]
  Eye-gaze Guided Multi-modal Alignment (EGMA) framework harnesses eye-gaze data for better alignment of medical visual and textual features. We conduct downstream tasks of image classification and image-text retrieval on four medical datasets.
  arXiv  Detail & Related papers  (2024-03-19T03:59:14Z)
• Towards Automated Semantic Segmentation in Mammography Images [0.0]
  We propose a deep learning-based framework for the segmentation of the nipple, the pectoral muscle, the fibroglandular tissue, and the fatty tissue on standard-view mammography images.
  arXiv  Detail & Related papers  (2023-07-18T15:04:42Z)
• Contrastive learning for unsupervised medical image clustering and reconstruction [0.23624125155742057]
  We propose an unsupervised autoencoder framework which is augmented with a contrastive loss to encourage high separability in the latent space. Our method achieves similar performance to the supervised architecture, indicating that separation in the latent space reproduces expert medical observer-assigned labels.
  arXiv  Detail & Related papers  (2022-09-24T13:17:02Z)
• Semantic segmentation of multispectral photoacoustic images using deep learning [53.65837038435433]
  Photoacoustic imaging has the potential to revolutionise healthcare. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. We present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images.
  arXiv  Detail & Related papers  (2021-05-20T09:33:55Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding [60.89561661441736]
  We propose a novel method for few-shot medical image segmentation. We construct our few-shot image segmentor using a deep convolutional network trained episodically. We enhance discriminability of deep embedding to encourage clustering of the feature domains of the same class.
  arXiv  Detail & Related papers  (2020-12-10T04:01:07Z)
• Improving colonoscopy lesion classification using semi-supervised deep learning [2.568264809297699]
  Recent work in semi-supervised learning has shown that meaningful representations of images can be obtained from training with large quantities of unlabeled data. We demonstrate that an unsupervised jigsaw learning task, in combination with supervised training, results in up to a 9.8% improvement in correctly classifying lesions in colonoscopy images.
  arXiv  Detail & Related papers  (2020-09-07T15:25:35Z)
• Weakly supervised multiple instance learning histopathological tumor segmentation [51.085268272912415]
  We propose a weakly supervised framework for whole slide imaging segmentation. We exploit a multiple instance learning scheme for training models. The proposed framework has been evaluated on multi-locations and multi-centric public data from The Cancer Genome Atlas and the PatchCamelyon dataset.
  arXiv  Detail & Related papers  (2020-04-10T13:12:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.