Co-Generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data

• URL: http://arxiv.org/abs/2103.09276v1
• Date: Tue, 16 Mar 2021 18:41:18 GMT
• Title: Co-Generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data
• Authors: Megha Kalia, Tajwar Abrar Aleef, Nassir Navab, and Septimiu E. Salcudean
• Abstract summary: Surgical instrument segmentation for robot-assisted surgery is needed for accurate instrument tracking and augmented reality overlays. Deep learning-based methods have shown state-of-the-art performance for surgical instrument segmentation, but their results depend on labelled data. In this paper, we demonstrate the limited generalizability of these methods on different datasets, including human robot-assisted surgeries.
• Score: 49.419268399590045
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Surgical instrument segmentation for robot-assisted surgery is needed for accurate instrument tracking and augmented reality overlays. Therefore, the topic has been the subject of a number of recent papers in the CAI community. Deep learning-based methods have shown state-of-the-art performance for surgical instrument segmentation, but their results depend on labelled data. However, labelled surgical data is of limited availability and is a bottleneck in surgical translation of these methods. In this paper, we demonstrate the limited generalizability of these methods on different datasets, including human robot-assisted surgeries. We then propose a novel joint generation and segmentation strategy to learn a segmentation model with better generalization capability to domains that have no labelled data. The method leverages the availability of labelled data in a different domain. The generator does the domain translation from the labelled domain to the unlabelled domain and simultaneously, the segmentation model learns using the generated data while regularizing the generative model. We compared our method with state-of-the-art methods and showed its generalizability on publicly available datasets and on our own recorded video frames from robot-assisted prostatectomies. Our method shows consistently high mean Dice scores on both labelled and unlabelled domains when data is available only for one of the domains. *M. Kalia and T. Aleef contributed equally to the manuscript

Related papers

• Revisiting Surgical Instrument Segmentation Without Human Intervention: A Graph Partitioning View [7.594796294925481]
  We propose an unsupervised method by reframing the video frame segmentation as a graph partitioning problem. A self-supervised pre-trained model is firstly leveraged as a feature extractor to capture high-level semantic features. On the "deep" eigenvectors, a surgical video frame is meaningfully segmented into different modules like tools and tissues, providing distinguishable semantic information.
  arXiv  Detail & Related papers  (2024-08-27T05:31:30Z)
• AnatoMix: Anatomy-aware Data Augmentation for Multi-organ Segmentation [6.471203541258319]
  We propose a novel data augmentation strategy for increasing the generalizibility of multi-organ segmentation datasets. By object-level matching and manipulation, our method is able to generate new images with correct anatomy. Our augmentation method can lead to mean dice of 76.1, compared with 74.8 of the baseline method.
  arXiv  Detail & Related papers  (2024-03-05T21:07:50Z)
• Cross-Dataset Adaptation for Instrument Classification in Cataract Surgery Videos [54.1843419649895]
  State-of-the-art models, which perform this task well on a particular dataset, perform poorly when tested on another dataset. We propose a novel end-to-end Unsupervised Domain Adaptation (UDA) method called the Barlow Adaptor. In addition, we introduce a novel loss called the Barlow Feature Alignment Loss (BFAL) which aligns features across different domains.
  arXiv  Detail & Related papers  (2023-07-31T18:14:18Z)
• Self-supervised Registration and Segmentation of the Ossicles with A Single Ground Truth Label [1.6244541005112747]
  This paper presents a novel technique using a self-supervised 3D-UNet that produces a dense deformation field between an atlas and a target image. Our results show that our method outperforms traditional image segmentation methods and generates a more accurate boundary around the ossicles.
  arXiv  Detail & Related papers  (2023-02-15T22:11:31Z)
• PCA: Semi-supervised Segmentation with Patch Confidence Adversarial Training [52.895952593202054]
  We propose a new semi-supervised adversarial method called Patch Confidence Adrial Training (PCA) for medical image segmentation. PCA learns the pixel structure and context information in each patch to get enough gradient feedback, which aids the discriminator in convergent to an optimal state. Our method outperforms the state-of-the-art semi-supervised methods, which demonstrates its effectiveness for medical image segmentation.
  arXiv  Detail & Related papers  (2022-07-24T07:45:47Z)
• Pseudo-label Guided Cross-video Pixel Contrast for Robotic Surgical Scene Segmentation with Limited Annotations [72.15956198507281]
  We propose PGV-CL, a novel pseudo-label guided cross-video contrast learning method to boost scene segmentation. We extensively evaluate our method on a public robotic surgery dataset EndoVis18 and a public cataract dataset CaDIS.
  arXiv  Detail & Related papers  (2022-07-20T05:42:19Z)
• TraSeTR: Track-to-Segment Transformer with Contrastive Query for Instance-level Instrument Segmentation in Robotic Surgery [60.439434751619736]
  We propose TraSeTR, a Track-to-Segment Transformer that exploits tracking cues to assist surgical instrument segmentation. TraSeTR jointly reasons about the instrument type, location, and identity with instance-level predictions. The effectiveness of our method is demonstrated with state-of-the-art instrument type segmentation results on three public datasets.
  arXiv  Detail & Related papers  (2022-02-17T05:52:18Z)
• CvS: Classification via Segmentation For Small Datasets [52.821178654631254]
  This paper presents CvS, a cost-effective classifier for small datasets that derives the classification labels from predicting the segmentation maps. We evaluate the effectiveness of our framework on diverse problems showing that CvS is able to achieve much higher classification results compared to previous methods when given only a handful of examples.
  arXiv  Detail & Related papers  (2021-10-29T18:41:15Z)
• Cross-Domain Segmentation with Adversarial Loss and Covariate Shift for Biomedical Imaging [2.1204495827342438]
  This manuscript aims to implement a novel model that can learn robust representations from cross-domain data by encapsulating distinct and shared patterns from different modalities. The tests on CT and MRI liver data acquired in routine clinical trials show that the proposed model outperforms all other baseline with a large margin.
  arXiv  Detail & Related papers  (2020-06-08T07:35:55Z)
• Segmentation of Surgical Instruments for Minimally-Invasive Robot-Assisted Procedures Using Generative Deep Neural Networks [17.571763112459166]
  This work proves that semantic segmentation on minimally invasive surgical instruments can be improved by using training data. To achieve this, a CycleGAN model is used, which transforms a source dataset to approximate the domain distribution of a target dataset. This newly generated data with perfect labels is utilized to train a semantic segmentation neural network, U-Net.
  arXiv  Detail & Related papers  (2020-06-05T14:39:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.