Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the art

• URL: http://arxiv.org/abs/2304.13014v4
• Date: Tue, 23 Jan 2024 08:16:09 GMT
• Title: Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the art
• Authors: Tobias Rueckert (1), Daniel Rueckert (2 and 3), Christoph Palm (1 and 4) ((1) Regensburg Medical Image Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH Regensburg), Germany, (2) Artificial Intelligence in Healthcare and Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany, (3) Department of Computing, Imperial College London, UK, (4) Regensburg Center of Health Sciences and Technology (RCHST), OTH Regensburg, Germany)
• Abstract summary: We identify and characterize datasets used for method development and evaluation. The paper focuses on methods that work purely visually, without markers of any kind attached to the instruments. A discussion of the reviewed literature is provided, highlighting existing shortcomings and emphasizing the available potential for future developments.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In the field of computer- and robot-assisted minimally invasive surgery, enormous progress has been made in recent years based on the recognition of surgical instruments in endoscopic images and videos. In particular, the determination of the position and type of instruments is of great interest. Current work involves both spatial and temporal information, with the idea that predicting the movement of surgical tools over time may improve the quality of the final segmentations. The provision of publicly available datasets has recently encouraged the development of new methods, mainly based on deep learning. In this review, we identify and characterize datasets used for method development and evaluation and quantify their frequency of use in the literature. We further present an overview of the current state of research regarding the segmentation and tracking of minimally invasive surgical instruments in endoscopic images and videos. The paper focuses on methods that work purely visually, without markers of any kind attached to the instruments, considering both single-frame semantic and instance segmentation approaches, as well as those that incorporate temporal information. The publications analyzed were identified through the platforms Google Scholar, Web of Science, and PubMed. The search terms used were "instrument segmentation", "instrument tracking", "surgical tool segmentation", and "surgical tool tracking", resulting in a total of 741 articles published between 01/2015 and 07/2023, of which 123 were included using systematic selection criteria. A discussion of the reviewed literature is provided, highlighting existing shortcomings and emphasizing the available potential for future developments.

Related papers

• Amodal Segmentation for Laparoscopic Surgery Video Instruments [30.39518393494816]
  We introduce AmodalVis to the realm of surgical instruments in the medical field. This technique identifies both the visible and occluded parts of an object. To achieve this, we introduce a new Amoal Instruments dataset.
  arXiv  Detail & Related papers  (2024-08-02T07:40:34Z)
• Exploring Optical Flow Inclusion into nnU-Net Framework for Surgical Instrument Segmentation [1.3444601218847545]
  The nnU-Net framework excelled in semantic segmentation analyzing single frames without temporal information. Optical flow (OF) is a tool commonly used in video tasks to estimate motion and represent it in a single frame, containing temporal information. This work seeks to employ OF maps as an additional input to the nnU-Net architecture to improve its performance in the surgical instrument segmentation task.
  arXiv  Detail & Related papers  (2024-03-15T11:36:26Z)
• SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challenge [72.97934765570069]
  We release the first multimodal, publicly available, in-vivo, dataset for surgical action recognition and semantic instrumentation segmentation, containing 50 suturing video segments of Robotic Assisted Radical Prostatectomy (RARP) The aim of the challenge is to enable researchers to leverage the scale of the provided dataset and develop robust and highly accurate single-task action recognition and tool segmentation approaches in the surgical domain. A total of 12 teams participated in the challenge, contributing 7 action recognition methods, 9 instrument segmentation techniques, and 4 multitask approaches that integrated both action recognition and instrument segmentation.
  arXiv  Detail & Related papers  (2023-12-31T13:32:18Z)
• Visual-Kinematics Graph Learning for Procedure-agnostic Instrument Tip Segmentation in Robotic Surgeries [29.201385352740555]
  We propose a novel visual-kinematics graph learning framework to accurately segment the instrument tip given various surgical procedures. Specifically, a graph learning framework is proposed to encode relational features of instrument parts from both image and kinematics. A cross-modal contrastive loss is designed to incorporate robust geometric prior from kinematics to image for tip segmentation.
  arXiv  Detail & Related papers  (2023-09-02T14:52:58Z)
• Surgical tool classification and localization: results and methods from the MICCAI 2022 SurgToolLoc challenge [69.91670788430162]
  We present the results of the SurgLoc 2022 challenge. The goal was to leverage tool presence data as weak labels for machine learning models trained to detect tools. We conclude by discussing these results in the broader context of machine learning and surgical data science.
  arXiv  Detail & Related papers  (2023-05-11T21:44:39Z)
• Computer Vision on X-ray Data in Industrial Production and Security Applications: A survey [89.45221564651145]
  This survey reviews the recent research on using computer vision and machine learning for X-ray analysis in industrial production and security applications. It covers the applications, techniques, evaluation metrics, datasets, and performance comparison of those techniques on publicly available datasets.
  arXiv  Detail & Related papers  (2022-11-10T13:37:36Z)
• Co-Generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data [49.419268399590045]
  Surgical instrument segmentation for robot-assisted surgery is needed for accurate instrument tracking and augmented reality overlays. Deep learning-based methods have shown state-of-the-art performance for surgical instrument segmentation, but their results depend on labelled data. In this paper, we demonstrate the limited generalizability of these methods on different datasets, including human robot-assisted surgeries.
  arXiv  Detail & Related papers  (2021-03-16T18:41:18Z)
• Gesture Recognition in Robotic Surgery: a Review [16.322875532481326]
  This paper reviews the state-of-the-art in methods for automatic recognition of fine-grained gestures in robotic surgery. The research field is showing rapid expansion, with the majority of articles published in the last 4 years.
  arXiv  Detail & Related papers  (2021-01-29T19:13:13Z)
• A survey on shape-constraint deep learning for medical image segmentation [0.46023882211671957]
  convolutional deep neural networks and its many variants have completely changed the modern landscape of deep learning based medical image segmentation. The over dependence of these methods on pixel level classification and regression has been identified early on as a problem. To ensure the segmentation result is anatomically consistent, approaches based on Markov/ Conditional Random Fields, Statistical Shape Models are becoming increasingly popular.
  arXiv  Detail & Related papers  (2021-01-19T16:52:10Z)
• Medical Instrument Detection in Ultrasound-Guided Interventions: A Review [74.22397862400177]
  This article reviews medical instrument detection methods in the ultrasound-guided intervention. First, we present a comprehensive review of instrument detection methodologies, which include traditional non-data-driven methods and data-driven methods. We discuss the main clinical applications of medical instrument detection in ultrasound, including anesthesia, biopsy, prostate brachytherapy, and cardiac catheterization.
  arXiv  Detail & Related papers  (2020-07-09T13:50:18Z)
• Robust Medical Instrument Segmentation Challenge 2019 [56.148440125599905]
  Intraoperative tracking of laparoscopic instruments is often a prerequisite for computer and robotic-assisted interventions. Our challenge was based on a surgical data set comprising 10,040 annotated images acquired from a total of 30 surgical procedures. The results confirm the initial hypothesis, namely that algorithm performance degrades with an increasing domain gap.
  arXiv  Detail & Related papers  (2020-03-23T14:35:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.