Real-Time Segmentation of Non-Rigid Surgical Tools based on Deep Learning and Tracking

• URL: http://arxiv.org/abs/2009.03016v1
• Date: Mon, 7 Sep 2020 11:06:14 GMT
• Title: Real-Time Segmentation of Non-Rigid Surgical Tools based on Deep Learning and Tracking
• Authors: Luis C. Garc\'ia-Peraza-Herrera, Wenqi Li, Caspar Gruijthuijsen, Alain Devreker, George Attilakos, Jan Deprest, Emmanuel Vander Poorten, Danail Stoyanov, Tom Vercauteren, S\'ebastien Ourselin
• Abstract summary: Real-time tool segmentation is an essential component in computer-assisted surgical systems. We propose a novel real-time automatic method based on Fully Convolutional Networks (FCN) and optical flow tracking.
• Score: 12.408997542491152
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-time tool segmentation is an essential component in computer-assisted surgical systems. We propose a novel real-time automatic method based on Fully Convolutional Networks (FCN) and optical flow tracking. Our method exploits the ability of deep neural networks to produce accurate segmentations of highly deformable parts along with the high speed of optical flow. Furthermore, the pre-trained FCN can be fine-tuned on a small amount of medical images without the need to hand-craft features. We validated our method using existing and new benchmark datasets, covering both ex vivo and in vivo real clinical cases where different surgical instruments are employed. Two versions of the method are presented, non-real-time and real-time. The former, using only deep learning, achieves a balanced accuracy of 89.6% on a real clinical dataset, outperforming the (non-real-time) state of the art by 3.8% points. The latter, a combination of deep learning with optical flow tracking, yields an average balanced accuracy of 78.2% across all the validated datasets.

Related papers

• Automated Surgical Skill Assessment in Endoscopic Pituitary Surgery using Real-time Instrument Tracking on a High-fidelity Bench-top Phantom [9.41936397281689]
  Improved surgical skill is generally associated with improved patient outcomes, but assessment is subjective and labour-intensive. A new public dataset is introduced, focusing on simulated surgery, using the nasal phase of endoscopic pituitary surgery as an exemplar. A Multilayer Perceptron achieved 87% accuracy in predicting surgical skill level (novice or expert), with the "ratio of total procedure time to instrument visible time" correlated with higher surgical skill.
  arXiv  Detail & Related papers  (2024-09-25T15:27:44Z)
• Efficient Deformable Tissue Reconstruction via Orthogonal Neural Plane [58.871015937204255]
  We introduce Fast Orthogonal Plane (plane) for the reconstruction of deformable tissues. We conceptualize surgical procedures as 4D volumes, and break them down into static and dynamic fields comprised of neural planes. This factorization iscretizes four-dimensional space, leading to a decreased memory usage and faster optimization.
  arXiv  Detail & Related papers  (2023-12-23T13:27:50Z)
• Multicenter automatic detection of invasive carcinoma on breast whole slide images [0.0]
  It is challenging to develop fast and reliable algorithms that can be trusted by practitioners. We describe a patch-based algorithm that incorporates a convolutional neural network to detect and locate invasive carcinoma on breast whole-slide images.
  arXiv  Detail & Related papers  (2023-01-17T10:30:34Z)
• Dissecting Self-Supervised Learning Methods for Surgical Computer Vision [51.370873913181605]
  Self-Supervised Learning (SSL) methods have begun to gain traction in the general computer vision community. The effectiveness of SSL methods in more complex and impactful domains, such as medicine and surgery, remains limited and unexplored. We present an extensive analysis of the performance of these methods on the Cholec80 dataset for two fundamental and popular tasks in surgical context understanding, phase recognition and tool presence detection.
  arXiv  Detail & Related papers  (2022-07-01T14:17:11Z)
• Automatic identification of segmentation errors for radiotherapy using geometric learning [0.0]
  The aim of this work was to develop a tool to automatically identify errors in 3D OAR segmentations without a ground truth. The proposed model is trained using self-supervised learning using a synthetically-generated dataset of segmentations of the parotid. Our best performing model predicted errors on the parotid gland with a precision of 85.0% & 89.7% for internal and external errors respectively.
  arXiv  Detail & Related papers  (2022-06-27T14:01:52Z)
• Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phases [57.90226879210227]
  FedCy is a semi-supervised learning (FSSL) method that combines FL and self-supervised learning to exploit a decentralized dataset of both labeled and unlabeled videos. We demonstrate significant performance gains over state-of-the-art FSSL methods on the task of automatic recognition of surgical phases.
  arXiv  Detail & Related papers  (2022-03-14T17:44:53Z)
• Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network [51.44506007844284]
  We propose a shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection surgery. We first devise an algorithm to automatically generate relation keypoint heatmaps, which intuitively represent the prior knowledge of spatial relations among landmarks. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process.
  arXiv  Detail & Related papers  (2021-11-08T07:57:30Z)
• SurgeonAssist-Net: Towards Context-Aware Head-Mounted Display-Based Augmented Reality for Surgical Guidance [18.060445966264727]
  SurgeonAssist-Net is a framework making action-and-workflow-driven virtual assistance accessible to commercially available optical see-through head-mounted displays (OST-HMDs) Our implementation competes with state-of-the-art approaches in prediction accuracy for automated task recognition. It is capable of near real-time performance on the Microsoft HoloLens 2 OST-HMD.
  arXiv  Detail & Related papers  (2021-07-13T21:12:34Z)
• A parameter refinement method for Ptychography based on Deep Learning concepts [55.41644538483948]
  coarse parametrisation in propagation distance, position errors and partial coherence frequently menaces the experiment viability. A modern Deep Learning framework is used to correct autonomously the setup incoherences, thus improving the quality of a ptychography reconstruction. We tested our system on both synthetic datasets and also on real data acquired at the TwinMic beamline of the Elettra synchrotron facility.
  arXiv  Detail & Related papers  (2021-05-18T10:15:17Z)
• Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery [58.63306322525082]
  Most applications rely on accurate real-time segmentation of high-resolution surgical images. We design a light-weight and highly-efficient deep residual architecture which is tuned to perform real-time inference of high-resolution images.
  arXiv  Detail & Related papers  (2020-07-08T21:38:29Z)
• End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention [26.467626509096043]
  We present FW-Net, an end-to-end and real-time deep learning framework for endovascular intervention. We show that by effectively learning temporal continuity, the network can successfully segment and track the catheters in real-time sequences using only raw ground-truth for training.
  arXiv  Detail & Related papers  (2020-06-16T12:53:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.