Video-based assessment of intraoperative surgical skill

• URL: http://arxiv.org/abs/2205.06416v1
• Date: Fri, 13 May 2022 01:45:22 GMT
• Title: Video-based assessment of intraoperative surgical skill
• Authors: Sanchit Hira, Digvijay Singh, Tae Soo Kim, Shobhit Gupta, Gregory Hager, Shameema Sikder, S. Swaroop Vedula
• Abstract summary: We present and validate two deep learning methods that directly assess skill using RGB videos. In the first method, we predict instrument tips as keypoints, and learn surgical skill using temporal convolutional neural networks. In the second method, we propose a novel architecture for surgical skill assessment that includes a frame-wise encoder (2D convolutional neural network) followed by a temporal model (recurrent neural network)
• Score: 7.79874072121082
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Purpose: The objective of this investigation is to provide a comprehensive analysis of state-of-the-art methods for video-based assessment of surgical skill in the operating room. Methods: Using a data set of 99 videos of capsulorhexis, a critical step in cataract surgery, we evaluate feature based methods previously developed for surgical skill assessment mostly under benchtop settings. In addition, we present and validate two deep learning methods that directly assess skill using RGB videos. In the first method, we predict instrument tips as keypoints, and learn surgical skill using temporal convolutional neural networks. In the second method, we propose a novel architecture for surgical skill assessment that includes a frame-wise encoder (2D convolutional neural network) followed by a temporal model (recurrent neural network), both of which are augmented by visual attention mechanisms. We report the area under the receiver operating characteristic curve, sensitivity, specificity, and predictive values with each method through 5-fold cross-validation. Results: For the task of binary skill classification (expert vs. novice), deep neural network based methods exhibit higher AUC than the classical spatiotemporal interest point based methods. The neural network approach using attention mechanisms also showed high sensitivity and specificity. Conclusion: Deep learning methods are necessary for video-based assessment of surgical skill in the operating room. Our findings of internal validity of a network using attention mechanisms to assess skill directly using RGB videos should be evaluated for external validity in other data sets.

Related papers

• Cognitive-Motor Integration in Assessing Bimanual Motor Skills [0.0]
  This study introduces a novel approach by leveraging deep neural networks (DNNs) to analyze and integrate both cognitive decision-making and motor execution. We tested this methodology by assessing laparoscopic surgery skills within the Fundamentals of Laparoscopic Surgery program.
  arXiv  Detail & Related papers  (2024-04-16T20:20:23Z)
• Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgery [50.3022015601057]
  We propose a predictive neural network that is capable of understanding and predicting critical interactive aspects of surgical workflow from intra-abdominal video. We verify our approach on established surgical datasets and applications, including the detection and prediction of action triplets. Our results demonstrate the superiority of our approach compared to unstructured alternatives.
  arXiv  Detail & Related papers  (2024-02-03T00:58:05Z)
• Video-based Surgical Skills Assessment using Long term Tool Tracking [0.3324986723090368]
  We introduce a motion-based approach to automatically assess surgical skills from surgical case video feed. The proposed pipeline first tracks surgical tools reliably to create motion trajectories. We compare transformer-based skill assessment with traditional machine learning approaches using the proposed and state-of-the-art tracking.
  arXiv  Detail & Related papers  (2022-07-05T18:15:28Z)
• Dissecting Self-Supervised Learning Methods for Surgical Computer Vision [51.370873913181605]
  Self-Supervised Learning (SSL) methods have begun to gain traction in the general computer vision community. The effectiveness of SSL methods in more complex and impactful domains, such as medicine and surgery, remains limited and unexplored. We present an extensive analysis of the performance of these methods on the Cholec80 dataset for two fundamental and popular tasks in surgical context understanding, phase recognition and tool presence detection.
  arXiv  Detail & Related papers  (2022-07-01T14:17:11Z)
• End-to-End Blind Quality Assessment for Laparoscopic Videos using Neural Networks [9.481148895837812]
  We propose in this paper neural network-based approaches for distortion classification as well as quality prediction. To train the overall architecture (ResNet and FCNN models), transfer learning and end-to-end learning approaches are investigated. Experimental results, carried out on a new laparoscopic video quality database, have shown the efficiency of the proposed methods.
  arXiv  Detail & Related papers  (2022-02-09T15:29:02Z)
• A Deep Learning Approach to Predicting Collateral Flow in Stroke Patients Using Radiomic Features from Perfusion Images [58.17507437526425]
  Collateral circulation results from specialized anastomotic channels which provide oxygenated blood to regions with compromised blood flow. The actual grading is mostly done through manual inspection of the acquired images. We present a deep learning approach to predicting collateral flow grading in stroke patients based on radiomic features extracted from MR perfusion data.
  arXiv  Detail & Related papers  (2021-10-24T18:58:40Z)
• Deep Neural Networks for the Assessment of Surgical Skills: A Systematic Review [6.815366422701539]
  We have reviewed 530 papers, of which we selected 25 for this systematic review. We concluded that Deep Neural Networks are powerful tools for automated, objective surgical skill assessment using both kinematic and video data. The field would benefit from large, publicly available, annotated datasets that are representative of the surgical trainee and expert demographics and multimodal data beyond kinematics and videos.
  arXiv  Detail & Related papers  (2021-03-03T10:08:37Z)
• Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery [84.73764603474413]
  We propose a novel online approach of multi-modal graph network (i.e., MRG-Net) to dynamically integrate visual and kinematics information. The effectiveness of our method is demonstrated with state-of-the-art results on the public JIGSAWS dataset.
  arXiv  Detail & Related papers  (2020-11-03T11:00:10Z)
• Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detection [30.042965489804356]
  We propose a solution to the tool detection and localization open problem in RAS video understanding. We propose an architecture using multimodal convolutional neural networks for fast detection and localization of tools in RAS videos. Our results with an Average Precision (AP) of 91% and a mean time of 0.1 seconds per test frame detection indicate that our study is superior to conventionally used methods for medical imaging.
  arXiv  Detail & Related papers  (2020-07-29T10:59:15Z)
• LRTD: Long-Range Temporal Dependency based Active Learning for Surgical Workflow Recognition [67.86810761677403]
  We propose a novel active learning method for cost-effective surgical video analysis. Specifically, we propose a non-local recurrent convolutional network (NL-RCNet), which introduces non-local block to capture the long-range temporal dependency. We validate our approach on a large surgical video dataset (Cholec80) by performing surgical workflow recognition task.
  arXiv  Detail & Related papers  (2020-04-21T09:21:22Z)
• Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Search [63.07088785532908]
  We propose a framework based on reinforcement learning and tree search for joint surgical gesture segmentation and classification. Our framework consistently outperforms the existing methods on the suturing task of JIGSAWS dataset in terms of accuracy, edit score and F1 score.
  arXiv  Detail & Related papers  (2020-02-20T13:12:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.