Video-based Formative and Summative Assessment of Surgical Tasks using Deep Learning

• URL: http://arxiv.org/abs/2203.09589v1
• Date: Thu, 17 Mar 2022 20:07:48 GMT
• Title: Video-based Formative and Summative Assessment of Surgical Tasks using Deep Learning
• Authors: Erim Yanik, Uwe Kruger, Xavier Intes, Rahul Rahul, and Suvranu De
• Abstract summary: We propose a deep learning (DL) model that can automatically and objectively provide a high-stakes summative assessment of surgical skill execution. Formative assessment is generated using heatmaps of visual features that correlate with surgical performance.
• Score: 0.8612287536028312
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To ensure satisfactory clinical outcomes, surgical skill assessment must be objective, time-efficient, and preferentially automated - none of which is currently achievable. Video-based assessment (VBA) is being deployed in intraoperative and simulation settings to evaluate technical skill execution. However, VBA remains manually- and time-intensive and prone to subjective interpretation and poor inter-rater reliability. Herein, we propose a deep learning (DL) model that can automatically and objectively provide a high-stakes summative assessment of surgical skill execution based on video feeds and low-stakes formative assessment to guide surgical skill acquisition. Formative assessment is generated using heatmaps of visual features that correlate with surgical performance. Hence, the DL model paves the way to the quantitative and reproducible evaluation of surgical tasks from videos with the potential for broad dissemination in surgical training, certification, and credentialing.

Related papers

• Deep Learning for Surgical Instrument Recognition and Segmentation in Robotic-Assisted Surgeries: A Systematic Review [0.24342814271497581]
  Applying deep learning (DL) for annotating surgical instruments in robot-assisted minimally invasive surgeries represents a significant advancement in surgical technology. These sophisticated DL models have shown notable improvements in the precision and efficiency of detecting and segmenting surgical tools. The application of DL in surgical education is transformative.
  arXiv  Detail & Related papers  (2024-10-09T04:07:38Z)
• ZEAL: Surgical Skill Assessment with Zero-shot Tool Inference Using Unified Foundation Model [0.07143413923310668]
  This study introduces ZEAL (surgical skill assessment with Zero-shot surgical tool segmentation with a unifiEd foundAtion modeL) ZEAL predicts segmentation masks, capturing essential features of both instruments and surroundings. It produces a surgical skill score, offering an objective measure of proficiency.
  arXiv  Detail & Related papers  (2024-07-03T01:20:56Z)
• Video-based Surgical Skill Assessment using Tree-based Gaussian Process Classifier [2.3964255330849356]
  This paper presents a novel pipeline for automated surgical skill assessment using video data. The pipeline incorporates a representation flow convolutional neural network and a novel tree-based Gaussian process classifier. The proposed method has the potential to facilitate skill improvement among surgery fellows and enhance patient safety.
  arXiv  Detail & Related papers  (2023-12-15T21:06:22Z)
• Deep Multimodal Fusion for Surgical Feedback Classification [70.53297887843802]
  We leverage a clinically-validated five-category classification of surgical feedback. We then develop a multi-label machine learning model to classify these five categories of surgical feedback from inputs of text, audio, and video modalities. The ultimate goal of our work is to help automate the annotation of real-time contextual surgical feedback at scale.
  arXiv  Detail & Related papers  (2023-12-06T01:59:47Z)
• Video-based Surgical Skills Assessment using Long term Tool Tracking [0.3324986723090368]
  We introduce a motion-based approach to automatically assess surgical skills from surgical case video feed. The proposed pipeline first tracks surgical tools reliably to create motion trajectories. We compare transformer-based skill assessment with traditional machine learning approaches using the proposed and state-of-the-art tracking.
  arXiv  Detail & Related papers  (2022-07-05T18:15:28Z)
• CholecTriplet2021: A benchmark challenge for surgical action triplet recognition [66.51610049869393]
  This paper presents CholecTriplet 2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. We present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods and 19 new deep learning algorithms are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%.
  arXiv  Detail & Related papers  (2022-04-10T18:51:55Z)
• Deep Neural Networks for the Assessment of Surgical Skills: A Systematic Review [6.815366422701539]
  We have reviewed 530 papers, of which we selected 25 for this systematic review. We concluded that Deep Neural Networks are powerful tools for automated, objective surgical skill assessment using both kinematic and video data. The field would benefit from large, publicly available, annotated datasets that are representative of the surgical trainee and expert demographics and multimodal data beyond kinematics and videos.
  arXiv  Detail & Related papers  (2021-03-03T10:08:37Z)
• One-shot action recognition towards novel assistive therapies [63.23654147345168]
  This work is motivated by the automated analysis of medical therapies that involve action imitation games. The presented approach incorporates a pre-processing step that standardizes heterogeneous motion data conditions. We evaluate the approach on a real use-case of automated video analysis for therapy support with autistic people.
  arXiv  Detail & Related papers  (2021-02-17T19:41:37Z)
• LRTD: Long-Range Temporal Dependency based Active Learning for Surgical Workflow Recognition [67.86810761677403]
  We propose a novel active learning method for cost-effective surgical video analysis. Specifically, we propose a non-local recurrent convolutional network (NL-RCNet), which introduces non-local block to capture the long-range temporal dependency. We validate our approach on a large surgical video dataset (Cholec80) by performing surgical workflow recognition task.
  arXiv  Detail & Related papers  (2020-04-21T09:21:22Z)
• A Review of Computational Approaches for Evaluation of Rehabilitation Exercises [58.720142291102135]
  This paper reviews computational approaches for evaluating patient performance in rehabilitation programs using motion capture systems. The reviewed computational methods for exercise evaluation are grouped into three main categories: discrete movement score, rule-based, and template-based approaches.
  arXiv  Detail & Related papers  (2020-02-29T22:18:56Z)
• Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Search [63.07088785532908]
  We propose a framework based on reinforcement learning and tree search for joint surgical gesture segmentation and classification. Our framework consistently outperforms the existing methods on the suturing task of JIGSAWS dataset in terms of accuracy, edit score and F1 score.
  arXiv  Detail & Related papers  (2020-02-20T13:12:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.