Comparative Validation of Machine Learning Algorithms for Surgical
Workflow and Skill Analysis with the HeiChole Benchmark
- URL: http://arxiv.org/abs/2109.14956v1
- Date: Thu, 30 Sep 2021 09:34:13 GMT
- Title: Comparative Validation of Machine Learning Algorithms for Surgical
Workflow and Skill Analysis with the HeiChole Benchmark
- Authors: Martin Wagner, Beat-Peter M\"uller-Stich, Anna Kisilenko, Duc Tran,
Patrick Heger, Lars M\"undermann, David M Lubotsky, Benjamin M\"uller,
Tornike Davitashvili, Manuela Capek, Annika Reinke, Tong Yu, Armine
Vardazaryan, Chinedu Innocent Nwoye, Nicolas Padoy, Xinyang Liu, Eung-Joo
Lee, Constantin Disch, Hans Meine, Tong Xia, Fucang Jia, Satoshi Kondo,
Wolfgang Reiter, Yueming Jin, Yonghao Long, Meirui Jiang, Qi Dou, Pheng Ann
Heng, Isabell Twick, Kadir Kirtac, Enes Hosgor, Jon Lindstr\"om Bolmgren,
Michael Stenzel, Bj\"orn von Siemens, Hannes G. Kenngott, Felix Nickel,
Moritz von Frankenberg, Franziska Mathis-Ullrich, Lena Maier-Hein, Stefanie
Speidel, Sebastian Bodenstedt
- Abstract summary: Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems.
We investigated the generalizability of phase recognition algorithms in a multi-center setting.
- Score: 36.37186411201134
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: PURPOSE: Surgical workflow and skill analysis are key technologies for the
next generation of cognitive surgical assistance systems. These systems could
increase the safety of the operation through context-sensitive warnings and
semi-autonomous robotic assistance or improve training of surgeons via
data-driven feedback. In surgical workflow analysis up to 91% average precision
has been reported for phase recognition on an open data single-center dataset.
In this work we investigated the generalizability of phase recognition
algorithms in a multi-center setting including more difficult recognition tasks
such as surgical action and surgical skill. METHODS: To achieve this goal, a
dataset with 33 laparoscopic cholecystectomy videos from three surgical centers
with a total operation time of 22 hours was created. Labels included annotation
of seven surgical phases with 250 phase transitions, 5514 occurences of four
surgical actions, 6980 occurences of 21 surgical instruments from seven
instrument categories and 495 skill classifications in five skill dimensions.
The dataset was used in the 2019 Endoscopic Vision challenge, sub-challenge for
surgical workflow and skill analysis. Here, 12 teams submitted their machine
learning algorithms for recognition of phase, action, instrument and/or skill
assessment. RESULTS: F1-scores were achieved for phase recognition between
23.9% and 67.7% (n=9 teams), for instrument presence detection between 38.5%
and 63.8% (n=8 teams), but for action recognition only between 21.8% and 23.3%
(n=5 teams). The average absolute error for skill assessment was 0.78 (n=1
team). CONCLUSION: Surgical workflow and skill analysis are promising
technologies to support the surgical team, but are not solved yet, as shown by
our comparison of algorithms. This novel benchmark can be used for comparable
evaluation and validation of future work.
Related papers
- SAR-RARP50: Segmentation of surgical instrumentation and Action
Recognition on Robot-Assisted Radical Prostatectomy Challenge [72.97934765570069]
We release the first multimodal, publicly available, in-vivo, dataset for surgical action recognition and semantic instrumentation segmentation, containing 50 suturing video segments of Robotic Assisted Radical Prostatectomy (RARP)
The aim of the challenge is to enable researchers to leverage the scale of the provided dataset and develop robust and highly accurate single-task action recognition and tool segmentation approaches in the surgical domain.
A total of 12 teams participated in the challenge, contributing 7 action recognition methods, 9 instrument segmentation techniques, and 4 multitask approaches that integrated both action recognition and instrument segmentation.
arXiv Detail & Related papers (2023-12-31T13:32:18Z) - Surgical tool classification and localization: results and methods from
the MICCAI 2022 SurgToolLoc challenge [69.91670788430162]
We present the results of the SurgLoc 2022 challenge.
The goal was to leverage tool presence data as weak labels for machine learning models trained to detect tools.
We conclude by discussing these results in the broader context of machine learning and surgical data science.
arXiv Detail & Related papers (2023-05-11T21:44:39Z) - Dissecting Self-Supervised Learning Methods for Surgical Computer Vision [51.370873913181605]
Self-Supervised Learning (SSL) methods have begun to gain traction in the general computer vision community.
The effectiveness of SSL methods in more complex and impactful domains, such as medicine and surgery, remains limited and unexplored.
We present an extensive analysis of the performance of these methods on the Cholec80 dataset for two fundamental and popular tasks in surgical context understanding, phase recognition and tool presence detection.
arXiv Detail & Related papers (2022-07-01T14:17:11Z) - CholecTriplet2021: A benchmark challenge for surgical action triplet
recognition [66.51610049869393]
This paper presents CholecTriplet 2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos.
We present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge.
A total of 4 baseline methods and 19 new deep learning algorithms are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%.
arXiv Detail & Related papers (2022-04-10T18:51:55Z) - Real-time Informative Surgical Skill Assessment with Gaussian Process
Learning [12.019641896240245]
This work presents a novel Gaussian Process Learning-based automatic objective surgical skill assessment method for ESSBSs.
The proposed method projects the instrument movements into the endoscope coordinate to reduce the data dimensionality.
The experimental results show that the proposed method reaches 100% prediction precision for complete surgical procedures and 90% precision for real-time prediction assessment.
arXiv Detail & Related papers (2021-12-05T15:35:40Z) - Exploring Deep Learning Methods for Real-Time Surgical Instrument
Segmentation in Laparoscopy [0.4155459804992016]
We evaluate and compare some popular deep learning methods that can be explored for the automated segmentation of surgical instruments in laparoscopy.
Our experimental results exhibit that the Dual decoder attention network (DDNet) produces a superior result compared to other recent deep learning methods.
arXiv Detail & Related papers (2021-07-05T23:32:05Z) - MIcro-Surgical Anastomose Workflow recognition challenge report [12.252332806968756]
"MIcro-Surgical Anastomose recognition on training sessions" (MISAW) challenge provided a data set of 27 sequences of micro-surgical anastomosis on artificial blood vessels.
This data set was composed of videos, kinematics, and workflow annotations described at three different granularity levels: phase, step, and activity.
The best models achieved more than 95% AD-Accuracy for phase recognition, 80% for step recognition, 60% for activity recognition, and 75% for all granularity levels.
arXiv Detail & Related papers (2021-03-24T11:34:09Z) - Surgical Skill Assessment on In-Vivo Clinical Data via the Clearness of
Operating Field [18.643159726513133]
Surgical skill assessment is studied in this paper on a real clinical dataset.
The clearness of operating field (COF) is identified as a good proxy for overall surgical skills.
An objective and automated framework is proposed to predict surgical skills through the proxy of COF.
In experiments, the proposed method achieves 0.55 Spearman's correlation with the ground truth of overall technical skill.
arXiv Detail & Related papers (2020-08-27T07:12:16Z) - Robust Medical Instrument Segmentation Challenge 2019 [56.148440125599905]
Intraoperative tracking of laparoscopic instruments is often a prerequisite for computer and robotic-assisted interventions.
Our challenge was based on a surgical data set comprising 10,040 annotated images acquired from a total of 30 surgical procedures.
The results confirm the initial hypothesis, namely that algorithm performance degrades with an increasing domain gap.
arXiv Detail & Related papers (2020-03-23T14:35:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.