Metrics Matter in Surgical Phase Recognition
- URL: http://arxiv.org/abs/2305.13961v1
- Date: Tue, 23 May 2023 11:40:12 GMT
- Title: Metrics Matter in Surgical Phase Recognition
- Authors: Isabel Funke, Dominik Rivoir and Stefanie Speidel
- Abstract summary: This paper summarizes common deviations in the evaluation of phase recognition algorithms on the Cholec80 benchmark.
Greater attention to evaluation details could help achieve more consistent and comparable results on the surgical phase recognition task.
- Score: 0.04125187280299246
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Surgical phase recognition is a basic component for different context-aware
applications in computer- and robot-assisted surgery. In recent years, several
methods for automatic surgical phase recognition have been proposed, showing
promising results. However, a meaningful comparison of these methods is
difficult due to differences in the evaluation process and incomplete reporting
of evaluation details. In particular, the details of metric computation can
vary widely between different studies. To raise awareness of potential
inconsistencies, this paper summarizes common deviations in the evaluation of
phase recognition algorithms on the Cholec80 benchmark. In addition, a
structured overview of previously reported evaluation results on Cholec80 is
provided, taking known differences in evaluation protocols into account.
Greater attention to evaluation details could help achieve more consistent and
comparable results on the surgical phase recognition task, leading to more
reliable conclusions about advancements in the field and, finally, translation
into clinical practice.
Related papers
- AI-Driven Evaluation of Surgical Skill via Action Recognition [4.92174988745803]
We propose an AI-driven framework for the automated assessment of microanastomosis performance.<n>Performance is evaluated along five aspects of microanastomosis skill, including overall action execution, motion quality during procedure-critical actions, and general instrument handling.
arXiv Detail & Related papers (2025-12-30T18:45:34Z) - Quantitative Outcome-Oriented Assessment of Microsurgical Anastomosis [7.432334662327386]
We introduce a quantitative framework that uses image-processing techniques for objective assessment of microsurgical anastomoses.<n>The approach uses geometric modeling of errors along with a detection and scoring mechanism.<n>The results show that the geometric metrics effectively replicate expert raters' scoring for the errors considered in this work.
arXiv Detail & Related papers (2025-08-26T09:14:31Z) - Surgeons vs. Computer Vision: A comparative analysis on surgical phase recognition capabilities [65.66373425605278]
Automated Surgical Phase Recognition (SPR) uses Artificial Intelligence (AI) to segment the surgical workflow into its key events.
Previous research has focused on short and linear surgical procedures and has not explored if temporal context influences experts' ability to better classify surgical phases.
This research addresses these gaps, focusing on Robot-Assisted Partial Nephrectomy (RAPN) as a highly non-linear procedure.
arXiv Detail & Related papers (2025-04-26T15:37:22Z) - SuPRA: Surgical Phase Recognition and Anticipation for Intra-Operative
Planning [46.57714869178571]
We propose a dual approach that simultaneously recognises the current surgical phase and predicts upcoming ones.
Our novel method, Surgical Phase Recognition and Anticipation (SuPRA), leverages past and current information for accurate intra-operative phase recognition.
arXiv Detail & Related papers (2024-03-10T12:46:33Z) - Multi-task Explainable Skin Lesion Classification [54.76511683427566]
We propose a few-shot-based approach for skin lesions that generalizes well with few labelled data.
The proposed approach comprises a fusion of a segmentation network that acts as an attention module and classification network.
arXiv Detail & Related papers (2023-10-11T05:49:47Z) - Surgical Phase Recognition in Laparoscopic Cholecystectomy [57.929132269036245]
We propose a Transformer-based method that utilizes calibrated confidence scores for a 2-stage inference pipeline.
Our method outperforms the baseline model on the Cholec80 dataset, and can be applied to a variety of action segmentation methods.
arXiv Detail & Related papers (2022-06-14T22:55:31Z) - CholecTriplet2021: A benchmark challenge for surgical action triplet
recognition [66.51610049869393]
This paper presents CholecTriplet 2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos.
We present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge.
A total of 4 baseline methods and 19 new deep learning algorithms are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%.
arXiv Detail & Related papers (2022-04-10T18:51:55Z) - Towards a Guideline for Evaluation Metrics in Medical Image Segmentation [0.0]
This work provides an overview and interpretation guide on the following metrics for medical image segmentation evaluation in binary.
As a summary, we propose a guideline for standardized medical image segmentation evaluation.
arXiv Detail & Related papers (2022-02-10T13:38:05Z) - Open-Set Recognition of Breast Cancer Treatments [91.3247063132127]
Open-set recognition generalizes a classification task by classifying test samples as one of the known classes from training or "unknown"
We apply a recent existing Gaussian mixture variational autoencoder model, which achieves state-of-the-art results for image datasets, to breast cancer patient data.
Not only do we obtain more accurate and robust classification results, with a 24.5% average F1 increase compared to a recent method, but we also reexamine open-set recognition in terms of deployability to a clinical setting.
arXiv Detail & Related papers (2022-01-09T04:35:55Z) - Real-time Informative Surgical Skill Assessment with Gaussian Process
Learning [12.019641896240245]
This work presents a novel Gaussian Process Learning-based automatic objective surgical skill assessment method for ESSBSs.
The proposed method projects the instrument movements into the endoscope coordinate to reduce the data dimensionality.
The experimental results show that the proposed method reaches 100% prediction precision for complete surgical procedures and 90% precision for real-time prediction assessment.
arXiv Detail & Related papers (2021-12-05T15:35:40Z) - Evaluation of Various Open-Set Medical Imaging Tasks with Deep Neural
Networks [15.655519786176438]
We conduct rigorous evaluations amongst state-of-the-art open-set methods, exploring different open-set scenarios.
We show the main difference between general domain-trained and medical domain-trained open-set models.
arXiv Detail & Related papers (2021-10-21T04:19:41Z) - Systematic Clinical Evaluation of A Deep Learning Method for Medical
Image Segmentation: Radiosurgery Application [48.89674088331313]
We systematically evaluate a Deep Learning (DL) method in a 3D medical image segmentation task.
Our method is integrated into the radiosurgery treatment process and directly impacts the clinical workflow.
arXiv Detail & Related papers (2021-08-21T16:15:40Z) - A Methodology for Bi-Directional Knowledge-Based Assessment of
Compliance to Continuous Application of Clinical Guidelines [1.52292571922932]
We introduce a new approach for automated guideline-based quality assessment of the care process.
The BiKBAC method assesses the degree of compliance when applying clinical guidelines.
The DiscovErr system was evaluated in a separate study in the type 2 diabetes management domain.
arXiv Detail & Related papers (2021-03-13T20:43:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.