The SAGES Critical View of Safety Challenge: A Global Benchmark for AI-Assisted Surgical Quality Assessment

• URL: http://arxiv.org/abs/2509.17100v1
• Date: Sun, 21 Sep 2025 14:41:26 GMT
• Title: The SAGES Critical View of Safety Challenge: A Global Benchmark for AI-Assisted Surgical Quality Assessment
• Abstract summary: This study presents the SAGES Critical View of Safety (CVS) Challenge, the first AI competition organized by a surgical society.<n>A global collaboration across 54 institutions in 24 countries engaged hundreds of clinicians and engineers to curate 1,000 videos annotated by 20 surgical experts.<n>Thirteen international teams participated, achieving up to a 17% relative gain in assessment performance, over 80% reduction in calibration error, and a 17% relative improvement in robustness over the state-of-the-art.
• Score: 16.679380065522107
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Advances in artificial intelligence (AI) for surgical quality assessment promise to democratize access to expertise, with applications in training, guidance, and accreditation. This study presents the SAGES Critical View of Safety (CVS) Challenge, the first AI competition organized by a surgical society, using the CVS in laparoscopic cholecystectomy, a universally recommended yet inconsistently performed safety step, as an exemplar of surgical quality assessment. A global collaboration across 54 institutions in 24 countries engaged hundreds of clinicians and engineers to curate 1,000 videos annotated by 20 surgical experts according to a consensus-validated protocol. The challenge addressed key barriers to real-world deployment in surgery, including achieving high performance, capturing uncertainty in subjective assessment, and ensuring robustness to clinical variability. To enable this scale of effort, we developed EndoGlacier, a framework for managing large, heterogeneous surgical video and multi-annotator workflows. Thirteen international teams participated, achieving up to a 17\% relative gain in assessment performance, over 80\% reduction in calibration error, and a 17\% relative improvement in robustness over the state-of-the-art. Analysis of results highlighted methodological trends linked to model performance, providing guidance for future research toward robust, clinically deployable AI for surgical quality assessment.

Related papers

• Explainable AI-Powered Framework for Video-Based Skill Assessment in Cataract Surgery [5.586191108738563]
  We present the world's largest dataset of cataract surgery videos, comprising 2,000 recordings.<n>We propose an AI-powered analytical framework that employs advanced computer vision and signal-processing techniques to automatically evaluate surgical videos.<n>A significant advantage of our framework over previous methods lies precisely in its explainability of outputs, elevating it beyond merely an opaque skill classification tool.
  arXiv  Detail & Related papers  (2026-08-18T08:45:27Z)
• ResidencyRL: Reinforcement Learning in Simulated Clinical Environments [53.76751756107077]
  We present ResidencyRL, a reinforcement learning (RL) method for training clinical artificial intelligence (AI) agents through simulated multi-turn clinical encounters.<n>On heldout evaluations, the ResidencyRL agent improves diagnostic accuracy by 7.0% under adversarial conditions.<n>Blinded expert clinicians validated these gains, preferring the trained agent in 87.6% of side-by-side comparisons.
  arXiv  Detail & Related papers  (2026-08-07T17:04:41Z)
• AI for Quality Assurance in the Operating Room [7.402717868707822]
  AI for quality assurance should be understood as a set of tools for augmenting the surgical team, scaling expert review, and helping surgery evolve toward a learning system.<n>This chapter introduces AI-enabled Surgical Quality Assurance as a frame-work for using surgical data to support continuous assessment and improvement in the operating room.
  arXiv  Detail & Related papers  (2026-06-16T09:05:18Z)
• A Multi-Agent LLM Framework for Rating the Quality of Surgical Feedback [52.053200111780626]
  Verbal feedback delivered by attending surgeons in the operating room plays a critical formative role in resident skill acquisition.<n>We introduce a two-stage LLM-based framework that discovers interpretable feedback quality criteria grounded in the context of surgical training.<n>Our method uses multi-agent and surgical domain knowledge injection to discover a small set of human interpretable scoring criteria.
  arXiv  Detail & Related papers  (2026-05-25T05:31:44Z)
• OSS: Open Suturing Skills Vision-Based Assessment Challenge 2024-2025 [15.333165075116964]
  We present the results of a dedicated MII challenge designed to benchmark and advance vision-based skill assessment in open surgery.<n>The challenge dataset comprises of an open suturing training task recorded with a static GoPro camera in a dry-lab setting.<n>Participants submitted diverse solutions including deep learning-based video models, tracking-driven methods, and hybrid approaches.
  arXiv  Detail & Related papers  (2026-05-21T09:04:17Z)
• Cerebra: A Multidisciplinary AI Board for Multimodal Dementia Characterization and Risk Assessment [56.62016795093786]
  Cerebra is an interactive multi-agent AI team that coordinates specialized agents for EHR, clinical notes, and medical imaging analysis.<n>It supports privacy-preserving deployment by operating on structured representations and remains robust when modalities are incomplete.<n>Cerebra significantly improved expert performance, increasing accuracy by 17.5 percentage points in prospective dementia risk estimation.
  arXiv  Detail & Related papers  (2026-03-23T05:46:45Z)
• ClinConsensus: A Consensus-Based Benchmark for Evaluating Chinese Medical LLMs across Difficulty Levels [39.33170904610862]
  Large language models (LLMs) are increasingly applied to health management, showing promise across disease prevention, clinical decision-making, and long-term care.<n>We introduce ClinConsensus, a Chinese medical benchmark curated, validated and quality-controlled by clinical experts.<n> ClinConsensus comprises 2500 open-ended cases spanning the full continuum of care--from prevention and intervention to long-term follow-up--covering 36 medical specialties, 12 common clinical task types, and progressively increasing levels of complexity.
  arXiv  Detail & Related papers  (2026-03-02T17:17:18Z)
• A generalizable foundation model for intraoperative understanding across surgical procedures [1.0412442875956527]
  We introduce ZEN, a generalizable foundation model for intraoperative surgical video understanding trained on more than 4 million frames from over 21 procedures.<n>ZEN consistently outperforms existing surgical foundation models and demonstrates robust cross-procedure generalization.
  arXiv  Detail & Related papers  (2026-02-14T06:52:42Z)
• Augmenting Clinical Decision-Making with an Interactive and Interpretable AI Copilot: A Real-World User Study with Clinicians in Nephrology and Obstetrics [36.981753143345664]
  We present AICare, an interactive and interpretable AI copilot for collaborative clinical decision-making.<n>By analyzing longitudinal electronic health records, AICare grounds dynamic risk predictions in scrutable visualizations.
  arXiv  Detail & Related papers  (2026-01-31T13:41:32Z)
• Responsible Evaluation of AI for Mental Health [72.85175110624736]
  Current approaches to evaluating AI tools in mental health care are fragmented and poorly aligned with clinical practice, social context, and first-hand user experience.<n>This paper argues for a rethinking of responsible evaluation by introducing an interdisciplinary framework that integrates clinical soundness, social context, and equity.
  arXiv  Detail & Related papers  (2026-01-20T12:55:10Z)
• DermINO: Hybrid Pretraining for a Versatile Dermatology Foundation Model [69.20140430678092]
  DermNIO is a versatile foundation model for dermatology.<n>It incorporates a novel hybrid pretraining framework that augments the self-supervised learning paradigm.<n>It consistently outperforms state-of-the-art models across a wide range of tasks.
  arXiv  Detail & Related papers  (2025-08-17T00:41:39Z)
• ReCAP: Recursive Cross Attention Network for Pseudo-Label Generation in Robotic Surgical Skill Assessment [40.09498356923132]
  In surgical skill assessment, the Objective Assessments of Technical Skills (OSATS) and Global Rating Scale (GRS) are well-established tools for evaluating surgeons during training.<n>Recent research has focused on regressing GRS scores from kinematic data, video, or their combination.<n>We argue that regressing GRS alone is limiting, as it aggregates OSATS scores and overlooks clinically meaningful variations during a surgical trial.
  arXiv  Detail & Related papers  (2024-04-22T10:33:06Z)
• Automated Assessment of Critical View of Safety in Laparoscopic Cholecystectomy [51.240181118593114]
  Cholecystectomy (gallbladder removal) is one of the most common procedures in the US, with more than 1.2M procedures annually. LC is associated with an increase in bile duct injuries (BDIs), resulting in significant morbidity and mortality. In this paper, we develop deep-learning techniques to automate the assessment of critical view of safety (CVS) in LCs.
  arXiv  Detail & Related papers  (2023-09-13T22:01:36Z)
• Validating polyp and instrument segmentation methods in colonoscopy through Medico 2020 and MedAI 2021 Challenges [58.32937972322058]
  "Medico automatic polyp segmentation (Medico 2020)" and "MedAI: Transparency in Medical Image (MedAI 2021)" competitions. We present a comprehensive summary and analyze each contribution, highlight the strength of the best-performing methods, and discuss the possibility of clinical translations of such methods into the clinic.
  arXiv  Detail & Related papers  (2023-07-30T16:08:45Z)
• CholecTriplet2021: A benchmark challenge for surgical action triplet recognition [66.51610049869393]
  This paper presents CholecTriplet 2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. We present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods and 19 new deep learning algorithms are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%.
  arXiv  Detail & Related papers  (2022-04-10T18:51:55Z)
• The SARAS Endoscopic Surgeon Action Detection (ESAD) dataset: Challenges and methods [15.833413083110903]
  This paper presents ESAD, the first large-scale dataset designed to tackle the problem of surgeon action detection in endoscopic minimally invasive surgery. The dataset provides bounding box annotation for 21 action classes on real endoscopic video frames captured during prostatectomy, and was used as the basis of a recent MIDL 2020 challenge.
  arXiv  Detail & Related papers  (2021-04-07T15:11:51Z)
• Surgical Skill Assessment on In-Vivo Clinical Data via the Clearness of Operating Field [18.643159726513133]
  Surgical skill assessment is studied in this paper on a real clinical dataset. The clearness of operating field (COF) is identified as a good proxy for overall surgical skills. An objective and automated framework is proposed to predict surgical skills through the proxy of COF. In experiments, the proposed method achieves 0.55 Spearman's correlation with the ground truth of overall technical skill.
  arXiv  Detail & Related papers  (2020-08-27T07:12:16Z)
• AGE Challenge: Angle Closure Glaucoma Evaluation in Anterior Segment Optical Coherence Tomography [61.405005501608706]
  Angle closure glaucoma (ACG) is a more aggressive disease than open-angle glaucoma. Anterior Segment Optical Coherence Tomography (AS- OCT) imaging provides a fast and contactless way to discriminate angle closure from open angle. There is no public AS- OCT dataset available for evaluating the existing methods in a uniform way. We organized the Angle closure Glaucoma Evaluation challenge (AGE), held in conjunction with MICCAI 2019.
  arXiv  Detail & Related papers  (2020-05-05T14:55:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.