ExpOS: Explainable Open-Surgery Skills Assessment Using 3D Hand Reconstruction

• URL: http://arxiv.org/abs/2605.23653v1
• Date: Fri, 22 May 2026 14:06:41 GMT
• Title: ExpOS: Explainable Open-Surgery Skills Assessment Using 3D Hand Reconstruction
• Abstract summary: We present ExpOS, an explainable framework for data-driven assessment of open-surgery skills.<n>We trained and evaluated the method on 221 videos of medical students performing three open-surgery tasks.
• Score: 1.9116784879310027
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Timely and transparent feedback is essential for effective surgical training, yet current assessment remains dependent on expert observation, limiting scalability and opportunities for autonomous practice. We present ExpOS, an explainable framework for data-driven assessment of open-surgery skills designed to enable automatic, feedback-oriented evaluation. Rather than relying on expert-defined metrics, ExpOS learns discriminative temporal patterns directly from motion data and identifies the segments and behaviors most predictive of skill level. We trained and evaluated the method on 221 videos of medical students performing three open-surgery tasks. Hand poses and tool detections were extracted from each frame to derive kinematic descriptors and global motion statistics. Spatiotemporal hand-tool dynamics were modeled using a temporal convolutional backbone with attention-based pooling to generate frame-level importance maps. These representations were fused with global motion statistics to predict skill level and to provide interpretable feedback. ExpOS provides multi-level explainability by identifying when informative events occur through attention weights and which motion characteristics most influence predictions through global feature analysis. Across tasks, the framework achieved strong correlation with expert ratings, with best performance on fascial closure (r = 0.778, R2 = 0.74). These results demonstrate that combining weakly-supervised temporal importance learning with interpretable motion statistics enables scalable and actionable surgical skill assessment.

Related papers

• World Models for Embodied Intelligence: From Plausible to Controllable to Actionable [100.13060753774657]
  World models connect perception and decision-making in embodied intelligence by maintaining hidden state, anticipating consequences, comparing interventions, and adapting when execution departs from expectations.<n>This raises a central question: which predictive capabilities improve behavior?<n>We introduce three progressively stronger capability levels: Plausible models preserve task-relevant temporal, geometric, or physical structure; Controllable models additionally predict how interventions alter that structure; and Actionable models translate predictions into measurable gains in planning, action, learning, evaluation, verification, recovery, or data selection.
  arXiv  Detail & Related papers  (2026-09-15T06:22:42Z)
• AI-Assisted Competency Assessment from Egocentric Video in Simulation-Based Nursing Education [9.996771938125582]
  Vision-language models have emerged as a promising tool for understanding complex visual behavior.<n>We extract action timelines from egocentric nursing simulation video using frozen visual encoders and few-shot learning.<n>We observe a negative trend between recognition accuracy and competency.
  arXiv  Detail & Related papers  (2026-05-16T02:07:35Z)
• Model selection and real-time skill assessment for suturing in robotic surgery [33.229894165318]
  We examine methods to achieve real-time prediction of surgical skill level in real-time based on OSATS scores.<n>For the model design, we evaluate the effectiveness of multimodal deep learning models for predicting surgical skill levels.<n>For the skill-level-based cross-validation, we separately trained models on surgeons with different skill levels.
  arXiv  Detail & Related papers  (2026-01-17T11:28:59Z)
• AI-Driven Evaluation of Surgical Skill via Action Recognition [4.92174988745803]
  We propose an AI-driven framework for the automated assessment of microanastomosis performance.<n>Performance is evaluated along five aspects of microanastomosis skill, including overall action execution, motion quality during procedure-critical actions, and general instrument handling.
  arXiv  Detail & Related papers  (2025-12-30T18:45:34Z)
• Attention Trajectories as a Diagnostic Axis for Deep Reinforcement Learning [4.662814261930481]
  We introduce a scientific methodology for analyzing the learning process through quantitative analysis of saliency.<n>This approach aggregates saliency information at the object and modality level into hierarchical attention profiles.<n>This methodology uncovers algorithm-specific attention biases, reveals unintended reward-driven strategies, and diagnoses overfitting to redundant sensory channels.
  arXiv  Detail & Related papers  (2025-11-25T18:20:42Z)
• Beyond Description: Cognitively Benchmarking Fine-Grained Action for Embodied Agents [52.14392337070763]
  We introduce CFG-Bench, a new benchmark designed to systematically evaluate fine-grained action intelligence.<n>CFG-Bench consists of 1,368 curated videos paired with 19,562 three-modalities question-answer pairs targeting four cognitive abilities.<n>Our comprehensive evaluation on CFG-Bench reveals that leading MLLMs struggle to produce detailed instructions for physical interactions.
  arXiv  Detail & Related papers  (2025-11-24T02:02:29Z)
• Multi-Modal Self-Supervised Learning for Surgical Feedback Effectiveness Assessment [66.6041949490137]
  We propose a method that integrates information from transcribed verbal feedback and corresponding surgical video to predict feedback effectiveness. Our findings show that both transcribed feedback and surgical video are individually predictive of trainee behavior changes. Our results demonstrate the potential of multi-modal learning to advance the automated assessment of surgical feedback.
  arXiv  Detail & Related papers  (2024-11-17T00:13:00Z)
• Prospector Heads: Generalized Feature Attribution for Large Models & Data [82.02696069543454]
  We introduce prospector heads, an efficient and interpretable alternative to explanation-based attribution methods. We demonstrate how prospector heads enable improved interpretation and discovery of class-specific patterns in input data.
  arXiv  Detail & Related papers  (2024-02-18T23:01:28Z)
• Clairvoyance: A Pipeline Toolkit for Medical Time Series [95.22483029602921]
  Time-series learning is the bread and butter of data-driven *clinical decision support* Clairvoyance proposes a unified, end-to-end, autoML-friendly pipeline that serves as a software toolkit. Clairvoyance is the first to demonstrate viability of a comprehensive and automatable pipeline for clinical time-series ML.
  arXiv  Detail & Related papers  (2023-10-28T12:08:03Z)
• A Survey of the Impact of Self-Supervised Pretraining for Diagnostic Tasks with Radiological Images [71.26717896083433]
  Self-supervised pretraining has been observed to be effective at improving feature representations for transfer learning. This review summarizes recent research into its usage in X-ray, computed tomography, magnetic resonance, and ultrasound imaging.
  arXiv  Detail & Related papers  (2023-09-05T19:45:09Z)
• Information-Theoretic Odometry Learning [83.36195426897768]
  We propose a unified information theoretic framework for learning-motivated methods aimed at odometry estimation. The proposed framework provides an elegant tool for performance evaluation and understanding in information-theoretic language.
  arXiv  Detail & Related papers  (2022-03-11T02:37:35Z)
• Simulation-to-Real domain adaptation with teacher-student learning for endoscopic instrument segmentation [1.1047993346634768]
  We introduce a teacher-student learning approach that learns jointly from annotated simulation data and unlabeled real data. Empirical results on three datasets highlight the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2021-03-02T09:30:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.