When do they StOP?: A First Step Towards Automatically Identifying Team Communication in the Operating Room

• URL: http://arxiv.org/abs/2502.08299v2
• Date: Thu, 13 Feb 2025 09:28:41 GMT
• Title: When do they StOP?: A First Step Towards Automatically Identifying Team Communication in the Operating Room
• Abstract summary: We generate an OR dataset of real surgeries, called Team-OR, with more than one hundred hours of surgical videos captured by the multi-view camera system in the OR.<n>The dataset contains temporal annotations of 33 Time-out and 22 StOP?-protocol activities in total.<n>We then propose a novel group activity detection approach, where we encode both scene context and action features, and use an efficient neural network model to output the results.
• Score: 2.508931612863749
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Purpose: Surgical performance depends not only on surgeons' technical skills but also on team communication within and across the different professional groups present during the operation. Therefore, automatically identifying team communication in the OR is crucial for patient safety and advances in the development of computer-assisted surgical workflow analysis and intra-operative support systems. To take the first step, we propose a new task of detecting communication briefings involving all OR team members, i.e. the team Time-out and the StOP?-protocol, by localizing their start and end times in video recordings of surgical operations. Methods: We generate an OR dataset of real surgeries, called Team-OR, with more than one hundred hours of surgical videos captured by the multi-view camera system in the OR. The dataset contains temporal annotations of 33 Time-out and 22 StOP?-protocol activities in total. We then propose a novel group activity detection approach, where we encode both scene context and action features, and use an efficient neural network model to output the results. Results: The experimental results on the Team-OR dataset show that our approach outperforms existing state-of-the-art temporal action detection approaches. It also demonstrates the lack of research on group activities in the OR, proving the significance of our dataset. Conclusion: We investigate the Team Time-Out and the StOP?-protocol in the OR, by presenting the first OR dataset with temporal annotations of group activities protocols, and introducing a novel group activity detection approach that outperforms existing approaches. Code is available at https://github.com/CAMMA-public/Team-OR.

Related papers

• Where are they looking in the operating room? [10.583550132112924]
  We introduce the concept of gaze-following to the surgical domain, and demonstrate its great potential for understanding clinical roles, surgical phases, and team communications in the OR.<n>We propose novel approaches to address clinical role prediction, surgical phase recognition, and team communication detection using a gaze-following model.
  arXiv  Detail & Related papers  (2026-04-22T13:53:49Z)
• CliniDial: A Naturally Occurring Multimodal Dialogue Dataset for Team Reflection in Action During Clinical Operation [23.006540048395472]
  To understand how the team practices teamwork during the operation, we collected CliniDial from simulations of medical operations.<n>CliniDial includes the audio data and its transcriptions, the simulated physiology signals of the patient manikins, and how the team operates from two camera angles.
  arXiv  Detail & Related papers  (2025-06-15T18:39:24Z)
• Surgeons vs. Computer Vision: A comparative analysis on surgical phase recognition capabilities [65.66373425605278]
  Automated Surgical Phase Recognition (SPR) uses Artificial Intelligence (AI) to segment the surgical workflow into its key events. Previous research has focused on short and linear surgical procedures and has not explored if temporal context influences experts' ability to better classify surgical phases. This research addresses these gaps, focusing on Robot-Assisted Partial Nephrectomy (RAPN) as a highly non-linear procedure.
  arXiv  Detail & Related papers  (2025-04-26T15:37:22Z)
• SurgRAW: Multi-Agent Workflow with Chain-of-Thought Reasoning for Surgical Intelligence [16.584722724845182]
  Integration of Vision-Language Models in surgical intelligence is hindered by hallucinations, domain knowledge gaps, and limited understanding of task interdependencies. We present SurgRAW, a CoT-driven multi-agent framework that delivers transparent, interpretable insights for most tasks in robotic-assisted surgery.
  arXiv  Detail & Related papers  (2025-03-13T11:23:13Z)
• SurgicalVLM-Agent: Towards an Interactive AI Co-Pilot for Pituitary Surgery [15.263720052126853]
  Large vision-language models (VLMs) offer a promising solution by enabling dynamic task planning and predictive decision support. We introduce SurgicalVLM-Agent, an AI co-pilot for image-guided pituitary surgery, capable of conversation, planning, and task execution.
  arXiv  Detail & Related papers  (2025-03-12T15:30:39Z)
• OSSAR: Towards Open-Set Surgical Activity Recognition in Robot-assisted Surgery [13.843251369739908]
  We introduce an innovative Open-Set Surgical Activity Recognition (OSSAR) framework. Our solution leverages the hyperspherical reciprocal point strategy to enhance the distinction between known and unknown classes in the feature space. To support our assertions, we establish an open-set surgical activity benchmark utilizing the public JIGSAWS dataset.
  arXiv  Detail & Related papers  (2024-02-10T16:23:12Z)
• SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challenge [72.97934765570069]
  We release the first multimodal, publicly available, in-vivo, dataset for surgical action recognition and semantic instrumentation segmentation, containing 50 suturing video segments of Robotic Assisted Radical Prostatectomy (RARP) The aim of the challenge is to enable researchers to leverage the scale of the provided dataset and develop robust and highly accurate single-task action recognition and tool segmentation approaches in the surgical domain. A total of 12 teams participated in the challenge, contributing 7 action recognition methods, 9 instrument segmentation techniques, and 4 multitask approaches that integrated both action recognition and instrument segmentation.
  arXiv  Detail & Related papers  (2023-12-31T13:32:18Z)
• ST(OR)2: Spatio-Temporal Object Level Reasoning for Activity Recognition in the Operating Room [6.132617753806978]
  We propose a new sample-efficient and object-based approach for surgical activity recognition in the OR. Our method focuses on the geometric arrangements between clinicians and surgical devices, thus utilizing the significant object interaction dynamics in the OR.
  arXiv  Detail & Related papers  (2023-12-19T15:33:57Z)
• TUNeS: A Temporal U-Net with Self-Attention for Video-based Surgical Phase Recognition [1.5237530964650965]
  We propose a novel approach that uses attention more effectively and does not require hand-crafted constraints.<n>TuNeS is an efficient and simple temporal model that incorporates self-attention at the core of a convolutional U-Net structure.<n>TuNeS achieves state-of-the-art results on the Cholec80 dataset.
  arXiv  Detail & Related papers  (2023-07-19T14:10:55Z)
• Detecting and Optimising Team Interactions in Software Development [58.720142291102135]
  This paper presents a data-driven approach to detect the functional interaction structure for software development teams. Our approach considers differences in the activity levels of team members and uses a block-constrained configuration model. We show how our approach enables teams to compare their functional interaction structure against synthetically created benchmark scenarios.
  arXiv  Detail & Related papers  (2023-02-28T14:53:29Z)
• CholecTriplet2021: A benchmark challenge for surgical action triplet recognition [66.51610049869393]
  This paper presents CholecTriplet 2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. We present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods and 19 new deep learning algorithms are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%.
  arXiv  Detail & Related papers  (2022-04-10T18:51:55Z)
• TraSeTR: Track-to-Segment Transformer with Contrastive Query for Instance-level Instrument Segmentation in Robotic Surgery [60.439434751619736]
  We propose TraSeTR, a Track-to-Segment Transformer that exploits tracking cues to assist surgical instrument segmentation. TraSeTR jointly reasons about the instrument type, location, and identity with instance-level predictions. The effectiveness of our method is demonstrated with state-of-the-art instrument type segmentation results on three public datasets.
  arXiv  Detail & Related papers  (2022-02-17T05:52:18Z)
• Causal Scene BERT: Improving object detection by searching for challenging groups of data [125.40669814080047]
  Computer vision applications rely on learning-based perception modules parameterized with neural networks for tasks like object detection. These modules frequently have low expected error overall but high error on atypical groups of data due to biases inherent in the training process. Our main contribution is a pseudo-automatic method to discover such groups in foresight by performing causal interventions on simulated scenes.
  arXiv  Detail & Related papers  (2022-02-08T05:14:16Z)
• Gesture Recognition in Robotic Surgery: a Review [16.322875532481326]
  This paper reviews the state-of-the-art in methods for automatic recognition of fine-grained gestures in robotic surgery. The research field is showing rapid expansion, with the majority of articles published in the last 4 years.
  arXiv  Detail & Related papers  (2021-01-29T19:13:13Z)
• Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery [84.73764603474413]
  We propose a novel online approach of multi-modal graph network (i.e., MRG-Net) to dynamically integrate visual and kinematics information. The effectiveness of our method is demonstrated with state-of-the-art results on the public JIGSAWS dataset.
  arXiv  Detail & Related papers  (2020-11-03T11:00:10Z)
• Aggregating Long-Term Context for Learning Laparoscopic and Robot-Assisted Surgical Workflows [40.48632897750319]
  We propose a new temporal network structure that leverages task-specific network representation to collect long-term sufficient statistics. We demonstrate superior results over existing and novel state-of-the-art segmentation techniques on two laparoscopic cholecystectomy datasets.
  arXiv  Detail & Related papers  (2020-09-01T20:29:14Z)
• LRTD: Long-Range Temporal Dependency based Active Learning for Surgical Workflow Recognition [67.86810761677403]
  We propose a novel active learning method for cost-effective surgical video analysis. Specifically, we propose a non-local recurrent convolutional network (NL-RCNet), which introduces non-local block to capture the long-range temporal dependency. We validate our approach on a large surgical video dataset (Cholec80) by performing surgical workflow recognition task.
  arXiv  Detail & Related papers  (2020-04-21T09:21:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.