Actionable Real-Time Modeling of Surgical Team Dynamics via Time-Expanded Interaction Graphs

• URL: http://arxiv.org/abs/2605.04169v1
• Date: Tue, 05 May 2026 18:05:24 GMT
• Title: Actionable Real-Time Modeling of Surgical Team Dynamics via Time-Expanded Interaction Graphs
• Abstract summary: We propose a real-time actionable approach for modeling surgical team dynamics using time-expanded interaction graphs.<n>This work advances surgical AI toward real-time, team-aware, and actionable decision support in the operating room.
• Score: 9.813151093947097
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Surgical team performance arises from complex interactions between technical execution and non-technical skills, including communication and coordination dynamics. However, current surgical AI systems predominantly model visual workflow signals, lacking structured representations of intraoperative team interactions over time. We propose a real-time actionable approach for modeling surgical team dynamics using time-expanded interaction graphs, where team members are modeled as time-indexed nodes and communication exchanges define directed edges. This spatio-temporal expansion enables dynamic interaction modeling, while allowing efficient inference with a static graph neural network. The model predicts procedural efficiency as the deviation from the expected duration and supports real-time deployment. Beyond prediction, we perform a counterfactual analysis to identify minimal changes in communication structure and interpretable behavioral variables associated with improved predicted outcomes. Experiments on recorded surgical procedures show that structured modeling of team interactions improves early identification of prolonged interventions and provides coherent, actionable explanations. This work advances surgical AI toward real-time, team-aware, and actionable decision support in the operating room.

Related papers

• Towards Actionable Surgical Team Dynamics: from Teamwork to Counterfactual Annotations [8.839435095280962]
  We introduce an extended multimodal dataset for surgical team interaction analysis, built from real operating room recordings.<n>Team performance is assessed using a standardized surgical evaluation protocol, while interaction quality and individual attributes are annotated.<n>To further support the study of coordination breakdowns and performance variability, we introduce counterfactual annotations.<n>The dataset is designed to support the study of how individual actions, interaction patterns, and team-level processes jointly contribute to team outcomes in surgical settings.
  arXiv  Detail & Related papers  (2026-08-24T14:55:18Z)
• From Multimodal Observation to Interpretable Suggestions: Counterfactual Time-Expanded Relational Modeling of Surgical Teams [9.813151093947097]
  We propose a tempo-relational framework for modeling surgical team dynamics from multimodal observations.<n>Such modeling enables the generation of efficient, interpretable, and actionable suggestions for clinicians.<n>This work advances surgical AI beyond outcome-driven prediction towards a socially grounded, team-centric, and actionable paradigm.
  arXiv  Detail & Related papers  (2026-08-24T13:46:46Z)
• Engagement Process: Rethinking the Temporal Interface of Action and Observation [26.709060042723348]
  Engagement Process (EP) represents actions and observations as decoupled event streams along time, rather than updates paired at fixed decision steps.<n>EP captures single-agent timing issues such as deliberation, latency, delayed feedback, and persistent actions.<n>EP exposes temporal behaviors hidden by step-based interfaces and enables policies to adapt under explicit time costs.
  arXiv  Detail & Related papers  (2026-05-12T04:02:03Z)
• From Brain Models to Executable Digital Twins: Execution Semantics and Neuro-Neuromorphic Systems [51.82266520875928]
  This survey introduces physically constrained executability as a unifying perspective for comparing approaches at the level of execution.<n>I propose a taxonomy of execution regimes ranging from isolated offline models to coordinated co-simulation, to continuously executing digital twins sustained by online data.<n>The executability concept clarifies why accuracy alone is insufficient, and motivates an agenda centered on semantic interoperability, hybrid-time correctness, evaluation protocols, scalable assimilation, and safe closed-loop validation.
  arXiv  Detail & Related papers  (2026-04-15T07:34:46Z)
• From Perception to Action: An Interactive Benchmark for Vision Reasoning [51.11355591375073]
  Causal Hierarchy of Actions and Interactions (CHAIN) benchmark designed to evaluate whether models can understand, plan, and execute structured action sequences grounded in physical constraints.<n> CHAIN shifts evaluation from passive perception to active problem solving, spanning tasks such as interlocking mechanical puzzles and 3D stacking and packing.<n>Our results show that top-performing models still struggle to internalize physical structure and causal constraints, often failing to produce reliable long-horizon plans and cannot robustly translate perceived structure into effective actions.
  arXiv  Detail & Related papers  (2026-02-24T15:33:02Z)
• Interact2Ar: Full-Body Human-Human Interaction Generation via Autoregressive Diffusion Models [80.28579390566298]
  We introduce Interact2Ar, a text-conditioned autoregressive diffusion model for generating full-body, human-human interactions.<n>Hand kinematics are incorporated through dedicated parallel branches, enabling high-fidelity full-body generation.<n>Our model enables a series of downstream applications, including temporal motion composition, real-time adaptation to disturbances, and extension beyond dyadic to multi-person scenarios.
  arXiv  Detail & Related papers  (2025-12-22T18:59:50Z)
• Beyond Rigid AI: Towards Natural Human-Machine Symbiosis for Interoperative Surgical Assistance [6.832434059337678]
  This work introduces a novel Perception Agent that enables a more natural human-machine interaction in real-time surgical assistance.<n>Our agent offers the flexibility to segment both known and unseen elements in the surgical scene through intuitive interaction.
  arXiv  Detail & Related papers  (2025-07-30T20:42:24Z)
• Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation [9.329654505950199]
  We propose an adaptive graph learning framework for surgical workflow anticipation based on a novel spatial representation.<n>We develop a multi-horizon objective that balances learning objectives for different time horizons, allowing for unconstrained predictions.
  arXiv  Detail & Related papers  (2024-12-09T12:53:08Z)
• Tri-modal Confluence with Temporal Dynamics for Scene Graph Generation in Operating Rooms [47.31847567531981]
  We propose a Tri-modal (i.e., images, point clouds, and language) confluence with Temporal dynamics framework, termed TriTemp-OR. Our model performs temporal interactions across 2D frames and 3D point clouds, including a scale-adaptive multi-view temporal interaction (ViewTemp) and a geometric-temporal point aggregation (PointTemp) The proposed TriTemp-OR enables the aggregation of tri-modal features through relation-aware unification to predict relations so as to generate scene graphs.
  arXiv  Detail & Related papers  (2024-04-14T12:19:16Z)
• Causal Graph ODE: Continuous Treatment Effect Modeling in Multi-agent Dynamical Systems [70.84976977950075]
  Real-world multi-agent systems are often dynamic and continuous, where the agents co-evolve and undergo changes in their trajectories and interactions over time. We propose a novel model that captures the continuous interaction among agents using a Graph Neural Network (GNN) as the ODE function. The key innovation of our model is to learn time-dependent representations of treatments and incorporate them into the ODE function, enabling precise predictions of potential outcomes.
  arXiv  Detail & Related papers  (2024-02-29T23:07:07Z)
• Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgery [47.47211257890948]
  We propose a predictive neural network that is capable of understanding and predicting critical interactive aspects of surgical workflow from intra-abdominal video.<n>We verify our approach on established surgical datasets and applications, including the detection and prediction of action triplets.<n>Our results demonstrate the superiority of our approach compared to unstructured alternatives.
  arXiv  Detail & Related papers  (2024-02-03T00:58:05Z)
• Disentangled Neural Relational Inference for Interpretable Motion Prediction [38.40799770648501]
  We develop a variational auto-encoder framework that integrates graph-based representations and timesequence models. Our model infers dynamic interaction graphs augmented with interpretable edge features that characterize the interactions. We validate our approach through extensive experiments on both simulated and real-world datasets.
  arXiv  Detail & Related papers  (2024-01-07T22:49:24Z)
• Persistent-Transient Duality: A Multi-mechanism Approach for Modeling Human-Object Interaction [58.67761673662716]
  Humans are highly adaptable, swiftly switching between different modes to handle different tasks, situations and contexts. In Human-object interaction (HOI) activities, these modes can be attributed to two mechanisms: (1) the large-scale consistent plan for the whole activity and (2) the small-scale children interactive actions that start and end along the timeline. This work proposes to model two concurrent mechanisms that jointly control human motion.
  arXiv  Detail & Related papers  (2023-07-24T12:21:33Z)
• Convolutions for Spatial Interaction Modeling [9.408751013132624]
  We consider the problem of spatial interaction modeling in the context of predicting the motion of actors around autonomous vehicles. We revisit convolutions and show that they can demonstrate comparable performance to graph networks in modeling spatial interactions with lower latency.
  arXiv  Detail & Related papers  (2021-04-15T00:41:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.