An Integrated Video-AI Platform for Action-Level Microanastomosis Training and Performance Feedback

• URL: http://arxiv.org/abs/2609.06380v1
• Date: Sun, 06 Sep 2026 04:36:53 GMT
• Title: An Integrated Video-AI Platform for Action-Level Microanastomosis Training and Performance Feedback
• Abstract summary: We present an integrated video-AI platform that turns a complete simulated procedure into inspectable, interactive feedback.<n>Object detection and tracking localize instrument tips within each action drive supervised classification of five NOMAT-aligned performance dimensions.<n>A grounded large language model (LLM) uses these structured outputs to answer user questions about the current scene, actions, motion, and predicted performance.
• Score: 6.042824968661832
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Developing microanastomosis skill requires repeated practice with timely, action-specific feedback, yet expert review of lengthy microscope videos does not scale to frequent or distributed training. We present an integrated video-AI platform that turns a complete simulated procedure into inspectable, interactive feedback through three connected modules. First, a proposed transformer segments the video into six surgical actions. Second, object detection and tracking localize instrument tips within each action; the resulting kinematic features and action statistics drive supervised classification of five NOMAT-aligned performance dimensions. Third, a grounded large language model (LLM) uses these structured outputs to answer user questions about the current scene, actions, motion, and predicted performance through a unified interface. In a two-site study, 17 participants completed 72 procedures comprising 576 suture placements. The action-segmentation module achieved 87.66\% accuracy and 82.86\% F1, increasing to 93.62\% and 88.32\% after workflow-aware refinement. The five performance classifiers achieved 76.0\% mean accuracy, with Cohen's $κ$ from 0.63 to 0.93. Although the language interface and educational benefit require prospective evaluation, these results establish the technical basis for an expert-supervised platform that can shorten review, expose the evidence behind performance estimates, and support scalable formative microsurgical training.

Related papers

• GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulation [60.95802802905149]
  We present GIFT (Guided Intermediate Feature Training), an architecture-flexible framework for learning intermediate features.<n>We instantiate GIFT in a Vision-Language-Action (VLA) policy, a direct-action World-Action Model (WAM), and an inverse-dynamics WAM while retaining each model's action formulation.<n>On RoboCasa, the three GIFT variants reach 61.4%, 83.6%, and 82.3%, outperforming their counterparts by 12.6, 9.0, and 8.4 points, respectively.
  arXiv  Detail & Related papers  (2026-09-03T17:59:03Z)
• Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent [81.19460113054048]
  Video-DR features a decoupled perception-temporal pipeline with stage-wise tool unlocking.<n>Our framework adopts a two-stage training recipe: supervised fine-tuning followed by Group Relative Policy Optimization.<n> Empirical results demonstrate that our Video-ResearchDeepB-A3B establishes a new state-of-the-art of 64.0% average accuracy.
  arXiv  Detail & Related papers  (2026-08-04T17:45:16Z)
• SkillSpotter: Pose-Aware Multi-View Skilled Action Detection and Grading in Ego-Exo Videos [27.95264551649598]
  We introduce SkillSpotter, a pose-aware multi-view architecture that jointly detects and grades skilled actions through three task-specific modules.<n>SkillSpotter improves class-specific mAP from 12.40 to 21.82 (+76%) and balanced accuracy from 55.99% to 60.40% over the best baseline.
  arXiv  Detail & Related papers  (2026-06-30T04:43:03Z)
• An Interpretable Closed-Loop Intelligent Tutoring System for Multimodal Affective Feedback in Asynchronous Presentation Training [0.0]
  The ITS maps multimodal inputs into evidence-based feedback that can be traced back to observable performance cues.<n>The system achieved rubric-aligned scoring with performance levels comparable to expert ratings.
  arXiv  Detail & Related papers  (2026-05-17T14:12:40Z)
• Video-Based Reward Modeling for Computer-Use Agents [40.27314571412647]
  We study reward modeling from execution video: a sequence of sequences from an agent trajectory independent of the agent's internal reasoning or actions.<n>We introduce Execution Video Reward 53k (ExeVR-53k), a dataset of 53k high-quality video--task--reward triplets.<n>Building on these components, we fine-tune an Execution Video Model (ExeVRM) that takes only a user instruction and a video-execution sequence to predict task success.
  arXiv  Detail & Related papers  (2026-03-10T19:17:22Z)
• AI-Driven Evaluation of Surgical Skill via Action Recognition [4.92174988745803]
  We propose an AI-driven framework for the automated assessment of microanastomosis performance.<n>Performance is evaluated along five aspects of microanastomosis skill, including overall action execution, motion quality during procedure-critical actions, and general instrument handling.
  arXiv  Detail & Related papers  (2025-12-30T18:45:34Z)
• VideoAgentTrek: Computer Use Pretraining from Unlabeled Videos [62.29924199978745]
  VideoAgentTrek is a scalable pipeline that automatically mines training data from publicly available screen-recorded videos at web scale.<n>Our approach addresses a key challenge: raw videos contain implicit demonstrations but lack explicit action labels.<n>applied to 39,000 YouTube tutorial videos, our pipeline generates 1.52 million interaction steps automatically.
  arXiv  Detail & Related papers  (2025-10-22T11:25:48Z)
• MoSFormer: Augmenting Temporal Context with Memory of Surgery for Surgical Phase Recognition [6.913838841605972]
  Memory of Surgery (MoS) is a framework that enriches temporal modeling by incorporating both semantic interpretable long-term surgical history and short-term impressions.<n>MoSFormer demonstrates state-of-the-art performance on multiple benchmarks.
  arXiv  Detail & Related papers  (2025-03-02T02:26:21Z)
• VEDIT: Latent Prediction Architecture For Procedural Video Representation Learning [59.68917139718813]
  We show that a strong off-the-shelf frozen pretrained visual encoder can achieve state-of-the-art (SoTA) performance in forecasting and procedural planning. By conditioning on frozen clip-level embeddings from observed steps to predict the actions of unseen steps, our prediction model is able to learn robust representations for forecasting.
  arXiv  Detail & Related papers  (2024-10-04T14:52:09Z)
• Early Action Recognition with Action Prototypes [62.826125870298306]
  We propose a novel model that learns a prototypical representation of the full action for each class. We decompose the video into short clips, where a visual encoder extracts features from each clip independently. Later, a decoder aggregates together in an online fashion features from all the clips for the final class prediction.
  arXiv  Detail & Related papers  (2023-12-11T18:31:13Z)
• It Takes Two: Masked Appearance-Motion Modeling for Self-supervised Video Transformer Pre-training [76.69480467101143]
  Self-supervised video transformer pre-training has recently benefited from the mask-and-predict pipeline. We explicitly investigate motion cues in videos as extra prediction target and propose our Masked Appearance-Motion Modeling framework. Our method learns generalized video representations and achieves 82.3% on Kinects-400, 71.3% on Something-Something V2, 91.5% on UCF101, and 62.5% on HMDB51.
  arXiv  Detail & Related papers  (2022-10-11T08:05:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.