Automated Procedural Analysis via Video-Language Models for AI-assisted Nursing Skills Assessment

• URL: http://arxiv.org/abs/2509.16810v1
• Date: Sat, 20 Sep 2025 21:11:33 GMT
• Title: Automated Procedural Analysis via Video-Language Models for AI-assisted Nursing Skills Assessment
• Authors: Shen Chang, Dennis Liu, Renran Tian, Kristen L. Swartzell, Stacie L. Klingler, Amy M. Nagle, Nan Kong,
• Abstract summary: We introduce a video-language model (VLM) based framework to develop the AI capability of automated procedural assessment and feedback for nursing skills training.<n>The framework follows a curriculum-inspired progression, advancing from high-level action recognition to procedural reasoning.<n>This work advances AI applications in nursing education, contributing to stronger workforce development and ultimately safer patient care.
• Score: 5.851959409921155
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Consistent high-quality nursing care is essential for patient safety, yet current nursing education depends on subjective, time-intensive instructor feedback in training future nurses, which limits scalability and efficiency in their training, and thus hampers nursing competency when they enter the workforce. In this paper, we introduce a video-language model (VLM) based framework to develop the AI capability of automated procedural assessment and feedback for nursing skills training, with the potential of being integrated into existing training programs. Mimicking human skill acquisition, the framework follows a curriculum-inspired progression, advancing from high-level action recognition, fine-grained subaction decomposition, and ultimately to procedural reasoning. This design supports scalable evaluation by reducing instructor workload while preserving assessment quality. The system provides three core capabilities: 1) diagnosing errors by identifying missing or incorrect subactions in nursing skill instruction videos, 2) generating explainable feedback by clarifying why a step is out of order or omitted, and 3) enabling objective, consistent formative evaluation of procedures. Validation on synthesized videos demonstrates reliable error detection and temporal localization, confirming its potential to handle real-world training variability. By addressing workflow bottlenecks and supporting large-scale, standardized evaluation, this work advances AI applications in nursing education, contributing to stronger workforce development and ultimately safer patient care.

Related papers

• A Unified XAI-LLM Approach for EndotrachealSuctioning Activity Recognition [0.1794226570005898]
  This study proposes a unified framework for video-based activity recognition benchmarked against conventional machine learning and deep learning approaches.<n>Within this framework, the Large Language Model (LLM) serves as the central reasoning module, performing bothtemporal activity recognition and explainable decision analysis from video data.<n> Experimental results demonstrate that the proposed LLM-based approach outperforms baseline models, achieving an improvement of approximately 15-20% in both accuracy and F1 score.
  arXiv  Detail & Related papers  (2026-01-29T14:46:48Z)
• NurseSchedRL: Attention-Guided Reinforcement Learning for Nurse-Patient Assignment [0.0]
  NurseSchedRL is a reinforcement learning framework for nurse-patient assignment.<n>It integrates structured state encoding, constrained action masking, and attention-based representations of skills, fatigue, and geographical context.
  arXiv  Detail & Related papers  (2025-09-10T21:41:42Z)
• From Coordination to Personalization: A Trust-Aware Simulation Framework for Emergency Department Decision Support [0.0]
  This study proposes a simulation-based framework for modeling doctors and nurses as intelligent agents guided by computational trust mechanisms.<n>The proposed framework demonstrates the potential of computational trust for evidence-based decision support in emergency medicine.
  arXiv  Detail & Related papers  (2025-09-09T18:00:44Z)
• Uncertainty-Driven Expert Control: Enhancing the Reliability of Medical Vision-Language Models [52.2001050216955]
  Existing methods aim to enhance the performance of Medical Vision Language Model (MedVLM) by adjusting model structure, fine-tuning with high-quality data, or through preference fine-tuning.<n>We propose an expert-in-the-loop framework named Expert-Controlled-Free Guidance (Expert-CFG) to align MedVLM with clinical expertise without additional training.
  arXiv  Detail & Related papers  (2025-07-12T09:03:30Z)
• AutoMedEval: Harnessing Language Models for Automatic Medical Capability Evaluation [55.2739790399209]
  We present AutoMedEval, an open-sourced automatic evaluation model with 13B parameters specifically engineered to measure the question-answering proficiency of medical LLMs.<n>The overarching objective of AutoMedEval is to assess the quality of responses produced by diverse models, aspiring to significantly reduce the dependence on human evaluation.
  arXiv  Detail & Related papers  (2025-05-17T07:44:54Z)
• AI-driven Automation of End-to-end Assessment of Suturing Expertise [6.4885743283287]
  We present an AI based approach to automate the End-to-end Assessment of Suturing Expertise (EASE)<n>EASE provides granular skills assessment related to suturing to provide trainees with an objective evaluation of their aptitude along with actionable insights.<n>The AI based approach solves this by enabling real-time score prediction with minimal resources during model inference.
  arXiv  Detail & Related papers  (2025-03-17T21:28:02Z)
• Multi-Modal Self-Supervised Learning for Surgical Feedback Effectiveness Assessment [66.6041949490137]
  We propose a method that integrates information from transcribed verbal feedback and corresponding surgical video to predict feedback effectiveness. Our findings show that both transcribed feedback and surgical video are individually predictive of trainee behavior changes. Our results demonstrate the potential of multi-modal learning to advance the automated assessment of surgical feedback.
  arXiv  Detail & Related papers  (2024-11-17T00:13:00Z)
• Query-Guided Self-Supervised Summarization of Nursing Notes [5.835276312834499]
  We introduce QGSumm, a novel query-guided self-supervised domain adaptation approach for abstractive nursing note summarization.<n>We study our approach and other state-of-the-art Large Language Models (LLMs) for nursing note summarization.
  arXiv  Detail & Related papers  (2024-07-04T18:54:30Z)
• U-PASS: an Uncertainty-guided deep learning Pipeline for Automated Sleep Staging [61.6346401960268]
  We propose a machine learning pipeline called U-PASS tailored for clinical applications that incorporates uncertainty estimation at every stage of the process. We apply our uncertainty-guided deep learning pipeline to the challenging problem of sleep staging and demonstrate that it systematically improves performance at every stage.
  arXiv  Detail & Related papers  (2023-06-07T08:27:36Z)
• Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing [53.096237570992294]
  Strategy training is a rehabilitation approach that teaches skills to reduce disability among those with cognitive impairments following a stroke. Standardized fidelity assessment is used to measure adherence to treatment principles. We developed a rule-based NLP algorithm, a long-short term memory (LSTM) model, and a bidirectional encoder representation from transformers (BERT) model for this task.
  arXiv  Detail & Related papers  (2022-09-14T15:33:30Z)
• Video-based Formative and Summative Assessment of Surgical Tasks using Deep Learning [0.8612287536028312]
  We propose a deep learning (DL) model that can automatically and objectively provide a high-stakes summative assessment of surgical skill execution. Formative assessment is generated using heatmaps of visual features that correlate with surgical performance.
  arXiv  Detail & Related papers  (2022-03-17T20:07:48Z)
• Opportunities of a Machine Learning-based Decision Support System for Stroke Rehabilitation Assessment [64.52563354823711]
  Rehabilitation assessment is critical to determine an adequate intervention for a patient. Current practices of assessment mainly rely on therapist's experience, and assessment is infrequently executed due to the limited availability of a therapist. We developed an intelligent decision support system that can identify salient features of assessment using reinforcement learning.
  arXiv  Detail & Related papers  (2020-02-27T17:04:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.