Trainee Action Recognition through Interaction Analysis in CCATT Mixed-Reality Training

• URL: http://arxiv.org/abs/2509.17888v1
• Date: Mon, 22 Sep 2025 15:19:45 GMT
• Title: Trainee Action Recognition through Interaction Analysis in CCATT Mixed-Reality Training
• Authors: Divya Mereddy, Marcos Quinones-Grueiro, Ashwin T S, Eduardo Davalos, Gautam Biswas, Kent Etherton, Tyler Davis, Katelyn Kay, Jill Lear, Benjamin Goldberg,
• Abstract summary: Critical Care Air Transport Team members must stabilize severely injured soldiers by managing ventilators, IV pumps, and suction devices during flight.<n>Recent advances in simulation and multimodal data analytics enable more objective and comprehensive performance evaluation.<n>This study examines how CCATT members are trained using mixed-reality simulations that replicate the high-pressure conditions of aeromedical evacuation.
• Score: 1.5641818606249476
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study examines how Critical Care Air Transport Team (CCATT) members are trained using mixed-reality simulations that replicate the high-pressure conditions of aeromedical evacuation. Each team - a physician, nurse, and respiratory therapist - must stabilize severely injured soldiers by managing ventilators, IV pumps, and suction devices during flight. Proficient performance requires clinical expertise and cognitive skills, such as situational awareness, rapid decision-making, effective communication, and coordinated task management, all of which must be maintained under stress. Recent advances in simulation and multimodal data analytics enable more objective and comprehensive performance evaluation. In contrast, traditional instructor-led assessments are subjective and may overlook critical events, thereby limiting generalizability and consistency. However, AI-based automated and more objective evaluation metrics still demand human input to train the AI algorithms to assess complex team dynamics in the presence of environmental noise and the need for accurate re-identification in multi-person tracking. To address these challenges, we introduce a systematic, data-driven assessment framework that combines Cognitive Task Analysis (CTA) with Multimodal Learning Analytics (MMLA). We have developed a domain-specific CTA model for CCATT training and a vision-based action recognition pipeline using a fine-tuned Human-Object Interaction model, the Cascade Disentangling Network (CDN), to detect and track trainee-equipment interactions over time. These interactions automatically yield performance indicators (e.g., reaction time, task duration), which are mapped onto a hierarchical CTA model tailored to CCATT operations, enabling interpretable, domain-relevant performance evaluations.

Related papers

• Integrating Virtual Reality and Large Language Models for Team-Based Non-Technical Skills Training and Evaluation in the Operating Room [0.5943985865141843]
  We introduce the Virtual Operating Room Team Experience (VOR), a multi-user virtual reality (VR) platform that integrates immersive team simulation with behavioral analytics.<n>VOR provides a scalable, privacy-compliant framework for objective assessment and automated, data-informed debriefing.<n>Twelve surgical professionals completed pilot sessions at the 2024S conference, rating VOR as intuitive, immersive, and valuable for developing teamwork and communication.
  arXiv  Detail & Related papers  (2026-01-19T21:34:00Z)
• Bridging the Knowledge-Action Gap by Evaluating LLMs in Dynamic Dental Clinical Scenarios [9.865786198063644]
  The transition of Large Language Models (LLMs) from passive knowledge retrievers to autonomous clinical agents demands a shift in evaluation-from static accuracy to dynamic behavioral reliability.<n>This study empirically charts the capability boundaries of dental LLMs, providing a roadmap for bridging the gap between standardized knowledge and safe, autonomous clinical practice.
  arXiv  Detail & Related papers  (2026-01-19T11:36:39Z)
• Video-Based Performance Evaluation for ECR Drills in Synthetic Training Environments [1.6162271703130058]
  This paper introduces a video-based assessment pipeline that derives performance analytics from training videos without requiring additional hardware.<n>We develop task-specific metrics that measure psychomotor fluency, situational awareness, and team coordination.<n>Future work includes expanding analysis to 3D video data and leveraging video analysis to enable scalable evaluation within STEs.
  arXiv  Detail & Related papers  (2025-12-29T19:30:41Z)
• SelfAI: Building a Self-Training AI System with LLM Agents [79.10991818561907]
  SelfAI is a general multi-agent platform that combines a User Agent for translating high-level research objectives into standardized experimental configurations.<n>An Experiment Manager orchestrates parallel, fault-tolerant training across heterogeneous hardware while maintaining a structured knowledge base for continuous feedback.<n>Across regression, computer vision, scientific computing, medical imaging, and drug discovery benchmarks, SelfAI consistently achieves strong performance and reduces redundant trials.
  arXiv  Detail & Related papers  (2025-11-29T09:18:39Z)
• Causal Reinforcement Learning based Agent-Patient Interaction with Clinical Domain Knowledge [3.2434118923825483]
  Reinforcement Learning (RL) faces significant challenges in adaptive healthcare interventions, such as dementia care, where data is scarce.<n>We present a novel framework called Causal structure-aware Reinforcement Learning (CRL) that explicitly integrates causal discovery and reasoning into policy optimization.
  arXiv  Detail & Related papers  (2025-11-17T22:38:03Z)
• UserRL: Training Interactive User-Centric Agent via Reinforcement Learning [104.63494870852894]
  Reinforcement learning (RL) has shown promise in training agentic models that engage in dynamic, multi-turn interactions.<n>We propose UserRL, a unified framework for training and evaluating user-centric abilities through standardized gym environments.
  arXiv  Detail & Related papers  (2025-09-24T03:33:20Z)
• Automated Clinical Problem Detection from SOAP Notes using a Collaborative Multi-Agent LLM Architecture [8.072932739333309]
  We introduce a collaborative multi-agent system (MAS) that models a clinical consultation team to address this gap.<n>The system is tasked with identifying clinical problems by analyzing only the Subjective (S) and Objective (O) sections of SOAP notes.<n>A Manager agent orchestrates a dynamically assigned team of specialist agents who engage in a hierarchical, iterative debate to reach a consensus.
  arXiv  Detail & Related papers  (2025-08-29T17:31:24Z)
• AgentMental: An Interactive Multi-Agent Framework for Explainable and Adaptive Mental Health Assessment [31.920800599579906]
  Mental health assessment is crucial for early intervention and effective treatment, yet traditional clinician-based approaches are limited by the shortage of qualified professionals.<n>Recent advances in artificial intelligence have sparked growing interest in automated psychological assessment, yet most existing approaches are constrained by their reliance on static text analysis.<n>We propose a multi-agent framework for mental health evaluation that simulates clinical doctor-patient dialogues.
  arXiv  Detail & Related papers  (2025-08-15T16:20:45Z)
• Leadership Assessment in Pediatric Intensive Care Unit Team Training [18.37408109860005]
  This paper addresses the task of assessing PICU team's leadership skills by developing an automated analysis framework based on egocentric vision.<n>We identify key behavioral cues, including fixation object, eye contact, and conversation patterns, as essential indicators of leadership assessment.
  arXiv  Detail & Related papers  (2025-05-30T09:19:33Z)
• Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Grounding [53.629132242389716]
  Vision-Language Models (VLM) can support clinicians by analyzing medical images and engaging in natural language interactions. VLMs often exhibit "hallucinogenic" behavior, generating textual outputs not grounded in contextual multimodal information. We propose a new alignment algorithm that uses symbolic representations of clinical reasoning to ground VLMs in medical knowledge.
  arXiv  Detail & Related papers  (2024-05-29T23:19:28Z)
• Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation [58.21683603243387]
  We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework. These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents. Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
  arXiv  Detail & Related papers  (2023-11-27T18:57:42Z)
• Adaptive Multi-Agent Deep Reinforcement Learning for Timely Healthcare Interventions [17.405080523382235]
  We propose a novel AI-driven patient monitoring framework using multi-agent deep reinforcement learning (DRL) Our approach deploys multiple learning agents, each dedicated to monitoring a specific physiological feature, such as heart rate, respiration, and temperature. We evaluate the performance of the proposed multi-agent DRL framework using real-world physiological and motion data from two datasets.
  arXiv  Detail & Related papers  (2023-09-20T00:42:08Z)
• Efficient Adaptive Human-Object Interaction Detection with Concept-guided Memory [64.11870454160614]
  We propose an efficient Adaptive HOI Detector with Concept-guided Memory (ADA-CM) ADA-CM has two operating modes. The first mode makes it tunable without learning new parameters in a training-free paradigm. Our proposed method achieves competitive results with state-of-the-art on the HICO-DET and V-COCO datasets with much less training time.
  arXiv  Detail & Related papers  (2023-09-07T13:10:06Z)
• Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach [84.02388020258141]
  We propose a new framework named ENIGMA for estimating human evaluation scores based on off-policy evaluation in reinforcement learning. ENIGMA only requires a handful of pre-collected experience data, and therefore does not involve human interaction with the target policy during the evaluation. Our experiments show that ENIGMA significantly outperforms existing methods in terms of correlation with human evaluation scores.
  arXiv  Detail & Related papers  (2021-02-20T03:29:20Z)
• Adversarial vs behavioural-based defensive AI with joint, continual and active learning: automated evaluation of robustness to deception, poisoning and concept drift [62.997667081978825]
  Recent advancements in Artificial Intelligence (AI) have brought new capabilities to behavioural analysis (UEBA) for cyber-security. In this paper, we present a solution to effectively mitigate this attack by improving the detection process and efficiently leveraging human expertise.
  arXiv  Detail & Related papers  (2020-01-13T13:54:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.