Real-Time Mobile Video Analytics for Pre-arrival Emergency Medical Services

• URL: http://arxiv.org/abs/2511.14119v1
• Date: Tue, 18 Nov 2025 04:12:21 GMT
• Title: Real-Time Mobile Video Analytics for Pre-arrival Emergency Medical Services
• Abstract summary: We present TeleEMS, a mobile live video analytics system that enables pre-arrival multimodal inference.<n>The TeleEMS Client runs across phones, smart glasses, and desktops to support bystanders, EMTs en route, and 911 dispatchers.<n>On top of EMSStream, the server hosts three real-time analytics modules: audio-to-symptom analytics via EMSLlama, video-to-vital analytics using state-of-the-art r methods for heart rate estimation.
• Score: 5.251367919801455
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Timely and accurate pre-arrival video streaming and analytics are critical for emergency medical services (EMS) to deliver life-saving interventions. Yet, current-generation EMS infrastructure remains constrained by one-to-one video streaming and limited analytics capabilities, leaving dispatchers and EMTs to manually interpret overwhelming, often noisy or redundant information in high-stress environments. We present TeleEMS, a mobile live video analytics system that enables pre-arrival multimodal inference by fusing audio and video into a unified decision-making pipeline before EMTs arrive on scene. TeleEMS comprises two key components: TeleEMS Client and TeleEMS Server. The TeleEMS Client runs across phones, smart glasses, and desktops to support bystanders, EMTs en route, and 911 dispatchers. The TeleEMS Server, deployed at the edge, integrates EMS-Stream, a communication backbone that enables smooth multi-party video streaming. On top of EMSStream, the server hosts three real-time analytics modules: (1) audio-to-symptom analytics via EMSLlama, a domain-specialized LLM for robust symptom extraction and normalization; (2) video-to-vital analytics using state-of-the-art rPPG methods for heart rate estimation; and (3) joint text-vital analytics via PreNet, a multimodal multitask model predicting EMS protocols, medication types, medication quantities, and procedures. Evaluation shows that EMSLlama outperforms GPT-4o (exact-match 0.89 vs. 0.57) and that text-vital fusion improves inference robustness, enabling reliable pre-arrival intervention recommendations. TeleEMS demonstrates the potential of mobile live video analytics to transform EMS operations, bridging the gap between bystanders, dispatchers, and EMTs, and paving the way for next-generation intelligent EMS infrastructure.

Related papers

• CarePilot: A Multi-Agent Framework for Long-Horizon Computer Task Automation in Healthcare [37.42599407869901]
  Multimodal agentic pipelines are transforming human-computer interaction by enabling efficient and accessible automation of complex, real-world tasks.<n>Recent efforts have focused on short-horizon or general-purpose applications, leaving long-horizon automation for domain-specific systems, particularly in healthcare, largely unexplored.<n>We introduce CarePilot, a multi-agent framework based on the actor-critic paradigm.<n>Our experiments show that CarePilot achieves state-of-the-art performance, outperforming strong closed-source and open-source multimodal baselines by approximately 15.26% and 3.38%, respectively.
  arXiv  Detail & Related papers  (2026-03-25T10:25:48Z)
• MOSAIC: Bridging the Sim-to-Real Gap in Generalist Humanoid Motion Tracking and Teleoperation with Rapid Residual Adaptation [50.643488406438514]
  Generalist humanoid motion trackers have recently achieved strong simulation metrics by scaling data and training, yet often remain brittle on hardware during sustained teleoperation.<n>We present MOSAIC, an open-source, full-stack system for humanoid motion tracking and whole-body teleoperation across multiple interfaces.
  arXiv  Detail & Related papers  (2026-02-09T12:35:01Z)
• A Smart-Glasses for Emergency Medical Services via Multimodal Multitask Learning [7.284746127785293]
  We present EMSGlass, a smart-glasses system powered by EMSNet, and EMSServe, a low-latency multimodal serving framework tailored to EMS scenarios.<n>EMSNet integrates text, vital signs, and scene images to construct a unified real-time understanding of EMS incidents.<n>EMSServe achieves 1.9x -- 11.7x speedup over direct PyTorch multimodal inference.
  arXiv  Detail & Related papers  (2025-11-17T07:27:52Z)
• EgoEMS: A High-Fidelity Multimodal Egocentric Dataset for Cognitive Assistance in Emergency Medical Services [3.0776354206437664]
  EgoEMS is the first end-to-end, high-fidelity, multimodal, multiperson dataset capturing over 20 hours of realistic, procedural EMS activities.<n>Developed in collaboration with EMS experts and aligned with national standards, EgoEMS is captured using an open-source, low-cost, and replicable data collection system.<n>We present a suite of benchmarks for real-time multimodal keystep recognition and action quality estimation, essential for developing AI support tools for EMS.
  arXiv  Detail & Related papers  (2025-11-13T02:55:40Z)
• Complementary and Contrastive Learning for Audio-Visual Segmentation [74.11434759171199]
  We present Complementary and Contrastive Transformer (CCFormer), a novel framework adept at processing both local and global information.<n>Our method sets new state-of-the-art benchmarks across the S4, MS3 and AVSS datasets.
  arXiv  Detail & Related papers  (2025-10-11T06:36:59Z)
• Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models [78.32948112203228]
  Video understanding represents the most challenging frontier in computer vision.<n>Recent emergence of Video-Large Multitemporal Models has demonstrated remarkable capabilities in video understanding tasks.<n>Survey aims to provide researchers and practitioners with a unified framework for advancing Video-LMM capabilities.
  arXiv  Detail & Related papers  (2025-10-06T17:10:44Z)
• AssistPDA: An Online Video Surveillance Assistant for Video Anomaly Prediction, Detection, and Analysis [52.261173507177396]
  We introduce AssistPDA, the first online video anomaly surveillance assistant (VAPDA) that unifies anomaly prediction, detection, and analysis (VAPDA) within a single framework.<n> AssistPDA enables real-time inference on streaming videos while supporting interactive user engagement.<n>We also introduce a novel event-level anomaly prediction task, enabling proactive anomaly forecasting before anomalies fully unfold.
  arXiv  Detail & Related papers  (2025-03-27T18:30:47Z)
• Towards Intelligent Transportation with Pedestrians and Vehicles In-the-Loop: A Surveillance Video-Assisted Federated Digital Twin Framework [62.47416496137193]
  We propose a surveillance video assisted federated digital twin (SV-FDT) framework to empower ITSs with pedestrians and vehicles in-the-loop.<n>The architecture consists of three layers: (i) the end layer, which collects traffic surveillance videos from multiple sources; (ii) the edge layer, responsible for semantic segmentation-based visual understanding, twin agent-based interaction modeling, and local digital twin system (LDTS) creation in local regions; and (iii) the cloud layer, which integrates LDTSs across different regions to construct a global DT model in realtime.
  arXiv  Detail & Related papers  (2025-03-06T07:36:06Z)
• VIMI: Grounding Video Generation through Multi-modal Instruction [89.90065445082442]
  Existing text-to-video diffusion models rely solely on text-only encoders for their pretraining. We construct a large-scale multimodal prompt dataset by employing retrieval methods to pair in-context examples with the given text prompts. We finetune the model from the first stage on three video generation tasks, incorporating multi-modal instructions.
  arXiv  Detail & Related papers  (2024-07-08T18:12:49Z)
• Real-Time Multimodal Cognitive Assistant for Emergency Medical Services [4.669165383466683]
  This paper presents CognitiveEMS, an end-to-end wearable cognitive assistant system. It can act as a collaborative virtual partner engaging in the real-time acquisition and analysis of multimodal data from an emergency scene.
  arXiv  Detail & Related papers  (2024-03-11T13:56:57Z)
• IoMT-Blockchain based Secured Remote Patient Monitoring Framework for Neuro-Stimulation Device [0.0]
  Real-time sensory data from patients may be delivered and analyzed through rapid development of wearable IoMT devices. Data from the Internet of Things is gathered, analyzed, and stored in a single location. Due to its decentralized nature, blockchain (BC) can alleviate these issues.
  arXiv  Detail & Related papers  (2023-08-31T16:59:58Z)
• FENXI: Deep-learning Traffic Analytics at the Edge [69.34903175081284]
  We present FENXI, a system to run complex analytics by leveraging TPU. FENXI decouples operations and traffic analytics which operates at different granularities. Our analysis shows that FENXI can sustain forwarding line rate traffic processing requiring only limited resources.
  arXiv  Detail & Related papers  (2021-05-25T08:02:44Z)
• Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement [9.825675909430611]
  We present a video-based and on-device optical cardiopulmonary vital sign measurement approach. It enables real-time cardiovascular and respiratory measurements on mobile platforms. We evaluate our system on an Advanced RISC Machine (ARM) CPU and achieve state-of-the-art accuracy while running at over 150 frames per second.
  arXiv  Detail & Related papers  (2020-06-06T06:31:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.