HERMES: A Unified Open-Source Framework for Realtime Multimodal Physiological Sensing, Edge AI, and Intervention in Closed-Loop Smart Healthcare Applications

• URL: http://arxiv.org/abs/2601.12610v1
• Date: Sun, 18 Jan 2026 22:54:46 GMT
• Title: HERMES: A Unified Open-Source Framework for Realtime Multimodal Physiological Sensing, Edge AI, and Intervention in Closed-Loop Smart Healthcare Applications
• Abstract summary: HERMES is an open-source high-performance Python framework for continuous multimodal sensing and AI processing at the edge.<n>It enables synchronized data collection, and realtime streaming inference with user PyTorch models.<n> HERMES is applicable to fixed-lab and free-living environments, of distributed commercial and custom sensors.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Intelligent assistive technologies are increasingly recognized as critical daily-use enablers for people with disabilities and age-related functional decline. Longitudinal studies, curation of quality datasets, live monitoring in activities of daily living, and intelligent intervention devices, share the largely unsolved need in reliable high-throughput multimodal sensing and processing. Streaming large heterogeneous data from distributed sensors, historically closed-source environments, and limited prior works on realtime closed-loop AI methodologies, inhibit such applications. To accelerate the emergence of clinical deployments, we deliver HERMES - an open-source high-performance Python framework for continuous multimodal sensing and AI processing at the edge. It enables synchronized data collection, and realtime streaming inference with user PyTorch models, on commodity computing devices. HERMES is applicable to fixed-lab and free-living environments, of distributed commercial and custom sensors. It is the first work to offer a holistic methodology that bridges cross-disciplinary gaps in real-world implementation strategies, and guides downstream AI model development. Its application on the closed-loop intelligent prosthesis use case illustrates the process of suitable AI model development from the generated constraints and trade-offs. Validation on the use case, with 4 synchronized hosts cooperatively capturing 18 wearable and off-body modalities, demonstrates performance and relevance of HERMES to the trajectory of the intelligent healthcare domain.

Related papers

• Agentic AI-driven Immersive Simulation: A Knowledge-Aware Virtual Training Platform forHigh Dose Rate (HDR) Brachytherapy [3.176435384389084]
  This paper presents a novel agentic AI-driven immersive simulation architecture specifically designed for High Dose Rate (cele) vaginal cylinder (VC) brachytherapy in cancer care.<n>By integrating Virtual Reality (VR) and mobile computing, the system establishes a high-fidelity, risk-free environment.<n>A core contribution of this work is the seamless integration of a knowledge-aware assistant leveraging Retrieval-Augmented Generation (RAG) to ground agent interactions in authoritative clinical guidelines.
  arXiv  Detail & Related papers  (2026-08-08T14:46:08Z)
• dashi: A Python library for Dataset Shift Characterization to Support Trustworthy AI Development and Deployment [0.4982315103487453]
  We introduce dashi, an open-source Python library designed for the exploration, quantification, and characterization of dataset shifts.<n>We demonstrate the utility of dashi on three simulated and real-world health AI case studies.
  arXiv  Detail & Related papers  (2026-05-29T14:33:45Z)
• Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaboration [53.772014300855375]
  We introduce NIAgent, a multi-agent system for autonomous end-to-end neuroimaging analysis.<n>Unlike conventional flat tool-calling agents, NIAgent adopts a code-centric execution paradigm.<n>We propose a hierarchical verification framework for autonomous quality control, integrating cohort-level metric screening with agentic visual inspection.
  arXiv  Detail & Related papers  (2026-05-10T06:30:19Z)
• NeuroClaw Technical Report [54.14941057366186]
  NeuroClaw is a domain-specialized multi-agent research assistant for neuroimaging research.<n>NeuroClaw operates directly on raw neuroimaging data across formats and modalities.<n>A three-tier skill/agent hierarchy separates user-facing interaction, high-level orchestration, and low-level tool skills.
  arXiv  Detail & Related papers  (2026-04-27T16:57:14Z)
• Design and Validation of a Lightweight 1D CNN for Affective Touch Classification in Soft Plush Companions [59.49967357689445]
  Soft, sensorized companions offer a physically safe and emotionally intuitive interface for socially assistive technologies.<n>This study presents a complete open-source dilated real-based framework for the development and validation of deep learning models for affective touch recognition in soft interactive companions.
  arXiv  Detail & Related papers  (2026-04-16T10:20:05Z)
• A Platform-Agnostic Multimodal Digital Human Modelling Framework: Neurophysiological Sensing in Game-Based Interaction [2.835421685630534]
  This paper presents a platform-agnostic DHM framework to support AI-ready multimodal interaction research.<n>The framework integrates the OpenBCI Galea headset as a unified multimodal sensing layer.<n>Rather than embedding AI models or behavioural inference, physiological signals are represented as structured, temporally aligned observables.
  arXiv  Detail & Related papers  (2026-03-11T11:45:17Z)
• SelfAI: Building a Self-Training AI System with LLM Agents [79.10991818561907]
  SelfAI is a general multi-agent platform that combines a User Agent for translating high-level research objectives into standardized experimental configurations.<n>An Experiment Manager orchestrates parallel, fault-tolerant training across heterogeneous hardware while maintaining a structured knowledge base for continuous feedback.<n>Across regression, computer vision, scientific computing, medical imaging, and drug discovery benchmarks, SelfAI consistently achieves strong performance and reduces redundant trials.
  arXiv  Detail & Related papers  (2025-11-29T09:18:39Z)
• Metrics and evaluations for computational and sustainable AI efficiency [26.52588349722099]
  Current approaches fail to provide a holistic view, making it difficult to compare and optimise systems.<n>We propose a unified and reproducible methodology for AI model inference that integrates computational and environmental metrics.<n>Our framework provides pragmatic, carbon-aware evaluation by systematically measuring latency and distributions throughput, energy consumption, and location-adjusted carbon emissions.
  arXiv  Detail & Related papers  (2025-10-18T03:30:15Z)
• Adaptive and Resource-efficient Agentic AI Systems for Mobile and Embedded Devices: A Survey [11.537225726120495]
  Foundation models have reshaped AI by unifying fragmented architectures into scalable backbones with multimodal reasoning and contextual adaptation.<n>With FMs as their cognitive core, agents transcend rule-based behaviors to achieve autonomy, generalization, and self-reflection.<n>This survey provides the first systematic characterization of adaptive, resource-efficient agentic AI systems.
  arXiv  Detail & Related papers  (2025-09-30T02:37:52Z)
• impuTMAE: Multi-modal Transformer with Masked Pre-training for Missing Modalities Imputation in Cancer Survival Prediction [75.43342771863837]
  We introduce impuTMAE, a novel transformer-based end-to-end approach with an efficient multimodal pre-training strategy.<n>It learns inter- and intra-modal interactions while simultaneously imputing missing modalities by reconstructing masked patches.<n>Our model is pre-trained on heterogeneous, incomplete data and fine-tuned for glioma survival prediction using TCGA-GBM/LGG and BraTS datasets.
  arXiv  Detail & Related papers  (2025-08-08T10:01:16Z)
• AI/ML Life Cycle Management for Interoperable AI Native RAN [50.61227317567369]
  Artificial intelligence (AI) and machine learning (ML) models are rapidly permeating the 5G Radio Access Network (RAN)<n>These developments lay the foundation for AI-native transceivers as a key enabler for 6G.
  arXiv  Detail & Related papers  (2025-07-24T16:04:59Z)
• Learning Multimodal AI Algorithms for Amplifying Limited User Input into High-dimensional Control Space [7.504214864070018]
  Current invasive assistive technologies are designed to infer high-dimensional motor control signals from severely paralyzed patients.<n> noninvasive alternatives often rely on artifact-prone signals, require lengthy user training, and struggle to deliver robust high-dimensional control for dexterous tasks.<n>This study introduces a novel human-centered multimodal AI approach as intelligent compensatory mechanisms for lost motor functions.
  arXiv  Detail & Related papers  (2025-05-16T15:31:40Z)
• KAITIAN: A Unified Communication Framework for Enabling Efficient Collaboration Across Heterogeneous Accelerators in Embodied AI Systems [5.241889216655924]
  KAITIAN is a novel distributed communication framework for AI workloads.<n>It integrates vendor-optimized communication libraries for intra-group efficiency with general-purpose communication protocols for inter-group interoperability.<n>It can accelerate training time by up to 42% compared to baseline homogeneous systems.
  arXiv  Detail & Related papers  (2025-05-15T11:29:43Z)
• Edge-Cloud Collaborative Computing on Distributed Intelligence and Model Optimization: A Survey [58.50944604905037]
  Edge-cloud collaborative computing (ECCC) has emerged as a pivotal paradigm for addressing the computational demands of modern intelligent applications.<n>Recent advancements in AI, particularly deep learning and large language models (LLMs), have dramatically enhanced the capabilities of these distributed systems.<n>This survey provides a structured tutorial on fundamental architectures, enabling technologies, and emerging applications.
  arXiv  Detail & Related papers  (2025-05-03T13:55:38Z)
• An AI-Driven Multimodal Smart Home Platform for Continuous Monitoring and Assistance in Post-Stroke Motor Impairment [23.865199757677036]
  We present a smart home platform designed for continuous, at-home rehabilitation of post-stroke patients.<n>A plantar pressure insole classifies users into motor recovery stages with up to 94% accuracy.<n>An optional head-mounted eye-tracking module, together with ambient sensors such as cameras and microphones, supports seamless hands-free control of household devices.
  arXiv  Detail & Related papers  (2024-11-28T09:04:39Z)
• SEDMamba: Enhancing Selective State Space Modelling with Bottleneck Mechanism and Fine-to-Coarse Temporal Fusion for Efficient Error Detection in Robot-Assisted Surgery [7.863539113283565]
  We propose a novel hierarchical model named SEDMamba, which incorporates the selective state space model (SSM) into surgical error detection.<n> SEDMamba enhances selective SSM with a bottleneck mechanism and fine-to-coarse temporal fusion (FCTF) to detect and temporally localize surgical errors in long videos.<n>Our work also contributes the first-of-its-kind, frame-level, in-vivo surgical error dataset to support error detection in real surgical cases.
  arXiv  Detail & Related papers  (2024-06-22T19:20:35Z)
• OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System [85.8338446357469]
  We introduce OmniForce, a human-centered AutoML system that yields both human-assisted ML and ML-assisted human techniques. We show how OmniForce can put an AutoML system into practice and build adaptive AI in open-environment scenarios.
  arXiv  Detail & Related papers  (2023-03-01T13:35:22Z)
• Towards AIOps in Edge Computing Environments [60.27785717687999]
  This paper describes the system design of an AIOps platform which is applicable in heterogeneous, distributed environments. It is feasible to collect metrics with a high frequency and simultaneously run specific anomaly detection algorithms directly on edge devices.
  arXiv  Detail & Related papers  (2021-02-12T09:33:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.