REMONI: An Autonomous System Integrating Wearables and Multimodal Large Language Models for Enhanced Remote Health Monitoring

• URL: http://arxiv.org/abs/2510.21445v1
• Date: Fri, 24 Oct 2025 13:23:38 GMT
• Title: REMONI: An Autonomous System Integrating Wearables and Multimodal Large Language Models for Enhanced Remote Health Monitoring
• Authors: Thanh Cong Ho, Farah Kharrat, Abderrazek Abid, Fakhri Karray,
• Abstract summary: This paper proposes REMONI, an autonomous REmote health MONItoring system that integrates multimodal large language models (MLLMs), the Internet of Things (IoT), and wearable devices.<n>The system automatically and continuously collects vital signs, accelerometer data from a special wearable (such as a smartwatch), and visual data in patient video clips collected from cameras.<n>A distinctive feature of our proposed system is the natural language processing component, developed with MLLMs capable of detecting and recognizing a patient's activity and emotion while responding to healthcare worker's inquiries.
• Score: 4.85305127684971
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: With the widespread adoption of wearable devices in our daily lives, the demand and appeal for remote patient monitoring have significantly increased. Most research in this field has concentrated on collecting sensor data, visualizing it, and analyzing it to detect anomalies in specific diseases such as diabetes, heart disease and depression. However, this domain has a notable gap in the aspect of human-machine interaction. This paper proposes REMONI, an autonomous REmote health MONItoring system that integrates multimodal large language models (MLLMs), the Internet of Things (IoT), and wearable devices. The system automatically and continuously collects vital signs, accelerometer data from a special wearable (such as a smartwatch), and visual data in patient video clips collected from cameras. This data is processed by an anomaly detection module, which includes a fall detection model and algorithms to identify and alert caregivers of the patient's emergency conditions. A distinctive feature of our proposed system is the natural language processing component, developed with MLLMs capable of detecting and recognizing a patient's activity and emotion while responding to healthcare worker's inquiries. Additionally, prompt engineering is employed to integrate all patient information seamlessly. As a result, doctors and nurses can access real-time vital signs and the patient's current state and mood by interacting with an intelligent agent through a user-friendly web application. Our experiments demonstrate that our system is implementable and scalable for real-life scenarios, potentially reducing the workload of medical professionals and healthcare costs. A full-fledged prototype illustrating the functionalities of the system has been developed and being tested to demonstrate the robustness of its various capabilities.

Related papers

• AI on the Pulse: Real-Time Health Anomaly Detection with Wearable and Ambient Intelligence [4.494833548150712]
  We introduce AI on the Pulse, a real-world-ready anomaly detection system that continuously monitors patients.<n>Powered by UniTS, a state-of-the-art (SoTA) universal time-series model, our framework autonomously learns each patient's unique physiological and behavioral patterns.
  arXiv  Detail & Related papers  (2025-08-05T13:24:15Z)
• Advancing Conversational Diagnostic AI with Multimodal Reasoning [44.1996223689966]
  Articulate Medical Intelligence Explorer (AMIE)<n>System implements a state-aware dialogue framework, where conversation flow is dynamically controlled by intermediate model outputs.<n>We compared AMIE to primary care physicians (PCPs) in a randomized, blinded, OSCE-style study of chat-based consultations with patient actors.
  arXiv  Detail & Related papers  (2025-05-06T20:52:01Z)
• Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Survey [48.60253527030446]
  The integration of facial expression analysis with Internet-of-Thing (IoT) systems has significant potential across diverse scenarios.<n>Our work aims to provide a comprehensive overview of research progress in facial expression analysis and explores its potential integration with IoT systems.<n>We highlight challenges and future directions for the convergence of facial expression-based technologies and IoT systems.
  arXiv  Detail & Related papers  (2024-12-23T14:41:01Z)
• A Survey of Medical Vision-and-Language Applications and Their Techniques [48.268198631277315]
  Medical vision-and-language models (MVLMs) have attracted substantial interest due to their capability to offer a natural language interface for interpreting complex medical data. Here, we provide a comprehensive overview of MVLMs and the various medical tasks to which they have been applied. We also examine the datasets used for these tasks and compare the performance of different models based on standardized evaluation metrics.
  arXiv  Detail & Related papers  (2024-11-19T03:27:05Z)
• Towards Next-Generation Medical Agent: How o1 is Reshaping Decision-Making in Medical Scenarios [46.729092855387165]
  We study the choice of the backbone LLM for medical AI agents, which is the foundation for the agent's overall reasoning and action generation.<n>Our findings demonstrate o1's ability to enhance diagnostic accuracy and consistency, paving the way for smarter, more responsive AI tools.
  arXiv  Detail & Related papers  (2024-11-16T18:19:53Z)
• Scaling Wearable Foundation Models [54.93979158708164]
  We investigate the scaling properties of sensor foundation models across compute, data, and model size. Using a dataset of up to 40 million hours of in-situ heart rate, heart rate variability, electrodermal activity, accelerometer, skin temperature, and altimeter per-minute data from over 165,000 people, we create LSM. Our results establish the scaling laws of LSM for tasks such as imputation, extrapolation, both across time and sensor modalities.
  arXiv  Detail & Related papers  (2024-10-17T15:08:21Z)
• Advancing Medical Education through the cINnAMON Web Application [0.0]
  cINnAMON EUREKA Traditional project endeavours to revolutionize indoor lighting positioning and monitoring. Current variant of the intelligent bulb prototype offers a comparative analysis of the project's bulb against commercially available smart bulbs. Initial smart bracelet prototype showcases its ability to collect and analyse data from an array of built-in sensors.
  arXiv  Detail & Related papers  (2023-11-30T10:49:51Z)
• A Health Monitoring System Based on Flexible Triboelectric Sensors for Intelligence Medical Internet of Things and its Applications in Virtual Reality [4.522609963399036]
  The Internet of Medical Things (IoMT) is a platform that combines Internet of Things (IoT) technology with medical applications. In this study, we designed a robust and intelligent IoMT system through the synergistic integration of flexible wearable triboelectric sensors and deep learning-assisted data analytics. We embedded four triboelectric sensors into a wristband to detect and analyze limb movements in patients suffering from Parkinson's Disease (PD) This innovative approach enabled us to accurately capture and scrutinize the subtle movements and fine motor of PD patients, thus providing insightful feedback and comprehensive assessment of the patients conditions.
  arXiv  Detail & Related papers  (2023-09-13T01:01:16Z)
• MIMO: Mutual Integration of Patient Journey and Medical Ontology for Healthcare Representation Learning [49.57261599776167]
  We propose an end-to-end robust Transformer-based solution, Mutual Integration of patient journey and Medical Ontology (MIMO) for healthcare representation learning and predictive analytics.
  arXiv  Detail & Related papers  (2021-07-20T07:04:52Z)
• Detecting Anomalous User Behavior in Remote Patient Monitoring [0.26249027950824505]
  We present an anomaly detection model for Remote Patient Monitoring utilizing IoMT and smart home devices. We propose Hidden Markov Model (HMM) based anomaly detection that analyzes normal user behavior in the context of RPM. Proposed HMM based anomaly detection model achieved over 98% accuracy in identifying the anomalies in the context of RPM.
  arXiv  Detail & Related papers  (2021-06-22T14:59:34Z)
• An Overview of Human Activity Recognition Using Wearable Sensors: Healthcare and Artificial Intelligence [4.04762671215916]
  Human activity recognition (HAR) has been applied in a variety of domains such as security and surveillance, human-robot interaction, and entertainment. We present our emerging HAR projects for healthcare: identification of human activities for intensive care unit (ICU) patients and Duchenne muscular dystrophy (DMD) patients. Our HAR systems include hardware design to collect sensor data from ICU patients and DMD patients and accurate AI models to recognize patients' activities.
  arXiv  Detail & Related papers  (2021-03-29T23:48:51Z)
• Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery [84.73764603474413]
  We propose a novel online approach of multi-modal graph network (i.e., MRG-Net) to dynamically integrate visual and kinematics information. The effectiveness of our method is demonstrated with state-of-the-art results on the public JIGSAWS dataset.
  arXiv  Detail & Related papers  (2020-11-03T11:00:10Z)
• AEGIS: A real-time multimodal augmented reality computer vision based system to assist facial expression recognition for individuals with autism spectrum disorder [93.0013343535411]
  This paper presents the development of a multimodal augmented reality (AR) system which combines the use of computer vision and deep convolutional neural networks (CNN) The proposed system, which we call AEGIS, is an assistive technology deployable on a variety of user devices including tablets, smartphones, video conference systems, or smartglasses. We leverage both spatial and temporal information in order to provide an accurate expression prediction, which is then converted into its corresponding visualization and drawn on top of the original video frame.
  arXiv  Detail & Related papers  (2020-10-22T17:20:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.