An AI-Driven Multimodal Smart Home Platform for Continuous Monitoring and Assistance in Post-Stroke Motor Impairment

• URL: http://arxiv.org/abs/2411.19000v4
• Date: Wed, 15 Oct 2025 21:33:15 GMT
• Title: An AI-Driven Multimodal Smart Home Platform for Continuous Monitoring and Assistance in Post-Stroke Motor Impairment
• Authors: Chenyu Tang, Ruizhi Zhang, Shuo Gao, Zihe Zhao, Zibo Zhang, Jiaqi Wang, Cong Li, Junliang Chen, Yanning Dai, Shengbo Wang, Ruoyu Juan, Qiaoying Li, Ruimou Xie, Xuhang Chen, Xinkai Zhou, Yunjia Xia, Jianan Chen, Fanghao Lu, Xin Li, Ninglli Wang, Peter Smielewski, Yu Pan, Hubin Zhao, Luigi G. Occhipinti,
• Abstract summary: We present a smart home platform designed for continuous, at-home rehabilitation of post-stroke patients.<n>A plantar pressure insole classifies users into motor recovery stages with up to 94% accuracy.<n>An optional head-mounted eye-tracking module, together with ambient sensors such as cameras and microphones, supports seamless hands-free control of household devices.
• Score: 23.865199757677036
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: At-home rehabilitation for post-stroke patients presents significant challenges, as continuous, personalized care is often limited outside clinical settings. Moreover, the lack of integrated solutions capable of simultaneously monitoring motor recovery and providing intelligent assistance in home environments hampers rehabilitation outcomes. Here, we present a multimodal smart home platform designed for continuous, at-home rehabilitation of post-stroke patients, integrating wearable sensing, ambient monitoring, and adaptive automation. A plantar pressure insole equipped with a machine learning pipeline classifies users into motor recovery stages with up to 94\% accuracy, enabling quantitative tracking of walking patterns during daily activities. An optional head-mounted eye-tracking module, together with ambient sensors such as cameras and microphones, supports seamless hands-free control of household devices with a 100\% success rate and sub-second response time. These data streams are fused locally via a hierarchical Internet of Things (IoT) architecture, ensuring low latency and data privacy. An embedded large language model (LLM) agent, Auto-Care, continuously interprets multimodal data to provide real-time interventions -- issuing personalized reminders, adjusting environmental conditions, and notifying caregivers. Implemented in a post-stroke context, this integrated smart home platform increased mean user satisfaction from 3.9 $\pm$ 0.8 in conventional home environments to 8.4 $\pm$ 0.6 with the full system ($n=20$). Beyond stroke, the system offers a scalable, patient-centered framework with potential for long-term use in broader neurorehabilitation and aging-in-place applications.

Related papers

• EgoPoseFormer v2: Accurate Egocentric Human Motion Estimation for AR/VR [43.739084350055435]
  We present a transformer-based model for temporally consistent and spatially grounded body pose estimation.<n>We also present an auto-labeling system that enables the use of large unlabeled datasets for training.<n>On the EgoBody3M benchmark, with a 0.8 ms latency on GPU, our model outperforms two state-of-the-art methods by 12.2% and 19.4% in accuracy, and reduces temporal jitter by 22.2% and 51.7%.
  arXiv  Detail & Related papers  (2026-03-04T14:01:16Z)
• SurgAtt-Tracker: Online Surgical Attention Tracking via Temporal Proposal Reranking and Motion-Aware Refinement [45.37105164372227]
  SurgAtt-Tracker is a holistic framework that robustly tracks surgical attention.<n>Experiments on multiple surgical datasets demonstrate that SurgAtt-Tracker achieves consistently state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-02-24T07:30:51Z)
• HERMES: A Unified Open-Source Framework for Realtime Multimodal Physiological Sensing, Edge AI, and Intervention in Closed-Loop Smart Healthcare Applications [0.0]
  HERMES is an open-source high-performance Python framework for continuous multimodal sensing and AI processing at the edge.<n>It enables synchronized data collection, and realtime streaming inference with user PyTorch models.<n> HERMES is applicable to fixed-lab and free-living environments, of distributed commercial and custom sensors.
  arXiv  Detail & Related papers  (2026-01-18T22:54:46Z)
• An AI-IoT Based Smart Wheelchair with Gesture-Controlled Mobility, Deep Learning-Based Obstacle Detection, Multi-Sensor Health Monitoring, and Emergency Alert System [0.6465251961564605]
  We propose a comprehensive AI-IoT based smart wheelchair system that incorporates glove-based gesture control for hands-free navigation, real-time object detection using YOLOv8 with auditory feedback for obstacle avoidance, and ultrasonic for immediate collision avoidance.<n>Built on a modular and low-cost architecture, the gesture control achieved a 95.5% success rate, ultrasonic obstacle detection reached 94% accuracy, and YOLOv8-based object detection delivered 91.5% Precision, 90.2% Recall, and a 90.8% F1-score.
  arXiv  Detail & Related papers  (2026-01-17T09:37:11Z)
• REMONI: An Autonomous System Integrating Wearables and Multimodal Large Language Models for Enhanced Remote Health Monitoring [4.85305127684971]
  This paper proposes REMONI, an autonomous REmote health MONItoring system that integrates multimodal large language models (MLLMs), the Internet of Things (IoT), and wearable devices.<n>The system automatically and continuously collects vital signs, accelerometer data from a special wearable (such as a smartwatch), and visual data in patient video clips collected from cameras.<n>A distinctive feature of our proposed system is the natural language processing component, developed with MLLMs capable of detecting and recognizing a patient's activity and emotion while responding to healthcare worker's inquiries.
  arXiv  Detail & Related papers  (2025-10-24T13:23:38Z)
• AI on the Pulse: Real-Time Health Anomaly Detection with Wearable and Ambient Intelligence [4.494833548150712]
  We introduce AI on the Pulse, a real-world-ready anomaly detection system that continuously monitors patients.<n>Powered by UniTS, a state-of-the-art (SoTA) universal time-series model, our framework autonomously learns each patient's unique physiological and behavioral patterns.
  arXiv  Detail & Related papers  (2025-08-05T13:24:15Z)
• MetAdv: A Unified and Interactive Adversarial Testing Platform for Autonomous Driving [85.04826012938642]
  MetAdv is a novel adversarial testing platform that enables realistic, dynamic, and interactive evaluation.<n>It supports flexible 3D vehicle modeling and seamless transitions between simulated and physical environments.<n>It enables real-time capture of physiological signals and behavioral feedback from drivers.
  arXiv  Detail & Related papers  (2025-08-04T03:07:54Z)
• Learning Multimodal AI Algorithms for Amplifying Limited User Input into High-dimensional Control Space [7.504214864070018]
  Current invasive assistive technologies are designed to infer high-dimensional motor control signals from severely paralyzed patients.<n> noninvasive alternatives often rely on artifact-prone signals, require lengthy user training, and struggle to deliver robust high-dimensional control for dexterous tasks.<n>This study introduces a novel human-centered multimodal AI approach as intelligent compensatory mechanisms for lost motor functions.
  arXiv  Detail & Related papers  (2025-05-16T15:31:40Z)
• MELON: Multimodal Mixture-of-Experts with Spectral-Temporal Fusion for Long-Term Mobility Estimation in Critical Care [1.5237145555729716]
  We introduce MELON, a novel framework designed to predict 12-hour mobility status in the critical care setting. We trained and evaluated the MELON model on the multimodal dataset of 126 patients recruited from nine Intensive Care Units at the University of Florida Health Shands Hospital main campus in Gainesville, Florida. Results showed that MELON outperforms conventional approaches for 12-hour mobility status estimation.
  arXiv  Detail & Related papers  (2025-03-10T19:47:46Z)
• Continuous Patient Monitoring with AI: Real-Time Analysis of Video in Hospital Care Settings [0.0]
  This study introduces an AI-driven platform for continuous and passive patient monitoring in hospital settings. The platform provides real-time insights into patient behavior and interactions through video analysis. The dataset, compiled in collaboration with 11 hospital partners, encompasses over 300 high-risk fall patients.
  arXiv  Detail & Related papers  (2024-12-17T18:23:33Z)
• OmniBuds: A Sensory Earable Platform for Advanced Bio-Sensing and On-Device Machine Learning [46.3331254985615]
  Sensory earables have evolved from basic audio enhancement devices into sophisticated platforms for clinical-grade health monitoring and wellbeing management. This paper introduces OmniBuds, an advanced sensory earable platform integrating multiple biosensors and onboard computation powered by a machine learning accelerator.
  arXiv  Detail & Related papers  (2024-10-07T06:30:59Z)
• Socially Interactive Agents for Robotic Neurorehabilitation Training: Conceptualization and Proof-of-concept Study [7.365940126473552]
  We introduce an AI-based system aimed at delivering personalized, out-of-hospital assistance during neurorehabilitation training. With the assistance of a professional, the envisioned system is designed to accommodate the unique rehabilitation requirements of an individual patient. Our approach involves the integration of an interactive socially-aware virtual agent into a neurorehabilitation robotic framework.
  arXiv  Detail & Related papers  (2024-06-17T19:07:05Z)
• L-SFAN: Lightweight Spatially-focused Attention Network for Pain Behavior Detection [44.016805074560295]
  Chronic Low Back Pain (CLBP) afflicts millions globally, significantly impacting individuals' well-being and imposing economic burdens on healthcare systems. While artificial intelligence (AI) and deep learning offer promising avenues for analyzing pain-related behaviors to improve rehabilitation strategies, current models, including convolutional neural networks (CNNs), have limitations. We introduce hbox EmoL-SFAN, a lightweight CNN architecture incorporating 2D filters designed to capture the spatial-temporal interplay of data from motion capture and surface electromyography sensors.
  arXiv  Detail & Related papers  (2024-06-07T12:01:37Z)
• Towards Privacy-Aware and Personalised Assistive Robots: A User-Centred Approach [55.5769013369398]
  This research pioneers user-centric, privacy-aware technologies such as Federated Learning (FL) FL enables collaborative learning without sharing sensitive data, addressing privacy and scalability issues. This work includes developing solutions for smart wheelchair assistance, enhancing user independence and well-being.
  arXiv  Detail & Related papers  (2024-05-23T13:14:08Z)
• A Telerehabilitation System for the Selection, Evaluation and Remote Management of Therapies [0.044998333629984864]
  The main contribution of this paper is to present, as a whole, all the features supported by the innovative Kinect-based Telerehabilitation System (KiReS) The knowledge extraction functionality handles knowledge about the physical therapy record of patients and treatment protocols. The teleimmersion functionality provides a convenient, effective and user-friendly experience when performing the telerehabilitation.
  arXiv  Detail & Related papers  (2024-01-16T08:35:36Z)
• Clairvoyance: A Pipeline Toolkit for Medical Time Series [95.22483029602921]
  Time-series learning is the bread and butter of data-driven *clinical decision support* Clairvoyance proposes a unified, end-to-end, autoML-friendly pipeline that serves as a software toolkit. Clairvoyance is the first to demonstrate viability of a comprehensive and automatable pipeline for clinical time-series ML.
  arXiv  Detail & Related papers  (2023-10-28T12:08:03Z)
• A Health Monitoring System Based on Flexible Triboelectric Sensors for Intelligence Medical Internet of Things and its Applications in Virtual Reality [4.522609963399036]
  The Internet of Medical Things (IoMT) is a platform that combines Internet of Things (IoT) technology with medical applications. In this study, we designed a robust and intelligent IoMT system through the synergistic integration of flexible wearable triboelectric sensors and deep learning-assisted data analytics. We embedded four triboelectric sensors into a wristband to detect and analyze limb movements in patients suffering from Parkinson's Disease (PD) This innovative approach enabled us to accurately capture and scrutinize the subtle movements and fine motor of PD patients, thus providing insightful feedback and comprehensive assessment of the patients conditions.
  arXiv  Detail & Related papers  (2023-09-13T01:01:16Z)
• Design, Development, and Evaluation of an Interactive Personalized Social Robot to Monitor and Coach Post-Stroke Rehabilitation Exercises [68.37238218842089]
  We develop an interactive social robot exercise coaching system for personalized rehabilitation. This system integrates a neural network model with a rule-based model to automatically monitor and assess patients' rehabilitation exercises. Our system can adapt to new participants and achieved 0.81 average performance to assess their exercises, which is comparable to the experts' agreement level.
  arXiv  Detail & Related papers  (2023-05-12T17:37:04Z)
• AI-Enhanced Intensive Care Unit: Revolutionizing Patient Care with Pervasive Sensing [2.7503982558916906]
  The intensive care unit (ICU) is a specialized hospital space where critically ill patients receive intensive care and monitoring. Comprehensive monitoring is imperative in assessing patients conditions, in particular acuity, and ultimately the quality of care. Currently, visual assessments for acuity, including fine details such as facial expressions, posture, and mobility, are sporadically captured, or not captured at all.
  arXiv  Detail & Related papers  (2023-03-11T00:25:55Z)
• Remote patient monitoring using artificial intelligence: Current state, applications, and challenges [13.516357215412024]
  This study aims to do a comprehensive review of RPM systems including adopted advanced technologies, AI impact on RPM, challenges and trends in AI-enabled RPM. The role of AI in RPM ranges from physical activity classification to chronic disease monitoring and vital signs monitoring in emergency settings. This review results show that AI-enabled RPM architectures have transformed healthcare monitoring applications.
  arXiv  Detail & Related papers  (2023-01-19T06:22:14Z)
• PulseImpute: A Novel Benchmark Task for Pulsative Physiological Signal Imputation [54.839600943189915]
  Mobile Health (mHealth) is the ability to use wearable sensors to monitor participant physiology at high frequencies during daily life to enable temporally-precise health interventions. Despite a rich imputation literature, existing techniques are ineffective for the pulsative signals which comprise many mHealth applications. We address this gap with PulseImpute, the first large-scale pulsative signal imputation challenge which includes realistic mHealth missingness models, an extensive set of baselines, and clinically-relevant downstream tasks.
  arXiv  Detail & Related papers  (2022-12-14T21:39:15Z)
• Reducing a complex two-sided smartwatch examination for Parkinson's Disease to an efficient one-sided examination preserving machine learning accuracy [63.20765930558542]
  We have recorded participants performing technology-based assessments in a prospective study to research Parkinson's Disease (PD) This study provided the largest PD sample size of two-hand synchronous smartwatch measurements.
  arXiv  Detail & Related papers  (2022-05-11T09:12:59Z)
• An adaptive closed-loop ECoG decoder for long-term and stable bimanual control of an exoskeleton by a tetraplegic [91.6474995587871]
  High performance control of diverse effectors for complex tasks must be robust over time and of high decoding performance without continuous recalibration of the decoders. We developed an adaptive online tensor-based decoder: the Recursive Exponentially Weighted Markov-Switching multi- Linear Model (REW-MSLM) We demonstrated over a period of 6 months the stability of the 8-dimensional alternative bimanual control of the exoskeleton and its virtual avatar using REW-MSLM without recalibration of the decoder.
  arXiv  Detail & Related papers  (2022-01-25T16:51:29Z)
• Adherence Forecasting for Guided Internet-Delivered Cognitive Behavioral Therapy: A Minimally Data-Sensitive Approach [59.535699822923]
  Internet-delivered psychological treatments (IDPT) are seen as an effective and scalable pathway to improving the accessibility of mental healthcare. This work proposes a deep-learning approach to perform automatic adherence forecasting, while relying on minimally sensitive login/logout data. The proposed Self-Attention Network achieved over 70% average balanced accuracy, when only 1/3 of the treatment duration had elapsed.
  arXiv  Detail & Related papers  (2022-01-11T13:55:57Z)
• Optimal discharge of patients from intensive care via a data-driven policy learning framework [58.720142291102135]
  It is important that the patient discharge task addresses the nuanced trade-off between decreasing a patient's length of stay and the risk of readmission or even death following the discharge decision. This work introduces an end-to-end general framework for capturing this trade-off to recommend optimal discharge timing decisions. A data-driven approach is used to derive a parsimonious, discrete state space representation that captures a patient's physiological condition.
  arXiv  Detail & Related papers  (2021-12-17T04:39:33Z)
• Certainty Modeling of a Decision Support System for Mobile Monitoring of Exercise induced Respiratory Conditions [0.0]
  The aim is to develop a mobile tool to assist patients in managing their conditions. We present the proposed model architecture and then describe an application scenario in a clinical setting.
  arXiv  Detail & Related papers  (2021-10-15T07:26:36Z)
• Personalized Rehabilitation Robotics based on Online Learning Control [62.6606062732021]
  We propose a novel online learning control architecture, which is able to personalize the control force at run time to each individual user. We evaluate our method in an experimental user study, where the learning controller is shown to provide personalized control, while also obtaining safe interaction forces.
  arXiv  Detail & Related papers  (2021-10-01T15:28:44Z)
• Early Mobility Recognition for Intensive Care Unit Patients Using Accelerometers [3.772793938066986]
  We propose a new healthcare application of human activity recognition, early mobility recognition for Intensive Care Unit(ICU) patients. Our system includes accelerometer-based data collection from ICU patients and an AI model to recognize patients' early mobility. Our results show that our system improves model accuracy from 77.78% to 81.86% and reduces the model instability (standard deviation) from 16.69% to 6.92%.
  arXiv  Detail & Related papers  (2021-06-28T22:59:31Z)
• Wearable Health Monitoring System for Older Adults in a Smart Home Environment [1.14219428942199]
  We present the design of a wearable health monitoring system suitable for older adults in a smart home context. The proposed system offers solutions to monitor the stress, blood pressure, and location of an individual within a smart home environment. A voice-based prototype is also implemented and the feasibility of the proposed system for integration in a smart home environment is analyzed.
  arXiv  Detail & Related papers  (2021-06-09T03:16:54Z)
• The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulation [81.72197368690031]
  We present a new benchmarking suite designed specifically for medical sequential decision making. The Medkit-Learn(ing) Environment is a publicly available Python package providing simple and easy access to high-fidelity synthetic medical data.
  arXiv  Detail & Related papers  (2021-06-08T10:38:09Z)
• AEGIS: A real-time multimodal augmented reality computer vision based system to assist facial expression recognition for individuals with autism spectrum disorder [93.0013343535411]
  This paper presents the development of a multimodal augmented reality (AR) system which combines the use of computer vision and deep convolutional neural networks (CNN) The proposed system, which we call AEGIS, is an assistive technology deployable on a variety of user devices including tablets, smartphones, video conference systems, or smartglasses. We leverage both spatial and temporal information in order to provide an accurate expression prediction, which is then converted into its corresponding visualization and drawn on top of the original video frame.
  arXiv  Detail & Related papers  (2020-10-22T17:20:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.