Detecting Activities of Daily Living in Egocentric Video to Contextualize Hand Use at Home in Outpatient Neurorehabilitation Settings

• URL: http://arxiv.org/abs/2412.10846v1
• Date: Sat, 14 Dec 2024 14:38:27 GMT
• Title: Detecting Activities of Daily Living in Egocentric Video to Contextualize Hand Use at Home in Outpatient Neurorehabilitation Settings
• Authors: Adesh Kadambi, José Zariffa,
• Abstract summary: We show that an object-centric approach, focusing on what objects patients interact with rather than how they move, can effectively recognize Activities of Daily Living (ADL) in real-world rehabilitation settings. We evaluated our models on a complex dataset collected in the wild comprising 2261 minutes of egocentric video from 16 participants with impaired hand function.
• Score: 2.9158689853305693
• License:
• Abstract: Wearable egocentric cameras and machine learning have the potential to provide clinicians with a more nuanced understanding of patient hand use at home after stroke and spinal cord injury (SCI). However, they require detailed contextual information (i.e., activities and object interactions) to effectively interpret metrics and meaningfully guide therapy planning. We demonstrate that an object-centric approach, focusing on what objects patients interact with rather than how they move, can effectively recognize Activities of Daily Living (ADL) in real-world rehabilitation settings. We evaluated our models on a complex dataset collected in the wild comprising 2261 minutes of egocentric video from 16 participants with impaired hand function. By leveraging pre-trained object detection and hand-object interaction models, our system achieves robust performance across different impairment levels and environments, with our best model achieving a mean weighted F1-score of 0.78 +/- 0.12 and maintaining an F1-score > 0.5 for all participants using leave-one-subject-out cross validation. Through qualitative analysis, we observe that this approach generates clinically interpretable information about functional object use while being robust to patient-specific movement variations, making it particularly suitable for rehabilitation contexts with prevalent upper limb impairment.

Related papers

• Automating Feedback Analysis in Surgical Training: Detection, Categorization, and Assessment [65.70317151363204]
  This work introduces the first framework for reconstructing surgical dialogue from unstructured real-world recordings. In surgical training, the formative verbal feedback that trainers provide to trainees during live surgeries is crucial for ensuring safety, correcting behavior immediately, and facilitating long-term skill acquisition. Our framework integrates voice activity detection, speaker diarization, and automated speech recaognition, with a novel enhancement that removes hallucinations.
  arXiv  Detail & Related papers  (2024-12-01T10:35:12Z)
• Visual-Geometric Collaborative Guidance for Affordance Learning [63.038406948791454]
  We propose a visual-geometric collaborative guided affordance learning network that incorporates visual and geometric cues. Our method outperforms the representative models regarding objective metrics and visual quality.
  arXiv  Detail & Related papers  (2024-10-15T07:35:51Z)
• A Medical Low-Back Pain Physical Rehabilitation Dataset for Human Body Movement Analysis [0.6990493129893111]
  This article addresses four challenges to address and propose a medical dataset of clinical patients carrying out low back-pain rehabilitation exercises. The dataset includes 3D Kinect skeleton positions and orientations, RGB videos, 2D skeleton data, and medical annotations to assess the correctness, and error classification and localisation of body part and timespan.
  arXiv  Detail & Related papers  (2024-06-29T19:50:06Z)
• Assessment and treatment of visuospatial neglect using active learning with Gaussian processes regression [0.3262230127283452]
  Visuospatial neglect is a disorder characterised by impaired awareness for visual stimuli located in regions of space and frames of reference. We present an artificial intelligence solution designed to accurately assess a patient's visuospatial neglect in a three-dimensional setting.
  arXiv  Detail & Related papers  (2023-09-29T09:18:32Z)
• Design, Development, and Evaluation of an Interactive Personalized Social Robot to Monitor and Coach Post-Stroke Rehabilitation Exercises [68.37238218842089]
  We develop an interactive social robot exercise coaching system for personalized rehabilitation. This system integrates a neural network model with a rule-based model to automatically monitor and assess patients' rehabilitation exercises. Our system can adapt to new participants and achieved 0.81 average performance to assess their exercises, which is comparable to the experts' agreement level.
  arXiv  Detail & Related papers  (2023-05-12T17:37:04Z)
• Mimetic Muscle Rehabilitation Analysis Using Clustering of Low Dimensional 3D Kinect Data [1.53119329713143]
  This paper discusses an unsupervised approach to rehabilitating patients who have temporary facial paralysis due to damage in mimetic muscles. The work aims to make the rehabilitation process objective compared to the current subjective approach, such as House-Brackmann (HB) scale. The study contains data set of 85 distinct patients with 120 measurements obtained using a Kinect stereo-vision camera.
  arXiv  Detail & Related papers  (2023-02-15T09:45:27Z)
• Easing Automatic Neurorehabilitation via Classification and Smoothness Analysis [1.44744639843118]
  We propose an automatic assessment pipeline that starts by recognizing patients' movements by means of a shallow deep learning architecture, then measuring the movement quality using jerk measure and related measures. A particularity of this work is that the dataset used is clinically relevant, since it represents movements inspired from Fugl-Meyer a well common upper-limb clinical stroke assessment scale for stroke patients. We show that it is possible to detect the contrast between healthy and patients movements in terms of smoothness, besides achieving conclusions about the patients' progress during the rehabilitation sessions that correspond to the clinicians' findings about each case.
  arXiv  Detail & Related papers  (2022-12-09T13:59:14Z)
• Measuring hand use in the home after cervical spinal cord injury using egocentric video [2.1064122195521926]
  Egocentric video has emerged as a potential solution for monitoring hand function in individuals living with tetraplegia in the community. We develop and validate a wearable vision-based system for measuring hand use in the home among individuals living with tetraplegia.
  arXiv  Detail & Related papers  (2022-03-31T12:43:23Z)
• Skeleton-Based Mutually Assisted Interacted Object Localization and Human Action Recognition [111.87412719773889]
  We propose a joint learning framework for "interacted object localization" and "human action recognition" based on skeleton data. Our method achieves the best or competitive performance with the state-of-the-art methods for human action recognition.
  arXiv  Detail & Related papers  (2021-10-28T10:09:34Z)
• One-shot action recognition towards novel assistive therapies [63.23654147345168]
  This work is motivated by the automated analysis of medical therapies that involve action imitation games. The presented approach incorporates a pre-processing step that standardizes heterogeneous motion data conditions. We evaluate the approach on a real use-case of automated video analysis for therapy support with autistic people.
  arXiv  Detail & Related papers  (2021-02-17T19:41:37Z)
• A Review of Computational Approaches for Evaluation of Rehabilitation Exercises [58.720142291102135]
  This paper reviews computational approaches for evaluating patient performance in rehabilitation programs using motion capture systems. The reviewed computational methods for exercise evaluation are grouped into three main categories: discrete movement score, rule-based, and template-based approaches.
  arXiv  Detail & Related papers  (2020-02-29T22:18:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.