Cross-Modal Video to Body-joints Augmentation for Rehabilitation Exercise Quality Assessment

• URL: http://arxiv.org/abs/2306.09546v1
• Date: Thu, 15 Jun 2023 23:23:35 GMT
• Title: Cross-Modal Video to Body-joints Augmentation for Rehabilitation Exercise Quality Assessment
• Authors: Ali Abedi, Mobin Malmirian, and Shehroz S. Khan
• Abstract summary: Exercise-based rehabilitation programs have been shown to enhance quality of life and reduce mortality and rehospitalizations. AI-driven virtual rehabilitation programs enable patients to complete exercises independently at home while AI algorithms can analyze exercise data to provide feedback to patients and report their progress to clinicians. This paper introduces a novel approach to assessing the quality of rehabilitation exercises using RGB video. Sequences of skeletal body joints are extracted from consecutive RGB video frames and analyzed by many-to-one sequential neural networks to evaluate exercise quality.
• Score: 3.544570529705401
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Exercise-based rehabilitation programs have been shown to enhance quality of life and reduce mortality and rehospitalizations. AI-driven virtual rehabilitation programs enable patients to complete exercises independently at home while AI algorithms can analyze exercise data to provide feedback to patients and report their progress to clinicians. This paper introduces a novel approach to assessing the quality of rehabilitation exercises using RGB video. Sequences of skeletal body joints are extracted from consecutive RGB video frames and analyzed by many-to-one sequential neural networks to evaluate exercise quality. Existing datasets for exercise rehabilitation lack adequate samples for training deep sequential neural networks to generalize effectively. A cross-modal data augmentation approach is proposed to resolve this problem. Visual augmentation techniques are applied to video data, and body joints extracted from the resulting augmented videos are used for training sequential neural networks. Extensive experiments conducted on the KInematic assessment of MOvement and clinical scores for remote monitoring of physical REhabilitation (KIMORE) dataset, demonstrate the superiority of the proposed method over previous baseline approaches. The ablation study highlights a significant enhancement in exercise quality assessment following cross-modal augmentation.

Related papers

• A Medical Low-Back Pain Physical Rehabilitation Dataset for Human Body Movement Analysis [0.6990493129893111]
  This article addresses four challenges to address and propose a medical dataset of clinical patients carrying out low back-pain rehabilitation exercises. The dataset includes 3D Kinect skeleton positions and orientations, RGB videos, 2D skeleton data, and medical annotations to assess the correctness, and error classification and localisation of body part and timespan.
  arXiv  Detail & Related papers  (2024-06-29T19:50:06Z)
• Rehabilitation Exercise Quality Assessment through Supervised Contrastive Learning with Hard and Soft Negatives [2.166000001057538]
  Exercise-based rehabilitation programs have proven to be effective in enhancing the quality of life and reducing mortality and rehospitalization rates. These programs commonly prescribe a variety of exercise types, leading to a distinct challenge in rehabilitation exercise assessment datasets. This paper introduces a novel supervised contrastive learning framework with hard and soft negative samples to train a single model applicable to all exercise types.
  arXiv  Detail & Related papers  (2024-03-05T08:38:25Z)
• D-STGCNT: A Dense Spatio-Temporal Graph Conv-GRU Network based on transformer for assessment of patient physical rehabilitation [0.3626013617212666]
  This paper introduces a new graph-based model for assessing rehabilitation exercises. Dense connections and GRU mechanisms are used to rapidly process large 3D skeleton inputs. The evaluation of our proposed approach on the KIMORE and UI-PRMD datasets highlighted its potential.
  arXiv  Detail & Related papers  (2023-12-21T00:38:31Z)
• Design, Development, and Evaluation of an Interactive Personalized Social Robot to Monitor and Coach Post-Stroke Rehabilitation Exercises [68.37238218842089]
  We develop an interactive social robot exercise coaching system for personalized rehabilitation. This system integrates a neural network model with a rule-based model to automatically monitor and assess patients' rehabilitation exercises. Our system can adapt to new participants and achieved 0.81 average performance to assess their exercises, which is comparable to the experts' agreement level.
  arXiv  Detail & Related papers  (2023-05-12T17:37:04Z)
• Rehabilitation Exercise Repetition Segmentation and Counting using Skeletal Body Joints [6.918076156491651]
  This paper presents a novel approach for segmenting and counting the repetitions of rehabilitation exercises performed by patients. Skeletal body joints can be acquired through depth cameras or computer vision techniques applied to RGB videos of patients. Various sequential neural networks are designed to analyze the sequences of skeletal body joints and perform repetition segmentation and counting.
  arXiv  Detail & Related papers  (2023-04-19T15:22:15Z)
• Mimetic Muscle Rehabilitation Analysis Using Clustering of Low Dimensional 3D Kinect Data [1.53119329713143]
  This paper discusses an unsupervised approach to rehabilitating patients who have temporary facial paralysis due to damage in mimetic muscles. The work aims to make the rehabilitation process objective compared to the current subjective approach, such as House-Brackmann (HB) scale. The study contains data set of 85 distinct patients with 120 measurements obtained using a Kinect stereo-vision camera.
  arXiv  Detail & Related papers  (2023-02-15T09:45:27Z)
• Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing [53.096237570992294]
  Strategy training is a rehabilitation approach that teaches skills to reduce disability among those with cognitive impairments following a stroke. Standardized fidelity assessment is used to measure adherence to treatment principles. We developed a rule-based NLP algorithm, a long-short term memory (LSTM) model, and a bidirectional encoder representation from transformers (BERT) model for this task.
  arXiv  Detail & Related papers  (2022-09-14T15:33:30Z)
• Neural Maximum A Posteriori Estimation on Unpaired Data for Motion Deblurring [87.97330195531029]
  We propose a Neural Maximum A Posteriori (NeurMAP) estimation framework for training neural networks to recover blind motion information and sharp content from unpaired data. The proposed NeurMAP is an approach to existing deblurring neural networks, and is the first framework that enables training image deblurring networks on unpaired datasets.
  arXiv  Detail & Related papers  (2022-04-26T08:09:47Z)
• On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy [70.71457102672545]
  We compare the impact of different training procedures for diabetic retinopathy grading. We investigate different aspects such as quantitative performance, statistics of the learned feature representations, interpretability and robustness to image distortions. Our results indicate that models from ImageNet pretraining report a significant increase in performance, generalization and robustness to image distortions.
  arXiv  Detail & Related papers  (2021-06-25T08:32:45Z)
• One-shot action recognition towards novel assistive therapies [63.23654147345168]
  This work is motivated by the automated analysis of medical therapies that involve action imitation games. The presented approach incorporates a pre-processing step that standardizes heterogeneous motion data conditions. We evaluate the approach on a real use-case of automated video analysis for therapy support with autistic people.
  arXiv  Detail & Related papers  (2021-02-17T19:41:37Z)
• A Review of Computational Approaches for Evaluation of Rehabilitation Exercises [58.720142291102135]
  This paper reviews computational approaches for evaluating patient performance in rehabilitation programs using motion capture systems. The reviewed computational methods for exercise evaluation are grouped into three main categories: discrete movement score, rule-based, and template-based approaches.
  arXiv  Detail & Related papers  (2020-02-29T22:18:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.