VidSole: A Multimodal Dataset for Joint Kinetics Quantification and Disease Detection with Deep Learning

• URL: http://arxiv.org/abs/2501.17890v1
• Date: Tue, 28 Jan 2025 15:18:00 GMT
• Title: VidSole: A Multimodal Dataset for Joint Kinetics Quantification and Disease Detection with Deep Learning
• Authors: Archit Kambhamettu, Samantha Snyder, Maliheh Fakhar, Samuel Audia, Ross Miller, Jae Kun Shim, Aniket Bera,
• Abstract summary: Internal joint loading is critical for diagnosing gait-related diseases such as knee osteoarthritis. Our novel instrumented insole measures the tri-axial forces and moments across five high-pressure points under the foot. VidSole consists of the forces and moments measured by these insoles along with corresponding RGB video from two viewpoints, 3D body motion capture, and force plate data.
• Score: 10.033899828022525
• License:
• Abstract: Understanding internal joint loading is critical for diagnosing gait-related diseases such as knee osteoarthritis; however, current methods of measuring joint risk factors are time-consuming, expensive, and restricted to lab settings. In this paper, we enable the large-scale, cost-effective biomechanical analysis of joint loading via three key contributions: the development and deployment of novel instrumented insoles, the creation of a large multimodal biomechanics dataset (VidSole), and a baseline deep learning pipeline to predict internal joint loading factors. Our novel instrumented insole measures the tri-axial forces and moments across five high-pressure points under the foot. VidSole consists of the forces and moments measured by these insoles along with corresponding RGB video from two viewpoints, 3D body motion capture, and force plate data for over 2,600 trials of 52 diverse participants performing four fundamental activities of daily living (sit-to-stand, stand-to-sit, walking, and running). We feed the insole data and kinematic parameters extractable from video (i.e., pose, knee angle) into a deep learning pipeline consisting of an ensemble Gated Recurrent Unit (GRU) activity classifier followed by activity-specific Long Short Term Memory (LSTM) regression networks to estimate knee adduction moment (KAM), a biomechanical risk factor for knee osteoarthritis. The successful classification of activities at an accuracy of 99.02 percent and KAM estimation with mean absolute error (MAE) less than 0.5 percent*body weight*height, the current threshold for accurately detecting knee osteoarthritis with KAM, illustrates the usefulness of our dataset for future research and clinical settings.

Related papers

• SAMRI-2: A Memory-based Model for Cartilage and Meniscus Segmentation in 3D MRIs of the Knee Joint [0.7879983966759583]
  This study introduces a deep learning (DL) method for cartilage and meniscus segmentation from 3D MRIs using memory-based VFMs. We trained four AI models-a CNN-based 3D-VNet, two automatic transformer-based models (SaMRI2D and SaMRI3D), and a transformer-based promptable memory-based VFM (SAMRI-2)-on 3D knee MRIs from 270 patients. SAMRI-2 model, trained with HSS, outperformed all other models, achieving an average improvement of 5 points, with a peak improvement of 12 points for tibial cartilage.
  arXiv  Detail & Related papers  (2025-02-14T21:18:01Z)
• Computer-Aided Osteoporosis Diagnosis Using Transfer Learning with Enhanced Features from Stacked Deep Learning Modules [6.398513051441461]
  Knee osteoporosis weakens the bone tissue in the knee joint, increasing fracture risk. Early detection through X-ray images enables timely intervention and improved patient outcomes. We propose a computer-aided diagnosis (CAD) system for knee osteoporosis, combining transfer learning with stacked feature enhancement deep learning blocks.
  arXiv  Detail & Related papers  (2024-12-12T14:59:10Z)
• Scaling Wearable Foundation Models [54.93979158708164]
  We investigate the scaling properties of sensor foundation models across compute, data, and model size. Using a dataset of up to 40 million hours of in-situ heart rate, heart rate variability, electrodermal activity, accelerometer, skin temperature, and altimeter per-minute data from over 165,000 people, we create LSM. Our results establish the scaling laws of LSM for tasks such as imputation, extrapolation, both across time and sensor modalities.
  arXiv  Detail & Related papers  (2024-10-17T15:08:21Z)
• Cascade learning in multi-task encoder-decoder networks for concurrent bone segmentation and glenohumeral joint assessment in shoulder CT scans [0.8974531206817744]
  This work introduces an innovative deep-learning framework processing shoulder CT scans. It features the semantic segmentation of the humerus and scapula, the 3D reconstruction of bone surfaces, and the identification of the glenohumeral joint region. The pipeline comprises two cascaded CNN architectures: 3D CEL-UNet for segmentation and 3D Arthro-Net for threefold classification.
  arXiv  Detail & Related papers  (2024-10-16T15:00:31Z)
• Automatic diagnosis of knee osteoarthritis severity using Swin transformer [55.01037422579516]
  Knee osteoarthritis (KOA) is a widespread condition that can cause chronic pain and stiffness in the knee joint. We propose an automated approach that employs the Swin Transformer to predict the severity of KOA.
  arXiv  Detail & Related papers  (2023-07-10T09:49:30Z)
• Strengths and Weaknesses of 3D Pose Estimation and Inertial Motion Capture System for Movement Therapy [0.0]
  3D pose estimation offers the opportunity for fast, non-invasive, and accurate motion analysis. We investigate the accuracy of the state-of-the-art 3D position estimation approach MeTrabs, compared to the established inertial sensor system MTw Awinda.
  arXiv  Detail & Related papers  (2023-06-01T20:35:06Z)
• Multimodal video and IMU kinematic dataset on daily life activities using affordable devices (VIDIMU) [0.0]
  The objective of the dataset is to pave the way towards affordable patient gross motor tracking solutions for daily life activities recognition and kinematic analysis. The novelty of dataset lies in: (i) the clinical relevance of the chosen movements, (ii) the combined utilization of affordable video and custom sensors, and (iii) the implementation of state-of-the-art tools for multimodal data processing of 3D body pose tracking and motion reconstruction.
  arXiv  Detail & Related papers  (2023-03-27T14:05:49Z)
• Pain level and pain-related behaviour classification using GRU-based sparsely-connected RNNs [61.080598804629375]
  People with chronic pain unconsciously adapt specific body movements to protect themselves from injury or additional pain. Because there is no dedicated benchmark database to analyse this correlation, we considered one of the specific circumstances that potentially influence a person's biometrics during daily activities. We proposed a sparsely-connected recurrent neural networks (s-RNNs) ensemble with the gated recurrent unit (GRU) that incorporates multiple autoencoders. We conducted several experiments which indicate that the proposed method outperforms the state-of-the-art approaches in classifying both pain level and pain-related behaviour.
  arXiv  Detail & Related papers  (2022-12-20T12:56:28Z)
• Inertial Hallucinations -- When Wearable Inertial Devices Start Seeing Things [82.15959827765325]
  We propose a novel approach to multimodal sensor fusion for Ambient Assisted Living (AAL) We address two major shortcomings of standard multimodal approaches, limited area coverage and reduced reliability. Our new framework fuses the concept of modality hallucination with triplet learning to train a model with different modalities to handle missing sensors at inference time.
  arXiv  Detail & Related papers  (2022-07-14T10:04:18Z)
• Continuous Decoding of Daily-Life Hand Movements from Forearm Muscle Activity for Enhanced Myoelectric Control of Hand Prostheses [78.120734120667]
  We introduce a novel method, based on a long short-term memory (LSTM) network, to continuously map forearm EMG activity onto hand kinematics. Ours is the first reported work on the prediction of hand kinematics that uses this challenging dataset. Our results suggest that the presented method is suitable for the generation of control signals for the independent and proportional actuation of the multiple DOFs of state-of-the-art hand prostheses.
  arXiv  Detail & Related papers  (2021-04-29T00:11:32Z)
• 4D Deep Learning for Multiple Sclerosis Lesion Activity Segmentation [49.32653090178743]
  We investigate whether extending this problem to full 4D deep learning using a history of MRI volumes can improve performance. We find that our proposed architecture outperforms previous approaches with a lesion-wise true positive rate of 0.84 at a lesion-wise false positive rate of 0.19.
  arXiv  Detail & Related papers  (2020-04-20T11:41:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.