An Examination of Wearable Sensors and Video Data Capture for Human Exercise Classification

• URL: http://arxiv.org/abs/2307.04516v1
• Date: Mon, 10 Jul 2023 12:24:04 GMT
• Title: An Examination of Wearable Sensors and Video Data Capture for Human Exercise Classification
• Authors: Ashish Singh and Antonio Bevilacqua and Timilehin B. Aderinola and Thach Le Nguyen and Darragh Whelan and Martin O'Reilly and Brian Caulfield and Georgiana Ifrim
• Abstract summary: We compare the performance of IMUs to a video-based approach for human exercise classification on two real-world datasets. We observe that an approach based on a single camera can outperform a single IMU by 10 percentage points on average. Our work opens up new and more realistic avenues for this application, where a video captured using a readily available smartphone camera, combined with a single sensor, can be used for effective human exercise classification.
• Score: 9.674125829493214
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Wearable sensors such as Inertial Measurement Units (IMUs) are often used to assess the performance of human exercise. Common approaches use handcrafted features based on domain expertise or automatically extracted features using time series analysis. Multiple sensors are required to achieve high classification accuracy, which is not very practical. These sensors require calibration and synchronization and may lead to discomfort over longer time periods. Recent work utilizing computer vision techniques has shown similar performance using video, without the need for manual feature engineering, and avoiding some pitfalls such as sensor calibration and placement on the body. In this paper, we compare the performance of IMUs to a video-based approach for human exercise classification on two real-world datasets consisting of Military Press and Rowing exercises. We compare the performance using a single camera that captures video in the frontal view versus using 5 IMUs placed on different parts of the body. We observe that an approach based on a single camera can outperform a single IMU by 10 percentage points on average. Additionally, a minimum of 3 IMUs are required to outperform a single camera. We observe that working with the raw data using multivariate time series classifiers outperforms traditional approaches based on handcrafted or automatically extracted features. Finally, we show that an ensemble model combining the data from a single camera with a single IMU outperforms either data modality. Our work opens up new and more realistic avenues for this application, where a video captured using a readily available smartphone camera, combined with a single sensor, can be used for effective human exercise classification.

Related papers

• MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition [2.7532797256542403]
  Human Activity Recognition is a longstanding problem in AI with applications in a broad range of areas. We show how we can improve HAR performance across different modalities using multimodal contrastive pretraining.
  arXiv  Detail & Related papers  (2024-06-06T08:42:36Z)
• HMP: Hand Motion Priors for Pose and Shape Estimation from Video [52.39020275278984]
  We develop a generative motion prior specific for hands, trained on the AMASS dataset which features diverse and high-quality hand motions. Our integration of a robust motion prior significantly enhances performance, especially in occluded scenarios. We demonstrate our method's efficacy via qualitative and quantitative evaluations on the HO3D and DexYCB datasets.
  arXiv  Detail & Related papers  (2023-12-27T22:35:33Z)
• Efficient Adaptive Human-Object Interaction Detection with Concept-guided Memory [64.11870454160614]
  We propose an efficient Adaptive HOI Detector with Concept-guided Memory (ADA-CM) ADA-CM has two operating modes. The first mode makes it tunable without learning new parameters in a training-free paradigm. Our proposed method achieves competitive results with state-of-the-art on the HICO-DET and V-COCO datasets with much less training time.
  arXiv  Detail & Related papers  (2023-09-07T13:10:06Z)
• Fast and Robust Video-Based Exercise Classification via Body Pose Tracking and Scalable Multivariate Time Series Classifiers [13.561233730881279]
  We present the application of classifying S&C exercises using video. We propose an approach named BodyMTS to turn video into time series by employing body pose tracking. We show that BodyMTS achieves an average accuracy of 87%, which is significantly higher than the accuracy of human domain experts.
  arXiv  Detail & Related papers  (2022-10-02T13:03:38Z)
• Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition [56.91538445510214]
  We present a learning algorithm for human activity recognition in videos. Our approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras. We conduct extensive experiments on the UAV Human dataset and the NEC Drone dataset.
  arXiv  Detail & Related papers  (2022-09-15T22:16:52Z)
• Muscle Vision: Real Time Keypoint Based Pose Classification of Physical Exercises [52.77024349608834]
  3D human pose recognition extrapolated from video has advanced to the point of enabling real-time software applications. We propose a new machine learning pipeline and web interface that performs human pose recognition on a live video feed to detect when common exercises are performed and classify them accordingly.
  arXiv  Detail & Related papers  (2022-03-23T00:55:07Z)
• Human Activity Recognition models using Limited Consumer Device Sensors and Machine Learning [0.0]
  Human activity recognition has grown in popularity with its increase of applications within daily lifestyles and medical environments. This paper presents the findings of different models that are limited to train using sensor data from smartphones and smartwatches. Results show promise for models trained strictly using limited sensor data collected from only smartphones and smartwatches coupled with traditional machine learning concepts and algorithms.
  arXiv  Detail & Related papers  (2022-01-21T06:54:05Z)
• Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors [71.29186299435423]
  We introduce (HPS) Human POSEitioning System, a method to recover the full 3D pose of a human registered with a 3D scan of the surrounding environment. We show that our optimization-based integration exploits the benefits of the two, resulting in pose accuracy free of drift. HPS could be used for VR/AR applications where humans interact with the scene without requiring direct line of sight with an external camera.
  arXiv  Detail & Related papers  (2021-03-31T17:58:31Z)
• One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video [71.43912903508765]
  MDAL is a dynamic online adaptive learning scheme for instrument segmentation in robot-assisted surgery. It learns the general knowledge of instruments and the fast adaptation ability through the video-specific meta-learning paradigm. It outperforms other state-of-the-art methods on two datasets.
  arXiv  Detail & Related papers  (2021-03-24T05:02:18Z)
• Yet it moves: Learning from Generic Motions to Generate IMU data from YouTube videos [5.008235182488304]
  We show how we can train a regression model on generic motions for both accelerometer and gyro signals to generate synthetic IMU data. We demonstrate that systems trained on simulated data generated by our regression model can come to within around 10% of the mean F1 score of a system trained on real sensor data.
  arXiv  Detail & Related papers  (2020-11-23T18:16:46Z)
• Decoupled Appearance and Motion Learning for Efficient Anomaly Detection in Surveillance Video [9.80717374118619]
  We propose a new neural network architecture that learns the normal behavior in a purely unsupervised fashion. Our model can process 16 to 45 times more frames per second than related approaches.
  arXiv  Detail & Related papers  (2020-11-10T11:40:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.