Muscle Vision: Real Time Keypoint Based Pose Classification of Physical Exercises

• URL: http://arxiv.org/abs/2203.12111v1
• Date: Wed, 23 Mar 2022 00:55:07 GMT
• Title: Muscle Vision: Real Time Keypoint Based Pose Classification of Physical Exercises
• Authors: Alex Moran, Bart Gebka, Joshua Goldshteyn, Autumn Beyer, Nathan Johnson, and Alexander Neuwirth
• Abstract summary: 3D human pose recognition extrapolated from video has advanced to the point of enabling real-time software applications. We propose a new machine learning pipeline and web interface that performs human pose recognition on a live video feed to detect when common exercises are performed and classify them accordingly.
• Score: 52.77024349608834
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in machine learning technology have enabled highly portable and performant models for many common tasks, especially in image recognition. One emerging field, 3D human pose recognition extrapolated from video, has now advanced to the point of enabling real-time software applications with robust enough output to support downstream machine learning tasks. In this work we propose a new machine learning pipeline and web interface that performs human pose recognition on a live video feed to detect when common exercises are performed and classify them accordingly. We present a model interface capable of webcam input with live display of classification results. Our main contributions include a keypoint and time series based lightweight approach for classifying a selected set of fitness exercises and a web-based software application for obtaining and visualizing the results in real time.

Related papers

• Is end-to-end learning enough for fitness activity recognition? [2.4273770300720012]
  We show that end-to-end learning can compete with state-of-the-art action recognition pipelines based on pose estimation. We also show that end-to-end learning can support temporally fine-grained tasks such as real-time repetition counting.
  arXiv  Detail & Related papers  (2023-05-14T16:00:03Z)
• Frozen CLIP Models are Efficient Video Learners [86.73871814176795]
  Video recognition has been dominated by the end-to-end learning paradigm. Recent advances in Contrastive Vision-Language Pre-training pave the way for a new route for visual recognition tasks. We present Efficient Video Learning -- an efficient framework for directly training high-quality video recognition models.
  arXiv  Detail & Related papers  (2022-08-06T17:38:25Z)
• A Spatio-Temporal Multilayer Perceptron for Gesture Recognition [70.34489104710366]
  We propose a multilayer state-weighted perceptron for gesture recognition in the context of autonomous vehicles. An evaluation of TCG and Drive&Act datasets is provided to showcase the promising performance of our approach. We deploy our model to our autonomous vehicle to show its real-time capability and stable execution.
  arXiv  Detail & Related papers  (2022-04-25T08:42:47Z)
• A Review of Deep Learning Techniques for Markerless Human Motion on Synthetic Datasets [0.0]
  Estimating human posture has recently gained increasing attention in the computer vision community. We present a model that can predict the skeleton of an animation based solely on 2D images. The implementation process uses DeepLabCut on its own dataset to perform many necessary steps.
  arXiv  Detail & Related papers  (2022-01-07T15:42:50Z)
• Which priors matter? Benchmarking models for learning latent dynamics [70.88999063639146]
  Several methods have proposed to integrate priors from classical mechanics into machine learning models. We take a sober look at the current capabilities of these models. We find that the use of continuous and time-reversible dynamics benefits models of all classes.
  arXiv  Detail & Related papers  (2021-11-09T23:48:21Z)
• SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild [62.450907796261646]
  Recognition of hand gestures can be performed directly from the stream of hand skeletons estimated by software. Despite the recent advancements in gesture and action recognition from skeletons, it is unclear how well the current state-of-the-art techniques can perform in a real-world scenario. This paper presents the results of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild contest.
  arXiv  Detail & Related papers  (2021-06-21T10:57:49Z)
• A Kinematic Bottleneck Approach For Pose Regression of Flexible Surgical Instruments directly from Images [17.32860829016479]
  We propose a self-supervised image-based method, exploiting, at training time only, the kinematic information provided by the robot. In order to avoid introducing time-consuming manual annotations, the problem is formulated as an auto-encoder. Validation of the method was performed on semi-synthetic, phantom and in-vivo datasets, obtained using a flexible robotized endoscope.
  arXiv  Detail & Related papers  (2021-02-28T18:41:18Z)
• Multi-Modal Learning of Keypoint Predictive Models for Visual Object Manipulation [6.853826783413853]
  Humans have impressive generalization capabilities when it comes to manipulating objects in novel environments. How to learn such body schemas for robots remains an open problem. We develop an self-supervised approach that can extend a robot's kinematic model when grasping an object from visual latent representations.
  arXiv  Detail & Related papers  (2020-11-08T01:04:59Z)
• Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery [84.73764603474413]
  We propose a novel online approach of multi-modal graph network (i.e., MRG-Net) to dynamically integrate visual and kinematics information. The effectiveness of our method is demonstrated with state-of-the-art results on the public JIGSAWS dataset.
  arXiv  Detail & Related papers  (2020-11-03T11:00:10Z)
• Complex Human Action Recognition in Live Videos Using Hybrid FR-DL Method [1.027974860479791]
  We address challenges of the preprocessing phase, by an automated selection of representative frames among the input sequences. We propose a hybrid technique using background subtraction and HOG, followed by application of a deep neural network and skeletal modelling method. We name our model as Feature Reduction & Deep Learning based action recognition method, or FR-DL in short.
  arXiv  Detail & Related papers  (2020-07-06T15:12:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.