Towards Activated Muscle Group Estimation in the Wild

• URL: http://arxiv.org/abs/2303.00952v5
• Date: Tue, 6 Aug 2024 02:39:05 GMT
• Title: Towards Activated Muscle Group Estimation in the Wild
• Abstract summary: We tackle the new task of video-based Activated Muscle Group Estimation (AMGE) aiming at identifying active muscle regions during physical activity in the wild. We provide the MuscleMap dataset featuring >15K video clips with 135 different activities and 20 labeled muscle groups. The proposed MuscleMap dataset is constructed with YouTube videos, specifically targeting High-Intensity Interval Training (HIIT) physical exercise in the wild.
• Score: 44.7050439903772
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we tackle the new task of video-based Activated Muscle Group Estimation (AMGE) aiming at identifying active muscle regions during physical activity in the wild. To this intent, we provide the MuscleMap dataset featuring >15K video clips with 135 different activities and 20 labeled muscle groups. This dataset opens the vistas to multiple video-based applications in sports and rehabilitation medicine under flexible environment constraints. The proposed MuscleMap dataset is constructed with YouTube videos, specifically targeting High-Intensity Interval Training (HIIT) physical exercise in the wild. To make the AMGE model applicable in real-life situations, it is crucial to ensure that the model can generalize well to numerous types of physical activities not present during training and involving new combinations of activated muscles. To achieve this, our benchmark also covers an evaluation setting where the model is exposed to activity types excluded from the training set. Our experiments reveal that the generalizability of existing architectures adapted for the AMGE task remains a challenge. Therefore, we also propose a new approach, TransM3E, which employs a multi-modality feature fusion mechanism between both the video transformer model and the skeleton-based graph convolution model with novel cross-modal knowledge distillation executed on multi-classification tokens. The proposed method surpasses all popular video classification models when dealing with both, previously seen and new types of physical activities. The database and code can be found at https://github.com/KPeng9510/MuscleMap.

Related papers

• Towards Embodied AI with MuscleMimic: Unlocking full-body musculoskeletal motor learning at scale [38.15316304510102]
  MuscleMimic is an open-source framework for motion imitation learning with physiologically realistic, muscle-driven models.<n>By lowering the computational and data barriers to musculoskeletal simulation, MuscleMimic enables systematic model validation across diverse dynamic movements.
  arXiv  Detail & Related papers  (2026-03-26T15:18:37Z)
• Multimodal Foundation Model for Cross-Modal Retrieval and Activity Recognition Tasks [3.1976901430982063]
  We propose a foundational model integrating four modalities: third-person video, motion capture, IMU, and text.<n>By incorporating third-person video and motion capture data, the model enables a detailed and multidimensional understanding of human activity.
  arXiv  Detail & Related papers  (2025-05-29T01:47:43Z)
• Video-based Exercise Classification and Activated Muscle Group Prediction with Hybrid X3D-SlowFast Network [2.9312156642007303]
  This paper introduces a simple yet effective strategy for exercise classification and muscle group activation prediction (MGAP) Previous research in the field is mostly dominated by the reliance on mounted sensors and a limited scope of exercises. Our research employs a video-based deep learning framework that encompasses a broad spectrum of exercises and muscle groups.
  arXiv  Detail & Related papers  (2024-06-10T18:05:02Z)
• AICL: Action In-Context Learning for Video Diffusion Model [124.39948693332552]
  We propose AICL, which empowers the generative model with the ability to understand action information in reference videos. Extensive experiments demonstrate that AICL effectively captures the action and achieves state-of-the-art generation performance.
  arXiv  Detail & Related papers  (2024-03-18T07:41:19Z)
• Muscles in Action [22.482090207522358]
  We present a new dataset, Muscles in Action (MIA), to learn to incorporate muscle activity into human motion representations. We learn a bidirectional representation that predicts muscle activation from video, and conversely, reconstructs motion from muscle activation. Putting muscles into computer vision systems will enable richer models of virtual humans, with applications in sports, fitness, and AR/VR.
  arXiv  Detail & Related papers  (2022-12-05T16:47:09Z)
• REST: REtrieve & Self-Train for generative action recognition [54.90704746573636]
  We propose to adapt a pre-trained generative Vision & Language (V&L) Foundation Model for video/action recognition. We show that direct fine-tuning of a generative model to produce action classes suffers from severe overfitting. We introduce REST, a training framework consisting of two key components.
  arXiv  Detail & Related papers  (2022-09-29T17:57:01Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• Learning Local Recurrent Models for Human Mesh Recovery [50.85467243778406]
  We present a new method for video mesh recovery that divides the human mesh into several local parts following the standard skeletal model. We then model the dynamics of each local part with separate recurrent models, with each model conditioned appropriately based on the known kinematic structure of the human body. This results in a structure-informed local recurrent learning architecture that can be trained in an end-to-end fashion with available annotations.
  arXiv  Detail & Related papers  (2021-07-27T14:30:33Z)
• EAN: Event Adaptive Network for Enhanced Action Recognition [66.81780707955852]
  We propose a unified action recognition framework to investigate the dynamic nature of video content. First, when extracting local cues, we generate the spatial-temporal kernels of dynamic-scale to adaptively fit the diverse events. Second, to accurately aggregate these cues into a global video representation, we propose to mine the interactions only among a few selected foreground objects by a Transformer.
  arXiv  Detail & Related papers  (2021-07-22T15:57:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.