Muscles in Time: Learning to Understand Human Motion by Simulating Muscle Activations

• URL: http://arxiv.org/abs/2411.00128v1
• Date: Thu, 31 Oct 2024 18:28:53 GMT
• Title: Muscles in Time: Learning to Understand Human Motion by Simulating Muscle Activations
• Authors: David Schneider, Simon Reiß, Marco Kugler, Alexander Jaus, Kunyu Peng, Susanne Sutschet, M. Saquib Sarfraz, Sven Matthiesen, Rainer Stiefelhagen,
• Abstract summary: Muscles in Time (MinT) is a large-scale synthetic muscle activation dataset. It contains over nine hours of simulation data covering 227 subjects and 402 simulated muscle strands. We show results on neural network-based muscle activation estimation from human pose sequences.
• Score: 64.98299559470503
• License:
• Abstract: Exploring the intricate dynamics between muscular and skeletal structures is pivotal for understanding human motion. This domain presents substantial challenges, primarily attributed to the intensive resources required for acquiring ground truth muscle activation data, resulting in a scarcity of datasets. In this work, we address this issue by establishing Muscles in Time (MinT), a large-scale synthetic muscle activation dataset. For the creation of MinT, we enriched existing motion capture datasets by incorporating muscle activation simulations derived from biomechanical human body models using the OpenSim platform, a common approach in biomechanics and human motion research. Starting from simple pose sequences, our pipeline enables us to extract detailed information about the timing of muscle activations within the human musculoskeletal system. Muscles in Time contains over nine hours of simulation data covering 227 subjects and 402 simulated muscle strands. We demonstrate the utility of this dataset by presenting results on neural network-based muscle activation estimation from human pose sequences with two different sequence-to-sequence architectures. Data and code are provided under https://simplexsigil.github.io/mint.

Related papers

• Scaling Wearable Foundation Models [54.93979158708164]
  We investigate the scaling properties of sensor foundation models across compute, data, and model size. Using a dataset of up to 40 million hours of in-situ heart rate, heart rate variability, electrodermal activity, accelerometer, skin temperature, and altimeter per-minute data from over 165,000 people, we create LSM. Our results establish the scaling laws of LSM for tasks such as imputation, extrapolation, both across time and sensor modalities.
  arXiv  Detail & Related papers  (2024-10-17T15:08:21Z)
• AddBiomechanics Dataset: Capturing the Physics of Human Motion at Scale [6.704034895950097]
  We present the AddBiomechanics dataset 1.0, which includes physically accurate human dynamics of 273 human subjects. We propose a benchmark for estimating human dynamics from motion using this dataset, and present several baseline results.
  arXiv  Detail & Related papers  (2024-05-16T16:57:43Z)
• MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints [50.61346764110482]
  We integrate a musculoskeletal system with a learnable parametric hand model, MANO, to create MS-MANO. This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories. We also propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron network.
  arXiv  Detail & Related papers  (2024-04-16T02:18:18Z)
• Scaling Up Dynamic Human-Scene Interaction Modeling [58.032368564071895]
  TRUMANS is the most comprehensive motion-captured HSI dataset currently available. It intricately captures whole-body human motions and part-level object dynamics. We devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length.
  arXiv  Detail & Related papers  (2024-03-13T15:45:04Z)
• The bionic neural network for external simulation of human locomotor system [2.6311880922890842]
  This paper proposes a physics-informed deep learning method based on musculoskeletal (MSK) modeling to predict joint motion and muscle forces. The method can effectively identify subject-specific MSK physiological parameters and the trained physics-informed forward-dynamics surrogate yields accurate motion and muscle forces predictions.
  arXiv  Detail & Related papers  (2023-09-11T23:02:56Z)
• Muscles in Action [22.482090207522358]
  We present a new dataset, Muscles in Action (MIA), to learn to incorporate muscle activity into human motion representations. We learn a bidirectional representation that predicts muscle activation from video, and conversely, reconstructs motion from muscle activation. Putting muscles into computer vision systems will enable richer models of virtual humans, with applications in sports, fitness, and AR/VR.
  arXiv  Detail & Related papers  (2022-12-05T16:47:09Z)
• Transformer Inertial Poser: Attention-based Real-time Human Motion Reconstruction from Sparse IMUs [79.72586714047199]
  We propose an attention-based deep learning method to reconstruct full-body motion from six IMU sensors in real-time. Our method achieves new state-of-the-art results both quantitatively and qualitatively, while being simple to implement and smaller in size.
  arXiv  Detail & Related papers  (2022-03-29T16:24:52Z)
• OstrichRL: A Musculoskeletal Ostrich Simulation to Study Bio-mechanical Locomotion [8.849771760994273]
  We release a 3D musculoskeletal simulation of an ostrich based on the MuJoCo simulator. The model is based on CT scans and dissections used to gather actual muscle data. We also provide a set of reinforcement learning tasks, including reference motion tracking and a reaching task with the neck.
  arXiv  Detail & Related papers  (2021-12-11T19:58:11Z)
• Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
  Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
  arXiv  Detail & Related papers  (2021-06-05T01:31:51Z)
• Reinforcement Learning of Musculoskeletal Control from Functional Simulations [3.94716580540538]
  In this work, a deep reinforcement learning (DRL) based inverse dynamics controller is trained to control muscle activations of a biomechanical model of the human shoulder. Results are presented for a single-axis motion control of shoulder abduction for the task of following randomly generated angular trajectories.
  arXiv  Detail & Related papers  (2020-07-13T20:20:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.