Learning to Estimate External Forces of Human Motion in Video

• URL: http://arxiv.org/abs/2207.05845v1
• Date: Tue, 12 Jul 2022 21:20:47 GMT
• Title: Learning to Estimate External Forces of Human Motion in Video
• Authors: Nathan Louis, Tylan N. Templin, Travis D. Eliason, Daniel P. Nicolella, and Jason J. Corso
• Abstract summary: Ground reaction forces (GRFs) are exerted by the human body during certain movements. Standard practice uses physical markers paired with force plates in a controlled environment. We propose GRF inference from video.
• Score: 22.481658922173906
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Analyzing sports performance or preventing injuries requires capturing ground reaction forces (GRFs) exerted by the human body during certain movements. Standard practice uses physical markers paired with force plates in a controlled environment, but this is marred by high costs, lengthy implementation time, and variance in repeat experiments; hence, we propose GRF inference from video. While recent work has used LSTMs to estimate GRFs from 2D viewpoints, these can be limited in their modeling and representation capacity. First, we propose using a transformer architecture to tackle the GRF from video task, being the first to do so. Then we introduce a new loss to minimize high impact peaks in regressed curves. We also show that pre-training and multi-task learning on 2D-to-3D human pose estimation improves generalization to unseen motions. And pre-training on this different task provides good initial weights when finetuning on smaller (rarer) GRF datasets. We evaluate on LAAS Parkour and a newly collected ForcePose dataset; we show up to 19% decrease in error compared to prior approaches.

Related papers

• ExPLoRA: Parameter-Efficient Extended Pre-Training to Adapt Vision Transformers under Domain Shifts [52.1635661239108]
  We introduce ExPLoRA, a highly effective technique to improve transfer learning of pre-trained vision transformers (ViTs) under domain shifts. Our experiments demonstrate state-of-the-art results on satellite imagery, even outperforming fully pre-training and fine-tuning ViTs.
  arXiv  Detail & Related papers  (2024-06-16T15:14:56Z)
• A Reinforcement Learning Based Controller to Minimize Forces on the Crutches of a Lower-Limb Exoskeleton [1.4680035572775536]
  We use deep reinforcement learning to develop a controller that minimizes ground reaction forces (GRF) on crutches. We formulate a reward function to encourage the forward displacement of a human-exoskeleton system. We empirically show that our learning model can generate joint torques based on the joint angle, velocities, and the GRF on the feet and crutch tips.
  arXiv  Detail & Related papers  (2024-01-31T19:20:56Z)
• Gait Cycle-Inspired Learning Strategy for Continuous Prediction of Knee Joint Trajectory from sEMG [24.608475386117426]
  This paper proposes a model integrating two gait cycle-inspired learning strategies to mitigate the challenge for predicting human knee joint trajectory. By learning through separate network entities, the model manages to capture both the common and personalized gait features. Experimental results indicate that our model could predict knee angles with the average root mean square error (RMSE) of 3.03(0.49) degrees and 50ms ahead of time.
  arXiv  Detail & Related papers  (2023-07-25T02:23:58Z)
• GLA-GCN: Global-local Adaptive Graph Convolutional Network for 3D Human Pose Estimation from Monocular Video [33.801470193414325]
  This work focuses on improving 3D human pose lifting via ground truth data. Global-local Adaptive Graph Convolutional Network (GLA-GCN) is proposed in this work. Our GLA-GCN implemented with ground truth 2D poses significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-07-12T00:13:04Z)
• Decomposed Human Motion Prior for Video Pose Estimation via Adversarial Training [7.861513525154702]
  We propose to decompose holistic motion prior to joint motion prior, making it easier for neural networks to learn from prior knowledge. We also utilize a novel regularization loss to balance accuracy and smoothness introduced by motion prior. Our method achieves 9% lower PA-MPJPE and 29% lower acceleration error than previous methods tested on 3DPW.
  arXiv  Detail & Related papers  (2023-05-30T04:53:34Z)
• T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations [34.61255243742796]
  We show that a simple CNN-based VQ-VAE with commonly used training recipes (EMA and Code Reset) allows us to obtain high-quality discrete representations. Despite its simplicity, our T2M-GPT shows better performance than competitive approaches.
  arXiv  Detail & Related papers  (2023-01-15T09:34:42Z)
• STAR: Sparse Transformer-based Action Recognition [61.490243467748314]
  This work proposes a novel skeleton-based human action recognition model with sparse attention on the spatial dimension and segmented linear attention on the temporal dimension of data. Experiments show that our model can achieve comparable performance while utilizing much less trainable parameters and achieve high speed in training and inference.
  arXiv  Detail & Related papers  (2021-07-15T02:53:11Z)
• Synthetic Training for Monocular Human Mesh Recovery [100.38109761268639]
  This paper aims to estimate 3D mesh of multiple body parts with large-scale differences from a single RGB image. The main challenge is lacking training data that have complete 3D annotations of all body parts in 2D images. We propose a depth-to-scale (D2S) projection to incorporate the depth difference into the projection function to derive per-joint scale variants.
  arXiv  Detail & Related papers  (2020-10-27T03:31:35Z)
• Motion Guided 3D Pose Estimation from Videos [81.14443206968444]
  We propose a new loss function, called motion loss, for the problem of monocular 3D Human pose estimation from 2D pose. In computing motion loss, a simple yet effective representation for keypoint motion, called pairwise motion encoding, is introduced. We design a new graph convolutional network architecture, U-shaped GCN (UGCN), which captures both short-term and long-term motion information.
  arXiv  Detail & Related papers  (2020-04-29T06:59:30Z)
• Dense Regression Network for Video Grounding [97.57178850020327]
  We use the distances between the frame within the ground truth and the starting (ending) frame as dense supervisions to improve the video grounding accuracy. Specifically, we design a novel dense regression network (DRN) to regress the distances from each frame to the starting (ending) frame of the video segment. We also propose a simple but effective IoU regression head module to explicitly consider the localization quality of the grounding results.
  arXiv  Detail & Related papers  (2020-04-07T17:15:37Z)
• Human Motion Transfer from Poses in the Wild [61.6016458288803]
  We tackle the problem of human motion transfer, where we synthesize novel motion video for a target person that imitates the movement from a reference video. It is a video-to-video translation task in which the estimated poses are used to bridge two domains. We introduce a novel pose-to-video translation framework for generating high-quality videos that are temporally coherent even for in-the-wild pose sequences unseen during training.
  arXiv  Detail & Related papers  (2020-04-07T05:59:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.