MotionAug: Augmentation with Physical Correction for Human Motion Prediction

• URL: http://arxiv.org/abs/2203.09116v4
• Date: Thu, 17 Aug 2023 07:00:58 GMT
• Title: MotionAug: Augmentation with Physical Correction for Human Motion Prediction
• Authors: Takahiro Maeda and Norimichi Ukita
• Abstract summary: This paper presents a motion data augmentation scheme incorporating motion synthesis encouraging diversity and motion correction imposing physical plausibility. Our method outperforms previous noise-based motion augmentation methods by a large margin on both Recurrent Neural Network-based and Graph Convolutional Network-based human motion prediction models.
• Score: 19.240717471864723
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a motion data augmentation scheme incorporating motion synthesis encouraging diversity and motion correction imposing physical plausibility. This motion synthesis consists of our modified Variational AutoEncoder (VAE) and Inverse Kinematics (IK). In this VAE, our proposed sampling-near-samples method generates various valid motions even with insufficient training motion data. Our IK-based motion synthesis method allows us to generate a variety of motions semi-automatically. Since these two schemes generate unrealistic artifacts in the synthesized motions, our motion correction rectifies them. This motion correction scheme consists of imitation learning with physics simulation and subsequent motion debiasing. For this imitation learning, we propose the PD-residual force that significantly accelerates the training process. Furthermore, our motion debiasing successfully offsets the motion bias induced by imitation learning to maximize the effect of augmentation. As a result, our method outperforms previous noise-based motion augmentation methods by a large margin on both Recurrent Neural Network-based and Graph Convolutional Network-based human motion prediction models. The code is available at https://github.com/meaten/MotionAug.

Related papers

• Optimal-state Dynamics Estimation for Physics-based Human Motion Capture from Videos [6.093379844890164]
  We propose a novel method to selectively incorporate the physics models with the kinematics observations in an online setting. A recurrent neural network is introduced to realize a Kalman filter that attentively balances the kinematics input and simulated motion. The proposed approach excels in the physics-based human pose estimation task and demonstrates the physical plausibility of the predictive dynamics.
  arXiv  Detail & Related papers  (2024-10-10T10:24:59Z)
• Spectral Motion Alignment for Video Motion Transfer using Diffusion Models [54.32923808964701]
  Spectral Motion Alignment (SMA) is a framework that refines and aligns motion vectors using Fourier and wavelet transforms. SMA learns motion patterns by incorporating frequency-domain regularization, facilitating the learning of whole-frame global motion dynamics. Extensive experiments demonstrate SMA's efficacy in improving motion transfer while maintaining computational efficiency and compatibility across various video customization frameworks.
  arXiv  Detail & Related papers  (2024-03-22T14:47:18Z)
• DEMOS: Dynamic Environment Motion Synthesis in 3D Scenes via Local Spherical-BEV Perception [54.02566476357383]
  We propose the first Dynamic Environment MOtion Synthesis framework (DEMOS) to predict future motion instantly according to the current scene. We then use it to dynamically update the latent motion for final motion synthesis. The results show our method outperforms previous works significantly and has great performance in handling dynamic environments.
  arXiv  Detail & Related papers  (2024-03-04T05:38:16Z)
• Motion Flow Matching for Human Motion Synthesis and Editing [75.13665467944314]
  We propose emphMotion Flow Matching, a novel generative model for human motion generation featuring efficient sampling and effectiveness in motion editing applications. Our method reduces the sampling complexity from thousand steps in previous diffusion models to just ten steps, while achieving comparable performance in text-to-motion and action-to-motion generation benchmarks.
  arXiv  Detail & Related papers  (2023-12-14T12:57:35Z)
• Motion Matters: Neural Motion Transfer for Better Camera Physiological Measurement [25.27559386977351]
  Body motion is one of the most significant sources of noise when attempting to recover the subtle cardiac pulse from a video. We adapt a neural video synthesis approach to augment videos for the task of remote photoplethys. We demonstrate a 47% improvement over existing inter-dataset results using various state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-03-21T17:51:23Z)
• HumanMAC: Masked Motion Completion for Human Motion Prediction [62.279925754717674]
  Human motion prediction is a classical problem in computer vision and computer graphics. Previous effects achieve great empirical performance based on an encoding-decoding style. In this paper, we propose a novel framework from a new perspective.
  arXiv  Detail & Related papers  (2023-02-07T18:34:59Z)
• PhysDiff: Physics-Guided Human Motion Diffusion Model [101.1823574561535]
  Existing motion diffusion models largely disregard the laws of physics in the diffusion process. PhysDiff incorporates physical constraints into the diffusion process. Our approach achieves state-of-the-art motion quality and improves physical plausibility drastically.
  arXiv  Detail & Related papers  (2022-12-05T18:59:52Z)
• Skeleton2Humanoid: Animating Simulated Characters for Physically-plausible Motion In-betweening [59.88594294676711]
  Modern deep learning based motion synthesis approaches barely consider the physical plausibility of synthesized motions. We propose a system Skeleton2Humanoid'' which performs physics-oriented motion correction at test time. Experiments on the challenging LaFAN1 dataset show our system can outperform prior methods significantly in terms of both physical plausibility and accuracy.
  arXiv  Detail & Related papers  (2022-10-09T16:15:34Z)
• Physics-based Human Motion Estimation and Synthesis from Videos [0.0]
  We propose a framework for training generative models of physically plausible human motion directly from monocular RGB videos. At the core of our method is a novel optimization formulation that corrects imperfect image-based pose estimations. Results show that our physically-corrected motions significantly outperform prior work on pose estimation.
  arXiv  Detail & Related papers  (2021-09-21T01:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.